FYI, when I increase the test memory to 1G to resemble the default setting in cassandra the OOM doesn't happen, so that's a non-issue.
On Sat, Sep 18, 2010 at 9:02 PM, Ran Tavory <ran...@gmail.com> wrote: > I haven't tried the python tests. What's also interesting is that if I > run a single java test case then there's no oom. Only when I run the > entire class then I get it. Also if I ads random sleeps then there is > no oom but other more interesting errors surface. > > On Saturday, September 18, 2010, Nate McCall <n...@riptano.com> wrote: > > Ran, have you tried to reproduce this using cassandra's python system > > tests? Those have coverage of the system methods as well. > > > > On 9/18/10, Nate McCall <n...@riptano.com> wrote: > >> An additional point - we just added the system_* methods and this test > >> class does an add, rename then drop for both CF and keyspace. > >> > >> On 9/18/10, Ran Tavory <ran...@gmail.com> wrote: > >>> I started seeing OOM when running hector's unit-tests on 0.7.0. The OOM > >>> is > >>> in cassandra's code so either there had been recent changes within > >>> cassandra > >>> or it's just that we added new coverage which causes this OOM. > >>> > >>> On my mac it's 100% repro but other hector devs have reported they > don't > >>> get > >>> this OOM. Moreover, it I place a Thread.sleep(5000) in one of the tests > >>> the > >>> OOM goes away (but other errors are still present see more below) > >>> > >>> To reproduce the error you may checkout hector from > >>> http://github.com/rantav/hector, switch to the 0.7.0 branch and run: > >>> > >>> $ MAVEN_OPTS="-Xmx1G -Xms256M -XX:+UseParNewGC -XX:+UseConcMarkSweepGC > >>> -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 > >>> -XX:MaxTenuringThreshold=1" *mvn test -Dtest=CassandraClusterTest* > >>> > >>> (I'm using the default memory settings from 0.7.0) > >>> > >>> The test class source code is here > >>> > http://github.com/rantav/hector/blob/0.7.0/src/test/java/me/prettyprint/cassandra/service/CassandraClusterTest.java > >>> > >>> This test basically adds, removes, renames keyspaces etc. > >>> > >>> Here's the OOM log: > >>> > >>> 18:07:04,197 INFO ColumnFamilyStore:734 - Enqueuing flush of > >>> memtable-sch...@72177013(3455 bytes, 3 operations) > >>> 18:07:04,487 ERROR AbstractCassandraDaemon:88 - Fatal exception in > thread > >>> Thread[FLUSH-WRITER-POOL:1,5,main] > >>> java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: > Java > >>> heap space > >>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) > >>> at java.util.concurrent.FutureTask.get(FutureTask.java:83) > >>> at > >>> > org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:87) > >>> at > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888) > >>> at > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > >>> at java.lang.Thread.run(Thread.java:637) > >>> Caused by: java.lang.OutOfMemoryError: Java heap space > >>> at > >>> > org.apache.cassandra.io.util.BufferedRandomAccessFile.init(BufferedRandomAccessFile.java:151) > >>> at > >>> > org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:144) > >>> at > >>> > org.apache.cassandra.io.sstable.SSTableWriter.<init>(SSTableWriter.java:61) > >>> at > >>> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:151) > >>> at org.apache.cassandra.db.Memtable.access$000(Memtable.java:44) > >>> at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:168) > >>> at > >>> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) > >>> at > >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > >>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > >>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) > >>> at > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > >>> ... 2 more > >>> 18:07:04,491 INFO Memtable:150 - Writing memtable-sch...@72177013 > (3455 > >>> bytes, 3 operations) > >>> 18:07:04,812 INFO CompactionManager:323 - Compacted to > >>> > /Users/ran/dev/cassandra/hector/tmp/var/lib/cassandra/data/system/Schema-tmp-e-6-Data.db. > >>> 15,707 to 15,173 (~96% of original) bytes for 5 keys > > -- > /Ran > -- /Ran
