Then you'll want to use MAT to analyze the dump the JVM gave you of the heap at OOM time. (http://www.eclipse.org/mat/)
On Tue, Jul 12, 2011 at 3:22 PM, Anurag Gujral <anurag.guj...@gmail.com> wrote: > Hi Jonathan, > Thanks for your mail. But no-one of the things > mentioned in the link pertains to OOM error I we are seeing. > thanks > Anurag > > On Tue, Jul 12, 2011 at 10:42 AM, Jonathan Ellis <jbel...@gmail.com> wrote: >> >> Have you seen >> http://www.datastax.com/docs/0.8/troubleshooting/index#nodes-are-dying-with-oom-errors >> ? >> >> On Mon, Jul 11, 2011 at 1:55 PM, Anurag Gujral <anurag.guj...@gmail.com> >> wrote: >> > Hi All, >> > I am getting following error from cassandra: >> > ERROR [ReadStage:23] 2011-07-10 17:19:18,300 >> > DebuggableThreadPoolExecutor.java (line 103) Error in ThreadPoolExecutor >> > java.lang.OutOfMemoryError: Java heap space >> > at >> > >> > org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterSerializer.java:49) >> > at >> > >> > org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterSerializer.java:30) >> > at >> > >> > org.apache.cassandra.io.sstable.IndexHelper.defreezeBloomFilter(IndexHelper.java:117) >> > at >> > >> > org.apache.cassandra.io.sstable.IndexHelper.defreezeBloomFilter(IndexHelper.java:94) >> > at >> > >> > org.apache.cassandra.db.columniterator.SSTableNamesIterator.read(SSTableNamesIterator.java:107) >> > at >> > >> > org.apache.cassandra.db.columniterator.SSTableNamesIterator.<init>(SSTableNamesIterator.java:72) >> > at >> > >> > org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:59) >> > at >> > >> > org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1311) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1203) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1131) >> > at org.apache.cassandra.db.Table.getRow(Table.java:333) >> > at >> > >> > org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:60) >> > at >> > org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:69) >> > at >> > >> > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72) >> > at >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) >> > at >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) >> > at java.lang.Thread.run(Thread.java:636) >> > INFO [ScheduledTasks:1] 2011-07-10 17:19:18,306 StatusLogger.java (line >> > 66) >> > RequestResponseStage 0 0 >> > ERROR [ReadStage:23] 2011-07-10 17:19:18,306 >> > AbstractCassandraDaemon.java >> > (line 114) Fatal exception in thread Thread[ReadStage:23,5,main] >> > java.lang.OutOfMemoryError: Java heap space >> > at >> > >> > org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterSerializer.java:49) >> > at >> > >> > org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterSerializer.java:30) >> > at >> > >> > org.apache.cassandra.io.sstable.IndexHelper.defreezeBloomFilter(IndexHelper.java:117) >> > at >> > >> > org.apache.cassandra.io.sstable.IndexHelper.defreezeBloomFilter(IndexHelper.java:94) >> > at >> > >> > org.apache.cassandra.db.columniterator.SSTableNamesIterator.read(SSTableNamesIterator.java:107) >> > at >> > >> > org.apache.cassandra.db.columniterator.SSTableNamesIterator.<init>(SSTableNamesIterator.java:72) >> > at >> > >> > org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:59) >> > at >> > >> > org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1311) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1203) >> > at >> > >> > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1131) >> > >> > >> > Can someone please help debug this? The maximum heap size is 28G . >> > >> > I am not sure why cassandra is giving Out of memory error here. >> > >> > Thanks >> > Anurag >> > >> >> >> >> -- >> Jonathan Ellis >> Project Chair, Apache Cassandra >> co-founder of DataStax, the source for professional Cassandra support >> http://www.datastax.com > > -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
