Re: Ran into an issue where Cassandra Crashed when running out of heap space

2010-07-20 Thread Ryan King
On Tue, Jul 20, 2010 at 1:28 PM, Peter Schuller wrote: >> Attaching Jconsole shows that there is a growth of memory and weird >> spikes. Unfortunately I did not take a screen shot of the growth of >> the spike over time. I'll do that when it occurs again. > > Note that expected behavior for CMS is

Re: Ran into an issue where Cassandra Crashed when running out of heap space

2010-07-20 Thread Peter Schuller
> Attaching Jconsole shows that there is a growth of memory and weird > spikes. Unfortunately I did not take a screen shot of the growth of > the spike over time. I'll do that when it occurs again. Note that expected behavior for CMS is to have lots of small ups and downs as a result of young gene

Re: Ran into an issue where Cassandra Crashed when running out of heap space

2010-07-20 Thread Dathan Pattishall
The storage structure is rather simple. For every 1 key there is 1 column and a timestamp for that column. We don't enable pulling a huge amount of data and all other nodes are up servicing the same request. I suspect there may be another problem with Memory management inside Cassandra. Attac

Re: Ran into an issue where Cassandra Crashed when running out of heap space

2010-07-20 Thread Tristan Seligmann
On Tue, Jul 20, 2010 at 9:09 PM, Peter Schuller wrote: >> CassandraDaemon.java (line 83) Uncaught exception in thread >> Thread[pool-1-thread-37895,5,main] >> java.lang.OutOfMemoryError: Java heap space >>     at >> org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.jav

Re: Ran into an issue where Cassandra Crashed when running out of heap space

2010-07-20 Thread Peter Schuller
> CassandraDaemon.java (line 83) Uncaught exception in thread > Thread[pool-1-thread-37895,5,main] > java.lang.OutOfMemoryError: Java heap space >     at > org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:296) >     at > org.apache.thrift.protocol.TBinaryProt

Ran into an issue where Cassandra Crashed when running out of heap space

2010-07-20 Thread Dathan Pattishall
 INFO [HINTED-HANDOFF-POOL:1] 2010-07-20 15:10:43,721 HintedHandOffManager.java (line 210) Finished hinted handoff of 0 rows to endpoint /10.129.28.23 ERROR [pool-1-thread-37895] 2010-07-20 15:10:51,622 CassandraDaemon.java (line 83) Uncaught exception in thread Thread[pool-1-thread-37895,5,main] j