Hi Duncan, Thanks for your help.
I am at a loss as to what is causing this process to stop then. I would not expect the Cassandra process to finish until my code calls Process#destroy, but it seems to non-deterministically stop much earlier sometimes. FWIW I have seen failures on another machine this morning which also look orderly. These nodes never even get to the point where they announce they are listening for CQL clients. If anyone has any ideas on what to look for, I would really appreciate it. I will try turning logging up to DEBUG and see if that produces any useful errors. Best regards, Clint On Wed, Aug 6, 2014 at 1:11 AM, Duncan Sands <duncan.sa...@gmail.com> wrote: > Hi Clint, > > >> INFO [StorageServiceShutdownHook] 2014-08-05 19:14:51,903 >> ThriftServer.java (line 141) Stop listening to thrift clients >> INFO [StorageServiceShutdownHook] 2014-08-05 19:14:51,920 Server.java >> (line 182) Stop listening for CQL clients >> INFO [StorageServiceShutdownHook] 2014-08-05 19:14:51,930 >> Gossiper.java (line 1279) Announcing shutdown >> INFO [StorageServiceShutdownHook] 2014-08-05 19:14:53,930 >> MessagingService.java (line 683) Waiting for messaging service to >> quiesce >> INFO [ACCEPT-/127.0.0.10] 2014-08-05 19:14:53,931 >> MessagingService.java (line 923) MessagingService has terminated the >> accept() thread >> >> Does anyone have any ideas about how to debug this? Looking around on >> google I found some threads suggesting that this could occur from an >> OOM error >> (http://stackoverflow.com/questions/23755040/cassandra-exits-with-no-errors). > > > this doesn't look like an OOM to me. If the kernel OOM kills Cassandra then > Cassandra instantly vaporizes, and there will be nothing in the Cassandra > logs (you will find information about the OOM in the system logs though, eg > in dmesg). In the log snippet above you see an orderly shutdown, this is > completely different to the instant OOM kill. > > Ciao, Duncan.
