Swappiness is set to 60 though cassandra recommendations are to turn swap completely off(we still have not done that as far as I know though) and sure enough linux killed it.
Dean From: Juan Manuel Formoso <jform...@gmail.com<mailto:jform...@gmail.com>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Date: Wednesday, September 18, 2013 6:57 AM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: Re: cassandra just gone..no heap dump, no log info This shouldn't happen if you have swap active in the server On Wednesday, September 18, 2013, Franc Carter wrote: A random guess - possibly an OOM (Out of Memory) where Linux will kill a process to recover memory when it is desperately low on memory. Have a look in either your syslog output of the output of dmesg cheers On Wed, Sep 18, 2013 at 10:21 PM, Hiller, Dean <dean.hil...@nrel.gov<javascript:_e({},%20'cvml',%20'dean.hil...@nrel.gov');>> wrote: Anyone know how to debug cassandra processes just exiting? There is no info in the cassandra logs and there is no heap dump file(which in the past has shown up in /opt/cassandra/bin directory for me). This occurs when running a map/reduce job that put severe load on the system. The logs look completely fine. I find it odd 1. No logs of why it exited at all 2. No heap dump which would imply there would be no logs as it crashed Is there any other way a process can die and linux would log it somehow? (like running out of memory) Thanks, Dean -- Franc Carter| Systems architect|Sirca Ltd <javascript:_e({},%20'cvml',%20'marc.zianideferra...@sirca.org.au');> franc.car...@sirca.org.au<javascript:_e({},%20'cvml',%20'franc.car...@sirca.org.au');> | www.sirca.org.au<http://www.sirca.org.au/> Tel: +61 2 8355 2514 Level 4, 55 Harrington St, The Rocks NSW 2000 PO Box H58, Australia Square, Sydney NSW 1215 -- Juan Manuel Formoso Senior Geek http://twitter.com/juanformoso http://seniorgeek.com.ar<http://seniorgeek.com.ar/> LLAP
