Peter, Jonathon - thank you for your replies. I should probably have repeated myself in the body, but as I mentioned in the subject line, we're running Sun Java 1.6.
On 10 December 2010 18:37, Peter Schuller <peter.schul...@infidyne.com> wrote: > Memory-mapped files will account for both virtual and, to the extent > that they are resident in memory, to the resident size of the process. This bears further investigation. Would you consider a 3GB overhead on a 4GB heap a possibility? (From a position of some naivety, this seems a bit extreme, though I'm continually surprised by Java.) > However, your graph: > >> I've shared a Zabbix graph of system memory at: >> >> http://www.imagebam.com/image/3b4213110283969 > > Certainly indicates that it is not the explanation since you should be > seeing cached occupy the remainder of memory above heap size. In > addition the allocation failures from jmap indicates memory is truly > short. This was my concern. As mentioned, we're on 7GB machines, 4GB JVM heap, and an expectation that we'll get some gain from the remainder being used sensibly by Linux as cache. > Just to confirm, what does the free +/- buffers show if you run > 'free'? (I.e., middle line, under 'free' column) Dang. I thought I'd kept a cap of top and free on those boxes before restarting Cassandra. I'm confident that the item you're talking about matches top's 'cached' entry - which at the time was showing ~ 10MB. > A Java memory leak would likely indicate non-heap managed memory > (since I think it's unlikely that the JVM fails to limit the actual > heap size). The question is what.... I guess this comes back to my first question - (how) can a JVM with a heap of 4GB get to 7.1GB of resident? I understand a process can blow out its virtual footprint, and despite a lack of swap on those boxes, that bit isn't in itself a massive concern. On 12 December 2010 16:21, Jonathan Ellis <jbel...@gmail.com> wrote: > http://www.riptano.com/docs/0.6/troubleshooting/index#nodes-are-dying-with-oom-errors Jonathon - to clarify - this seems to be happening *to* Java, not within the JVM. I posted mostly as a heads up for others using similar profiles (4GB heap on ~8GB boxes) to keep an eye out for. I expect a few people, particularly if they're on Amazon EC2, are running this type of setup. On the other hand, mum always said I was unique. ;) cheers, Jedd.
