Hello,

on a quite capable machine with 32 physical cores (64 vCPUs) we see sporadic 
CPU usage up to 50% caused by nodetool on this box, thus digged a bit further. 
A few observations:

1) nodetool is reusing the $MAX_HEAP_SIZE environment variable, thus if we are 
running Cassandra with e.g. Xmx31G, nodetool is started with Xmx31G as well
2) As -XX:ParallelGCThreads is not explicitly set upon startup, this basically 
defaults to a value dependent on the number of cores. In our case, with the 
machine above, the number of parallel GC threads for the JVM is set to 43!
3) Test-wise, we have adapted the nodetool startup script in a way to get a 
Java Flight Recording file on JVM exit, thus with each nodetool invocation we 
can inspect a JFR file. Here we may have seen System.gc() calls (without 
visible knowledge where they come from), GC times for the entire JVM life-time 
(e.g. ~1min) showing high cpu. This happened for both Xmx128M (default as it 
seems) and Xmx31G

After explicitly setting -XX:ParallelGCThreads=1 in the nodetool startup 
script, CPU usage spikes by nodetool are entirely gone.

Is this something which has been already adapted/tackled in Cassandra versions 
> 2.1 or worth to be considered as some sort of RFC?

Thanks,
Thomas

The contents of this e-mail are intended for the named addressee only. It 
contains information that may be confidential. Unless you are the named 
addressee or an authorized designee, you may not copy or use it, or disclose it 
to anyone else. If you received it in error please notify us immediately and 
then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a 
company registered in Linz whose registered office is at 4040 Linz, Austria, 
Freist?dterstra?e 313

Reply via email to