Cassandra keeps certain data structures offheap, including bloom filters (scales with data size), compression metadata (scales with data size), and potentially memtables (scales with # of keyspaces/tables).
How much data on your node? Onheap or offheap memtables? From: ankit tyagi <ankittyagi.mn...@gmail.com> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> Date: Sunday, November 13, 2016 at 10:55 PM To: "user@cassandra.apache.org" <user@cassandra.apache.org> Subject: Too High resident memory of cassandra 2.2.8 Hi, we are using cassandra 2.2.8 version in production. we are seeing resident memory of cassndra process is very high 40G while heap size is only 8GB. root 23339 1 80 Nov11 ? 2-09:38:08 /opt/java8/bin/java -ea -javaagent:bin/../lib/jamm-0.3.0.jar -XX:+CMSClassUnloadingEnabled -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms8192M -Xmx8192M -Xmn2048M -XX:+HeapDumpOnOutOfMemoryError -Xss256k -XX:StringTableSize=1000003 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseTLAB -XX:+PerfDisableSharedMem -XX:CompileCommandFile=bin/../conf/hotspot_compiler -XX:CMSWaitDuration=10000 -XX:+CMSParallelInitialMarkEnabled -XX:+CMSEdenChunksRecordAlways -XX:CMSWaitDuration=10000 -XX:+UseCondCardMark -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+PrintPromotionFailure -Xloggc:bin/../logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=10M -Djava.net.preferIPv4Stack =true -Dcassandra.jmx.local.port=7199 -XX:+DisableExplicitGC -Djava.library.path=bin/../lib/sigar-bin -javaagent:/myntra/currentCassandra/lib/agent-1.2.jar=statsd.myntra.com:8125 -Dlogback.configurationFile=logback.xml -Dcassandra.logdir=bin/../logs -Dcassandra.storagedir=bin/../data -cp bin/../conf:bin/../build/classes/main:bin/../build/classes/thrift:bin/../lib/agent-1.2.jar:bin/../lib/airline-0.6.jar:bin/../lib/antlr-runtime-3.5.2.jar:bin/../lib/apache-cassandra-2.2.8.jar:bin/../lib/apache-cassandra-clientutil-2.2.8.jar:bin/../lib/apache-cassandra-thrift-2.2.8.jar:bin/../lib/cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-lang3-3.1.jar:bin/../lib/commons-math3-3.2.jar:bin/../lib/compress-lzf-0.8.4.jar:bin/../lib/concurrentlinkedhashmap-lru-1.4.jar:bin/../lib/crc32ex-0.1.1.jar:bin/../lib/disruptor-3.0.1.jar:bin/../lib/ecj-4.4.2.jar:bin/../lib/guava-16.0.jar:bin/../lib/high-scale-lib-1.0 .6.jar:bin/../lib/jackson-core-asl-1.9.2.jar:bin/../lib/jackson-mapper-asl-1.9.2.jar:bin/../lib/jamm-0.3.0.jar:bin/../lib/java-statsd-client-3.1.0.jar:bin/../lib/javax.inject.jar:bin/../lib/jbcrypt-0.3m.jar:bin/../lib/jcl-over-slf4j-1.7.7.jar:bin/../lib/jna-4.0.0.jar:bin/../lib/joda-time-2.4.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-0.9.2.jar:bin/../lib/log4j-over-slf4j-1.7.7.jar:bin/../lib/logback-classic-1.1.3.jar:bin/../lib/logback-core-1.1.3.jar:bin/../lib/lz4-1.3.0.jar:bin/../lib/metrics-core-2.2.0.jar:bin/../lib/metrics-core-3.1.0.jar:bin/../lib/metrics-jvm-3.1.0.jar:bin/../lib/metrics-logback-3.1.0.jar:bin/../lib/metrics-statsd-2.3.0.jar:bin/../lib/netty-all-4.0.23.Final.jar:bin/../lib/ohc-core-0.3.4.jar:bin/../lib/ohc-core-j8-0.3.4.jar:bin/../lib/reporter-config3-3.0.0.jar:bin/../lib/reporter-config-base-3.0.0.jar:bin/../lib/sigar-1.6.4.jar:bin/../lib/slf4j-api-1.7.7.jar:bin/../lib/snakeyaml-1.11.jar:bin/../lib/snappy-java-1.1.1.7.jar:bin/../lib/ST4-4.0.8.jar:b in/../lib/stream-2.5.2.jar:bin/../lib/super-csv-2.1.0.jar:bin/../lib/thrift-server-0.3.7.jar:bin/../lib/jsr223/*/*.jar org.apache.cassandra.service.CassandraDaemon below is the output of top command. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23339 root 20 0 295g 39g 28g S 107.5 63.5 3458:31 java I couldn't understand why cassandra process is using so much resident memory while heap size is only 8gb. Any insight??
smime.p7s
Description: S/MIME cryptographic signature
