Hi all
strange things here: we are using jna. Log file says mlockall was successful.
We start with -Xms2000M -Xmx2000M and run cassandra as root process so
RLIMIT_MEMLOCK limit should have no relevance. Still cassandra is swapping ...
Used swap varies between 100MB - 800MB
We removed the swap partition altogether now but I still dont understand why
this happens.
We see this on nodes with a longer uptime (> 2 weeks).
Here's some process info:
top - 14:27:35 up 146 days, 3:02, 1 user, load average: 0.89, 0.97, 0.93
Tasks: 122 total, 1 running, 121 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.4%us, 0.6%sy, 0.0%ni, 85.5%id, 12.6%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 6128360k total, 5852408k used, 275952k free, 4472k buffers
Swap: 1951892k total, 231008k used, 1720884k free, 1576720k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
29757 root 18 -2 251g 3.7g 298m S 6 63.8 1590:17 java
blnrzh019:/var/log/cassandra# ps axxx|grep 29757
29757 ? S<Ll 1589:56 /usr/bin/java -ea -Xms2000M -Xmx2000M
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled
-XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
-Dcassandra.compaction.priority=1 -Dcassandra.dynamic_snitch=true
-Dcom.sun.management.jmxremote.port=8080
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dstorage-config=/opt/smeet-cassandra/bin/../conf -cp
/opt/smeet-cassandra/bin/../conf:/opt/smeet-cassandra/bin/../build/classes:/opt/smeet-cassandra/bin/../lib/antlr-3.1.3.jar:/opt/smeet-cassandra/bin/../lib/apache-cassandra-0.6.12-patched.jar:/opt/smeet-cassandra/bin/../lib/clhm-production.jar:/opt/smeet-cassandra/bin/../lib/commons-cli-1.1.jar:/opt/smeet-cassandra/bin/../lib/commons-codec-1.2.jar:/opt/smeet-cassandra/bin/../lib/commons-collections-3.2.1.jar:/opt/smeet-cassandra/bin/../lib/commons-lang-2.4.jar:/opt/smeet-cassandra/bin/../lib/google-collections-1.0.jar:/opt/smeet-cassandra/bin/../lib/hadoop-core-0.20.1.jar:/opt/smeet-cassandra/bin/../lib/high-scale-lib.jar:/opt/smeet-cassandra/bin/../lib/ivy-2.1.0.jar:/opt/smeet-cassandra/bin/../lib/jackson-core-asl-1.4.0.jar:/opt/smeet-cassandra/bin/../lib/jackson-mapper-asl-1.4.0.jar:/opt/smeet-cassandra/bin/../lib/jline-0.9.94.jar:/opt/smeet-cassandra/bin/../lib/jna-3.2.7.jar:/opt/smeet-cassandra/bin/../lib/jna.jar:/opt/smeet-cassandra/bin/../lib/json-simple-1.1.jar:/opt/smeet-cassandra/bin/../lib/libthrift-r917130.jar:/opt/smeet-cassandra/bin/../lib/log4j-1.2.14.jar:/opt/smeet-cassandra/bin/../lib/slf4j-api-1.5.8.jar:/opt/smeet-cassandra/bin/../lib/slf4j-log4j12-1.5.8.jar:/opt/smeet-cassandra/bin/../lib/smeet-cassandra-contrib.jar
org.apache.cassandra.thrift.CassandraDaemon
blnrzh019:/var/log/cassandra# cat /proc/29757/smaps |grep -i swap| awk '{SUM +=
$2} END {print "SUM: " SUM " kB (" SUM/1024 " MB)"}'
SUM: 207844 kB (202.973 MB)
blnrzh019:/var/log/cassandra# grep JNA /var/log/cassandra/system.log*
/var/log/cassandra/system.log.1: INFO [main] 2011-01-27 17:38:11,201
CLibrary.java (line 86) JNA mlockall successful
/var/log/cassandra/system.log.1: INFO [main] 2011-02-16 07:47:24,788
CLibrary.java (line 86) JNA mlockall successful
/var/log/cassandra/system.log.1: INFO [main] 2011-02-18 12:29:39,958
CLibrary.java (line 86) JNA mlockall successful
/var/log/cassandra/system.log.1: INFO [main] 2011-02-25 11:59:42,318
CLibrary.java (line 86) JNA mlockall successful
