I have been going through the mailing list and compiling suggestions to
address the swapping due to mmap issue.
1) Use JNA (done but)
Are these steps also required:
- Start Cassandra with CAP_IPC_LOCK (or as "root"). (not done)
grep Unevictable /proc/meminfo
- set /proc/sys/vm/swappiness to 0 (not done)
- set initial -Xms max -Xmx heapsize to same value(done)
2) use mmap_index_only
3) Disable swap completely
4) use 'standard' disk access mode and try mmap only if more performance is
needed
I was wondering if there an order of preference for these options based on
experience of folks who have tried these suggestions before I trial and
error my way to what works for us otherwise I am assuming the above order of
preference.
Our cluster Info:
Linux 2.6.18-194.32.1.el5.centos.plus #1 SMP x86_64 x86_64 x86_64 GNU/Linux
Cassandra v 0.7.2 , 4 nodes (JNA enabled), 16GB RAM, 8GB
heapsize(initial/max computed by cassandra-env.sh) , 5 GB swap
Swap space got swamped on all four nodes with a light write only load of
around 5-10 MBps at load size of about 5-6 GB on each node. Heap usage was
not high(around 50%). We are running a 4 node hadoop cluster as client(4
mappers). The nodes were responding well but the swapping is sending alerts
and waking up our sys admin and we want to configure this correctly before
it becomes a problem.
INFO [main] 2011-03-14 12:36:20,276 AbstractCassandraDaemon.java (line 97)
Heap size: 8328839168/8329887744
INFO [main] 2011-03-14 12:36:24,141 CLibrary.java (line 106) JNA mlockall
successful
INFO [main] 2011-03-14 12:36:24,151 DatabaseDescriptor.java (line 121)
Loading settings from
file:/usr/local/apache-cassandra-0.7.2/conf/cassandra.yaml
INFO [main] 2011-03-14 12:36:24,308 DatabaseDescriptor.java (line 181)
DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
