Hello,
we are using Cassandra as a data storage for our caching system. Our
application generates about 20 put and get requests per second. An
average size of one cache item is about 500 Kb.
Cache items are placed into one column family with TTL set to 20 - 60
minutes. Keys and values are bytes (not utf8 strings). Compaction
strategy is SizeTieredCompactionStrategy.
We setup Cassandra 1.2.6 cluster of 4 nodes. Replication factor is 2.
Each node has 10GB of RAM and enough space on HDD.
Now when we're putting this cluster into the load it's quickly fills
with our runtime data (about 5 GB on every node) and we start observing
performance degradation with often timeouts on client side.
We see that on each node compaction starts very frequently and lasts for
several minutes to complete. It seems that each node usually busy with
compaction process.
Here the questions:
What are the recommended setup configuration for our use case?
Is it makes sense to somehow tell Cassandra to keep all data in memory
(memtables) to eliminate flushing it to disk (sstables) thus decreasing
number of compactions? How to achieve this behavior?
Cassandra is starting with default shell script that gives the following
command line:
jsvc.exec -user cassandra -home
/usr/lib/jvm/java-6-openjdk-amd64/jre/bin/../ -pidfile
/var/run/cassandra.pid -errfile &1 -outfile
/var/log/cassandra/output.log -cp <CLASSPATH_SKIPPED>
-Dlog4j.configuration=log4j-server.properties
-Dlog4j.defaultInitOverride=true
-XX:HeapDumpPath=/var/lib/cassandra/java_1371805844.hprof
-XX:ErrorFile=/var/lib/cassandra/hs_err_1371805844.log -ea
-javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar
-XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms2500M -Xmx2500M
-Xmn400M -XX:+HeapDumpOnOutOfMemoryError -Xss180k -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled
-XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
-XX:+UseTLAB -Djava.net.preferIPv4Stack=true
-Dcom.sun.management.jmxremote.port=7199
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
org.apache.cassandra.service.CassandraDaemon
--
Best regards,
Dmitry Olshansky
