I'd say GC. Please fill in form CASS-FREEZE-001 below and get back to us :-) ( sorry )
How big is your JVM heap ? How many CPUs ? Garbage collection taking long ? ( look for log lines from GCInspector) Running out of heap ? ( "heap is .. full" log lines ) Any tasks backing up / being dropped ? ( nodetool tpstats and ".. dropped in last .. ms" log lines ) Are writes really slow? ( nodetool cfhistograms Keyspace ColumnFamily ) How much is lots of data? Wide or skinny rows? Mutations/sec ? Which Compaction Strategy are you using? Output of show schema ( cassandra-cli ) for the relevant Keyspace/CF might help as well What consistency are you doing your writes with ? I assume ONE or ANY if you have a single node. What are the values for these settings in cassandra.yaml memtable_total_space_in_mb: memtable_flush_writers: memtable_flush_queue_size: compaction_throughput_mb_per_sec: concurrent_writes: Which version of Cassandra? Regards, Andras From: Joel Samuelsson <samuelsson.j...@gmail.com> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> Date: Wednesday 20 March 2013 13:06 To: "user@cassandra.apache.org" <user@cassandra.apache.org> Subject: Cassandra freezes Hello, I've been trying to load test a one node cassandra cluster. When I add lots of data, the Cassandra node freezes for 4-5 minutes during which neither reads nor writes are served. During this time, Cassandra takes 100% of a single CPU core. My initial thought was that this was Cassandra flushing memtables to the disk, however, the disk i/o is very low during this time. Any idea what my problem could be? I'm running in a virtual environment in which I have no control of drives. So commit log and data directory is (probably) on the same drive. Best regards, Joel Samuelsson
