Re: Compaction, Slow Ring, and bad behavior

2013-04-30 Thread aaron morton
Check the logs for warnings from the GCInspector. If you see messages that correlate with compaction running limit compaction to help stabilise thingsā€¦ * set concurrrent_compactions to 2 * if you have wide rows reduce in_memory_compaction_limit * reduce compaction_throughput If you have a lot (

Compaction, Slow Ring, and bad behavior

2013-04-29 Thread Drew from Zhrodague
Hi, we have a 9-node ring on m1.xlarge AWS hosts. We started having some trouble a while ago, and it's making me pull out all of my hair. The host in position #3 has been replaced 4 times. Each time, the host joins the ring, I do a nodetool repair -pr, and she seems fine for about a day. The