Are you using Leveled compaction strategy? If you fall behind on compaction in leveled (and you will during bootstrap), by default Cassandra will fall back to size tiered compaction in level 0. This will cause SSTables larger than sstable_size_in_mb, and those will be recompacted away into level 1. When level 1 gets full, those will be recompacted away into level 2. If level 2 gets full, into level 3 and so on.
Leveled compaction is more I/O intensive than size tiered compaction for reasons such as this. The same data can be compacted numerous times before a node settles down. The upside is that it puts a practical upper bound on the number of sstables which can be involved in a read (not more than one per level, not counting false positives from bloom filters). Leveled compaction is essentially trading increased I/O at write time (including and especially bootstrap or repair) for decreased I/O at read time. On Thu, Jan 8, 2015 at 6:12 AM, Robert Wille <rwi...@fold3.com> wrote: > After bootstrapping a node, the node repeatedly compacts the same tables > over and over, even though my cluster is completely idle. I’ve noticed the > same behavior after extended periods of heavy writes. I realize that during > bootstrapping (or extended periods of heavy writes) that compaction could > get seriously behind, but once a table has been compacted, I don’t see the > need to recompact the table dozens of more times. > > Possibly related, I often see that OpsCenter reports that nodes have a > large number of pending tasks, when Pending column of the Thread Pool Stats > doesn’t reflect that. > > Robert > >
