Hey guys, quick question: I've got a v2.1 cassandra cluster, 12 nodes on aws i3.2xl, commit log on one drive, data on nvme. That was working very well, it's a ts db and has been accumulating data for about 4weeks.
The nodes have increased in load and compaction seems to be falling behind. I used to get about 1 file per day for this column family, about ~30GB Data.db file per day. I am now getting hundreds per day at 1mb - 50mb. How to recover from this? I can scale out to give some breathing room but will it go back and compact the old days into nicely packed files for the day? I tried setting compaction throughput to 1000 from 256 and it seemed to make things worse for the CPU, it's configured on i3.2xl with 8 compaction threads. -B Lastly, I have mixed TTLs in this CF and need to run a repair (I think) to get rid of old tombstones, however running repairs in 2.1 on TWCS column families causes a very large spike in sstable counts due to anti-compaction which causes a lot of disruption, is there any other way?
