Hey! So I'm running Cassandra 2.1.2 and using the
SizeTieredCompactionStrategy. I'm doing about 3k writes/sec on a single
node. My read performance is terrible, all my queries just time out. So I
do nodetool cfstats:
Read Count: 42071
Read Latency: 67.47804242827601 ms.
Write Count: 131964300
Write Latency: 0.011721604274792501 ms.
Pending Flushes: 0
Table: metrics16513
SSTable count: 641
Space used (live): 6366740812
Space used (total): 6366740812
Space used by snapshots (total): 0
SSTable Compression Ratio: 0.25272488401992765
Memtable cell count: 0
Memtable data size: 0
Memtable switch count: 1016
Local read count: 42071
Local read latency: 67.479 ms
Local write count: 131964300
Local write latency: 0.012 ms
Pending flushes: 0
Bloom filter false positives: 994
Bloom filter false ratio: 0.00000
Bloom filter space used: 37840376
Compacted partition minimum bytes: 104
Compacted partition maximum bytes: 24601
Compacted partition mean bytes: 255
Average live cells per slice (last five minutes): 111.67243951154147
Maximum live cells per slice (last five minutes): 1588.0
Average tombstones per slice (last five minutes): 0.0
Maximum tombstones per slice (last five minutes): 0.0
and nodetool cfhistograms:
Percentile SSTables Write Latency Read Latency Partition
Size Cell Count
(micros) (micros)
(bytes)
50% 46.00 6.99 154844.95
149 1
75% 430.00 8.53 3518837.53
179 1
95% 430.00 11.32 7252897.25
215 2
98% 430.00 15.54 22103886.34
215 3
99% 430.00 29.86 22290608.19
1597 50
Min 0.00 1.66 26.91
104 0
Max 430.00 269795.38 27311364.89
24601 924
Gross!! There are 641 SSTables in there, and all my reads are hitting
hundreds of them and timing out. How could this possibly have happened, and
what can I do about it? Nodetool compactionstats says pending tasks: 0, by
the way. Thanks!
