After I use sstableloader from a old cluster, the new cluster node's data size
double. The new cluster has the same node with old cluster( 3 nodes).
The old cluster has 500G each node,after sstableloader completely, the new
cluster data size nearly 1T each node.
The problem is that compaction(LCS) doesn't endless for over 10 days. It seems
at level 0 ,it pick 32 sstables compaction into one large ones. And then pick
this large sstable and other 31 original sstable compaction into larger one.
SStable Info:
SSTable count: 6965
SSTables in each level: [6931/4, 32/10, 0, 0, 0, 0, 0, 0, 0]
Compaction throgutput: 200MB/s
The compaction logs:
INFO [CompactionExecutor:615] 2016-01-03 20:25:14,739 CompactionTask.java:274
- Compacted 32 sstables to
[/disk/data3/cassandra/test/t_calllog-43f6c250aa0d11e59ac60b2534
4dc914/test-t_calllog-ka-11299,]. 5,370,317,257 bytes to 5,370,305,557 (~99%
of original) in 10,766,638ms = 0.475684MB/s. 219,175 total partitions merged
to 219,175. P
artition merge counts were {1:219175, }
SStable size:
-rw-r--r-- 1 storage storage 47033344647 Jan 2 11:11
test-t_calllog-ka-11233-Data.db
-rw-r--r-- 1 storage storage 5370290840 Jan 3 23:09
test-t_calllog-ka-11303-Data.db
-rw-r--r-- 2 storage storage 1461236940 Jan 4 15:18
test-t_calllog-tmp-ka-11329-Data.db
-rw-r--r-- 2 storage storage 1461236940 Jan 4 15:18
test-t_calllog-tmplink-ka-11329-Data.db
-rw-r--r-- 1 storage storage 1204113103 Jan 2 11:11
test-t_calllog-ka-11233-Index.db
-rw-r--r-- 1 storage storage 168593618 Dec 25 19:59
test-t_calllog-ka-10505-Data.db
-rw-r--r-- 1 storage storage 168592670 Dec 24 22:29
test-t_calllog-ka-2564-Data.db
-rw-r--r-- 1 storage storage 168591491 Dec 24 20:46
test-t_calllog-ka-1528-Data.db
Does it is not a good usecase using sstableloader to load large cluster? As
which will create many uncompacted sstables.
