Re: BloomFilter is taking too much memory

2010-05-04 Thread Weijun Li
More insight for this sstable: the ArrayList for IndexSummary has 644195 entries, so total number of entries for this sstable is: 644195*128=~82mil. The problem is that the total bits for its BloomFilter (long[19400551] inside BitSet) is 19400551*64=1241635264, which means each key is taking ~15bit

Re: BloomFilter is taking too much memory

2010-05-04 Thread Jonathan Ellis
BloomFilter is not redundant, because it stores information about _all_ keys while the index summary stores every 1/128 key. On Tue, May 4, 2010 at 3:47 PM, Weijun Li wrote: > Hello, > > We stored about 47mil keys in one Cassandra node and what a memory dump > shows for one of the SStableReader: