It sure looks like what I'm seeing on my cluster where a 100G commit lot
partition fills up in 12 hours (0.8.x)
Le 23 sept. 2011 03:45, "Yang" <teddyyyy...@gmail.com> a écrit :
> in 1.0.0 we don't have memtable_throughput for each individual CF ,
> and instead
> which memtable/CF to flush is determined by "largest
> getTotalMemtableLiveSize() ".
> (MeteredFlusher.java line 81)
>
>
> what would happen in the following case ? : I have only 2 CF, the
> traffic for one CF is 1000 times that
> of the second CF,
> so the high-traffic CF constantly triggers total mem threshold , and
> every time, the busy CF is flushed.
>
> but the light-traffic CF is never flushed ( well, until we have
> flushed about 1000 times the busy CF),
> now we are left with many commit logs , each of them containing a few
> entries for the light-traffic table. we have to keep these commit logs
> because these entries are not flushed to sstable yet.
>
> then there are 2 problems: 1) to persist the few records from the
> light-traffic CF, you have to keep 1000 times the commit logs
> necessary, taking up disk space 2) when you do a recover on server
> restart, you'll have to read through all those commit logs .
>
> does the above hypothesis sound right?
>
> Thanks
> Yang
