Re: multithreaded compaction

2011-04-26 Thread Terje Marthinussen
To be honest, this started after feeding data to cassandra for a while with compaction disabled (sort of a test case). when I enabled it... boom... spectacular process with 2000% CPU usage (please note... there is compression in cassandra in this system). This system actually have SSD's so when t

Re: multithreaded compaction

2011-04-26 Thread Sylvain Lebresne
FYI, I've also created https://issues.apache.org/jira/browse/CASSANDRA-2559 as another approach to the problem. -- Sylvain On Tue, Apr 26, 2011 at 9:35 AM, Sylvain Lebresne wrote: > On Tue, Apr 26, 2011 at 9:01 AM, Terje Marthinussen > wrote: >> Hi, >> I was testing the multithreaded compactio

Re: multithreaded compaction

2011-04-26 Thread Sylvain Lebresne
On Tue, Apr 26, 2011 at 9:01 AM, Terje Marthinussen wrote: > Hi, > I was testing the multithreaded compactions and with 2x6 cores (24 with HT) > it does seem a bit crazy with 24 compactions running concurrently. > It is probably not very good in terms of random I/O. It does seems a bit overkill.

multithreaded compaction

2011-04-26 Thread Terje Marthinussen
Hi, I was testing the multithreaded compactions and with 2x6 cores (24 with HT) it does seem a bit crazy with 24 compactions running concurrently. It is probably not very good in terms of random I/O. As such, I think I agree with the argument in 2191 that there should be a config option for this.

multithreaded compaction causes mutation storms?

2011-04-24 Thread Terje Marthinussen
Tested out multithreaded compaction in 0.8 last night. We had first fed some data with compaction disabled so there was 1000+ sstables on the nodes and I decided to enable multithreaded compaction on one of them to see how it performed vs. nodes that had no compaction at all. Since this was sort