Thank you all for quick responses.
From: clohf...@blackbirdit.com
Subject: Re: high pending compactions
Date: Mon, 9 Jun 2014 14:11:36 -0500
To: user@cassandra.apache.org
Bean: org.apache.cassandra.db.CompactionManager
also nodetool compactionstats gives you how many are in the queue + estimate of
how many will be needed.
in 1.1 you will OOM far before you hit the limit,. In theory though, the
compaction executor is a little special cased and will actually throw an
exception (normally it will block)
Chris
On Jun 9, 2014, at 7:49 AM, S C <as...@outlook.com> wrote:Thank you all for
valuable suggestions. Couple more questions,
How to check the compaction queue? MBean/C* system log ?What happens if the
queue is full?
From: colinkuo...@gmail.com
Date: Mon, 9 Jun 2014 18:53:41 +0800
Subject: Re: high pending compactions
To: user@cassandra.apache.org
As Jake suggested, you could firstly increase
"compaction_throughput_mb_per_sec" and "concurrent_compactions" to suitable
values if system resource is allowed. From my understanding, major compaction
will internally acquire lock before running compaction. In your case, there
might be a major compaction blocking the pending following compaction tasks.
You could check the result of "nodetool compactionstats" and C* system log for
double confirm.
If the running compaction is compacting wide row for a long time, you could try
to tune "in_memory_compaction_limit_in_mb" value.
Thanks,
On Sun, Jun 8, 2014 at 11:27 PM, S C <as...@outlook.com> wrote:
I am using Cassandra 1.1 (sorry bit old) and I am seeing high pending
compaction count. "pending tasks: 67" while active compaction tasks are not
more than 5. I have a 24CPU machine. Shouldn't I be seeing more compactions? Is
this a pattern of high writes and compactions backing up? How can I improve
this? Here are my thoughts.
Increase memtable_total_space_in_mbIncrease
compaction_throughput_mb_per_secIncrease concurrent_compactions
Sorry if this was discussed already. Any pointers is much appreciated.
Thanks,Kumar
