I have the defaults as shown in your response.
On 09/10/2013 01:59 PM, sankalp kohli wrote:
What have you set these to?
# commitlog_sync may be either "periodic" or "batch."
# When in batch mode, Cassandra won't ack writes until the commit log
# has been fsynced to disk. It will wait up to
# commitlog_sync_batch_window_in_ms milliseconds for other writes, before
# performing the sync.
#
# commitlog_sync: batch
# commitlog_sync_batch_window_in_ms: 50
#
# the other option is "periodic" where writes may be acked immediately
# and the CommitLog is simply synced every commitlog_sync_period_in_ms
# milliseconds.
commitlog_sync: periodic
commitlog_sync_period_in_ms: 1000
On Tue, Sep 10, 2013 at 10:42 AM, Nate McCall <n...@thelastpickle.com
<mailto:n...@thelastpickle.com>> wrote:
With SSDs, you can turn up memtable_flush_writers - try 3
initially (1 by default) and see what happens. However, given that
there are no entries in 'All time blocked' for such, they may be
something else.
How are you inserting the data?
On Tue, Sep 10, 2013 at 12:40 PM, Keith Freeman <8fo...@gmail.com
<mailto:8fo...@gmail.com>> wrote:
On 09/10/2013 11:17 AM, Robert Coli wrote:
On Tue, Sep 10, 2013 at 7:55 AM, Keith Freeman
<8fo...@gmail.com <mailto:8fo...@gmail.com>> wrote:
On my 3-node cluster (v1.2.8) with 4-cores each and SSDs
for commitlog and data
On SSD, you don't need to separate commitlog and data. You
only win from this separation if you have a head to not-move
between appends to the commit log. You will get better IO
from a strip with an additional SSD.
Right, actually both partitions are on the same SSD.
Assuming you meant "stripe", would that really make a difference
Pool Name Active Pending Completed Blocked
All time blocked
MutationStage 1 9 290394 0
0
FlushWriter 1 2 20 0
0
I can't seem find information about the real meaning of
MutationStage, is this just normal for lots of inserts?
The mutation stage is the stage in which mutations to rows in
memtables ("writes") occur.
The FlushWriter stage is the stage that turns memtables into
SSTables by flushing them.
However, 9 pending mutations is a very small number. For
reference on an overloaded cluster which was being written to
death I recently saw.... 1216434 pending MutationStage. What
problem other than "high CPU load" are you experiencing? 2
Pending FlushWriters is slightly suggestive of some sort of
bound related to flushing..
So the basic problem is that write performance is lower than I
expected. I can't get sustained writing of 5000 ~1024-byte
records / sec at RF=2 on a good 3-node cluster, and my only
guess is that's because of the heavy CPU loads on the server
(loads over 10 on 4-CPU systems). I've tried both a single
client writing 5000 rows/second and 2 clients (on separate
boxes) writing 2500 rows/second, and in both cases the
server(s) doesn't respond quickly enough to maintain that
rate. It keeps up ok with 2000 or 3000 rows per second (and
has lower server loads).
