are you i/o bound? I believe counter ROW needs to perform a read of
the old value.
On Wed, Aug 10, 2011 at 7:17 PM, Ian Danforth <idanfo...@numenta.com> wrote:
> Hello all!
> Thanks for taking the time to read this, I'm new to Cassandra and really
> want to get it working :)
> Below you'll find the output from tpstats on the three nodes in my .8.1
> cluster. As you'll note from ring (also below), my cluster appears balanced.
> However, please note that NODE1 is the only node that has completed any
> ReplicateOnWriteStage actions. I've been having a lot of trouble with RoW
> and seen a single node in my various test clusters get hugely backed up on
> this stage to the point where it slows all writes to a crawl.
> I'm using counters in a column family defined like this:
> ==== KEYSPACE/CF ====
> create keyspace Keyspace1
> with strategy_options=[{replication_factor:3}]
> and placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy';
> use Keyspace1;
> create column family TwitterTest
> with default_validation_class = CounterColumnType
> and replicate_on_write = true;
> ====
>
> With any given batch_insert() I'm incrementing ~17,000 counters. I've backed
> off a bit and started using ConsistencyLevel.QUORUM, as it was suggested
> elsewhere that ANY or ONE could overwhelm the system.
>
> I hope I'm just doing something stupid and one you experienced folks can
> point out my error.
> Thanks in advance!
> Ian
> ==== MORE DATA ====
> Address DC Rack Status State Load Owns
> Token
>
> 113427455640312821154458202477256070484
> 10.87.9.57 datacenter1 rack1 Up Normal 193.86 MB
> 33.33% 0
> 10.86.223.32 datacenter1 rack1 Up Normal 191.1 MB
> 33.33% 56713727820156410577229101238628035242
> 10.86.229.24 datacenter1 rack1 Up Normal 193.63 MB
> 33.33% 113427455640312821154458202477256070484
>
>
>
> SEED
> Pool Name Active Pending Completed
> ReadStage 0 0 7
> RequestResponseStage 0 0 1479839
> MutationStage 0 0 1479908
> ReadRepairStage 0 0 0
> ReplicateOnWriteStage 0 0 0
> GossipStage 0 0 5176
> AntiEntropyStage 0 0 0
> MigrationStage 0 0 128
> MemtablePostFlusher 0 0 28
> StreamStage 0 0 0
> FlushWriter 0 0 28
> MiscStage 0 0 2
> FlushSorter 0 0 0
> InternalResponseStage 0 0 363
> HintedHandoff 0 0 0
> NODE1
> Pool Name Active Pending Completed
> ReadStage 0 0 7
> RequestResponseStage 0 0 2960208
> MutationStage 0 0 1480104
> ReadRepairStage 0 0 0
> ReplicateOnWriteStage 0 0 1480104
> GossipStage 0 0 4151
> AntiEntropyStage 0 0 0
> MigrationStage 0 0 128
> MemtablePostFlusher 0 0 31
> StreamStage 0 0 8
> FlushWriter 0 0 31
> MiscStage 0 0 8
> FlushSorter 0 0 0
> InternalResponseStage 0 0 1
> HintedHandoff 0 0 0
> NODE2
> Pool Name Active Pending Completed
> ReadStage 0 0 7
> RequestResponseStage 0 0 0
> MutationStage 0 0 1481530
> ReadRepairStage 0 0 0
> ReplicateOnWriteStage 0 0 0
> GossipStage 0 0 4201
> AntiEntropyStage 0 0 0
> MigrationStage 0 0 128
> MemtablePostFlusher 0 0 31
> StreamStage 0 0 6
> FlushWriter 0 0 31
> MiscStage 0 0 6
> FlushSorter 0 0 0
> InternalResponseStage 0 0 1
> HintedHandoff 0 0 0
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
