Counter increment is a special case in cassandra because the incur a local read before write. Normal column writes so not do this. So counter writes are intensive. If possible batch up the increments for less rpc calls and less reads.
On Saturday, December 10, 2011, Peter Schuller <peter.schul...@infidyne.com> wrote: >> I've got a batch process running every so often that issues a bunch of >> counter increments. I have noticed that when this process runs without being >> throttled it will raise the CPU to 80-90% utilization on the nodes handling >> the requests. This in turns timeouts and general lag on queries running on >> the cluster. > > This much is entirely expected. If you are not bottlenecking anywhere > else and saturing the cluster, you will be bound by it, and it will > affect the latency of other traffic, no matter how fast or slow > Cassandra is. > > You do say "nodes handling the requests". Two things to always keep in > mind is to (1) spread the requests evenly across all members of the > cluster, and (2) if you are doing a lot of work per row key, spread it > around and be concurrent so that you're not hitting a single row at a > time, which will be under the responsibility of a single set of RF > nodes (you want to put load on the entire cluster evently if you want > to maximize throughput). > >> Is there anything that can be done to increase the throughput, I've been >> looking on the wiki and the mailing list and didn't find any optimization >> suggestions (apart from spreading the load on more nodes). >> >> Cluster is 5 node, BOP, RF=3, AMD opteron 4174 CPU (6 x 2.3 Ghz cores), >> Gigabit ethernet, RAID-0 SATA2 disks > > For starters, what *is* the throughput? How many counter mutations are > you submitting per second? > > -- > / Peter Schuller (@scode, http://worldmodscode.wordpress.com) >
