I was reluctant to use the thrift as well, and I spent about a week trying to get the CQL inserts to work by partitioning the INSERTS in different ways and tuning the cluster.
However, nothing worked remotely as well as the batch_mutate when it came to writing a full wide-row at once. I think Cassandra 2.0 makes CQL work better for these cases (CASSANDRA-4693), but I haven't tested it yet. -Paul > -----Original Message----- > From: Keith Freeman [mailto:8fo...@gmail.com] > Sent: Wednesday, September 11, 2013 1:06 PM > To: user@cassandra.apache.org > Subject: Re: heavy insert load overloads CPUs, with MutationStage pending > > Thanks, I had seen your stackoverflow post. I've got hundreds of > (wide-) rows, and the writes are pretty well distributed across them. > I'm very reluctant to drop back to the thrift interface. > > On 09/11/2013 10:46 AM, Paul Cichonski wrote: > > How much of the data you are writing is going against the same row key? > > > > I've experienced some issues using CQL to write a full wide-row at once > (across multiple threads) that exhibited some of the symptoms you have > described (i.e., high cpu, dropped mutations). > > > > This question goes into it a bit > more:http://stackoverflow.com/questions/18522191/using-cassandra-and- > cql3-how-do-you-insert-an-entire-wide-row-in-a-single-reque . I was able to > solve my issue by switching to using the thrift batch_mutate to write a full > wide-row at once instead of using many CQL INSERT statements. > > > > -Paul > > > >> -----Original Message----- > >> From: Keith Freeman [mailto:8fo...@gmail.com] > >> Sent: Wednesday, September 11, 2013 9:16 AM > >> To:user@cassandra.apache.org > >> Subject: Re: heavy insert load overloads CPUs, with MutationStage > >> pending > >> > >> > >> On 09/10/2013 11:42 AM, Nate McCall wrote: > >>> With SSDs, you can turn up memtable_flush_writers - try 3 initially > >>> (1 by default) and see what happens. However, given that there are > >>> no entries in 'All time blocked' for such, they may be something else. > >> Tried that, it seems to have reduced the loads a little after > >> everything warmed-up, but not much. > >>> How are you inserting the data? > >> A java client on a separate box using the datastax java driver, 48 > >> threads writing 100 records each iteration as prepared batch statements. > >> > >> At 5000 records/sec, the servers just can't keep up, so the client backs > >> up. > >> That's only 5M of data/sec, which doesn't seem like much. As I > >> mentioned, switching to SSDs didn't help much, so I'm assuming at > >> this point that the server overloads are what's holding up the client.
