Jonathan, I think it's the case of large values in the columns. The problematic CF is a key-value store, so it has only one column per row, however the value of that column can be large. It's a java serialized object (uncompressed) which, may be 100s of bytes, maybe even a few megs. This CF also suffers from zero cache hits since each time a read is for a unique key.
I ran stress.py and I see much better results (reads are < 1ms) so I assume my cluster is healthy, so I need to fix the app. Would 1meg bytes object explain a 30ms (sometimes even more) read latency? The boxes aren't fancy, not sure exactly what hardware we have there but it's "commodity"... Thanks! On Thu, May 6, 2010 at 5:22 PM, Jonathan Ellis <jbel...@gmail.com> wrote: > columns, not CFs. > > put another way, how wide are the rows in the slow CF? > > On Wed, May 5, 2010 at 11:30 PM, Ran Tavory <ran...@gmail.com> wrote: > > I have a few CFs but the one I'm seeing slowness in, which is the one > with > > plenty of cache misses has only one column per key. > > Latency varies b/w 10m and 60ms but I'd say average is 30ms. > > > > On Thu, May 6, 2010 at 4:25 AM, Jonathan Ellis <jbel...@gmail.com> > wrote: > >> > >> How many columns are in the rows you are reading from? > >> > >> 30ms is quite high, so I suspect you have relatively large rows, in > >> which case decreasing the column index threshold may help. > > -- > Jonathan Ellis > Project Chair, Apache Cassandra > co-founder of Riptano, the source for professional Cassandra support > http://riptano.com >
