Re: Tracking down read latency

2011-02-04 Thread sridhar basam
On Fri, Feb 4, 2011 at 2:44 PM, David Dabbs wrote: > > Our data is on sdb, commit logs on sdc. > So do I read this correctly that we're 'await'ing 6+millis on average for > data drive (sdb) > requests to be serviced? > > That is right. Those numbers look pretty good for rotational media. What sor

RE: Tracking down read latency

2011-02-04 Thread David Dabbs
Thank you both for your advice. See my updated iostats below. >From: sridhar.ba...@gmail.com [mailto:sridhar.ba...@gmail.com] On Behalf Of sridhar basam >Sent: Thursday, February 03, 2011 10:58 AM >To: user@cassandra.apache.org >Subject: Re: Tracking down read latency > >Th

Re: Tracking down read latency

2011-02-03 Thread sridhar basam
The data provided is also a average value since boot time. Run the -x as suggested below but run it via a interval of around 5 seconds. You very well could be having i/o issue, it is hard to tell from the overall average value you provided. Collect "iostat -x 5" during the times when you see slow r

Re: Tracking down read latency

2011-02-03 Thread Peter Schuller
> $ iostat As rcoli already mentioned you don't seen to have an I/O problem, but as a point of general recommendation: When determining whether you are blocking on disk I/O, pretty much *always* use "iostat -x" rather than the much less useful default mode of iostat. The %util and queue wait/avera

Re: Tracking down read latency

2011-02-02 Thread Robert Coli
On Wed, Feb 2, 2011 at 9:35 PM, David Dabbs wrote: > We’re encountering some high read latency issues. What is reporting high read latency? > We're more read than write, though there doesn't seem to be many pending > reads. > I have seen active/pending row-read at three or four, though. In gene

Tracking down read latency

2011-02-02 Thread David Dabbs
Hello. We’re encountering some high read latency issues. But our main Cass expert is out-of-office so it falls to me. We're more read than write, though there doesn't seem to be many pending reads. I have seen active/pending row-read at three or four, though. Pool NameActive