Are all your nodes equally balanced in terms of read requests? Are you using RandomPartitioner? Are you reading using indexes?
First thing you can do is compare iostat -x output between the 2 nodes to rule out any io issues assuming your read requests are equally balanced. On Fri, Jan 6, 2012 at 10:11 AM, Daning Wang <dan...@netseer.com> wrote: > Hi all, > > We have 5 nodes cluster(0.8.6), but the performance from one node is way > behind others, I checked tpstats, It always show non-zero pending ReadStage, > I don't see this problem on other nodes. > > What caused the problem? I/O? Memory? Cpu usage is still low. How to fix > this problem? > > ~/bin/nodetool -h localhost tpstats > Pool Name Active Pending Completed Blocked All > time blocked > ReadStage 11 15 56960 > 0 0 > RequestResponseStage 0 0 606695 > 0 0 > MutationStage 0 0 538634 > 0 0 > ReadRepairStage 0 0 199997 > 0 0 > ReplicateOnWriteStage 0 0 0 > 0 0 > GossipStage 0 0 5734 > 0 0 > AntiEntropyStage 0 0 0 > 0 0 > MigrationStage 0 0 0 > 0 0 > MemtablePostFlusher 0 0 7 > 0 0 > StreamStage 0 0 0 > 0 0 > FlushWriter 0 0 8 > 0 0 > MiscStage 0 0 0 > 0 0 > FlushSorter 0 0 0 > 0 0 > InternalResponseStage 0 0 0 > 0 0 > HintedHandoff 1 4 0 > 0 0 > > Message type Dropped > RANGE_SLICE 0 > READ_REPAIR 0 > BINARY 0 > READ 9082 > MUTATION 0 > REQUEST_RESPONSE 0 > > Thanks you in advance. > > Daning >
