Dropped read messages occur when the node could not process a read task within rpc_timeout. It generally means the cluster has been overwhelmed at some point: too many requests, to much GC, compaction hurting, etc.
Check the server side logs for errors but I doubt it is related to the call stack below. Have you confirmed that you are using framed transport on the client? Cheers ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 24/05/2012, at 5:52 PM, Deno Vichas wrote: > i've notice the my nodes seem to have a large (?, not really sure what > acceptable numbers are) read dropped count from tpstats. could they be > related? > > On 5/23/2012 2:55 AM, aaron morton wrote: >> >> No sure but >> >>> at >>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) >> >> Looks like the client is not using framed transport. The server defaults to >> framed. >> >> Cheers >> >> ----------------- >> Aaron Morton >> Freelance Developer >> @aaronmorton >> http://www.thelastpickle.com >> >> On 23/05/2012, at 5:35 AM, Deno Vichas wrote: >> >>> could somebody clue me in to the cause of this exception? i see these >>> randomly. >>> >>> AnalyzerService-2 2012-05-22 13:28:00,385 :: WARN >>> cassandra.connection.HConnectionManager - Exception: >>> me.prettyprint.hector.api.exceptions.HectorTransportException: >>> org.apache.thrift.transport.TTransportException >>> at >>> me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:39) >>> at >>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$23.execute(KeyspaceServiceImpl.java:851) >>> at >>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$23.execute(KeyspaceServiceImpl.java:840) >>> at >>> me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:99) >>> at >>> me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:243) >>> at >>> me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131) >>> at >>> me.prettyprint.cassandra.service.KeyspaceServiceImpl.getColumn(KeyspaceServiceImpl.java:857) >>> at >>> me.prettyprint.cassandra.model.thrift.ThriftColumnQuery$1.doInKeyspace(ThriftColumnQuery.java:57) >>> at >>> me.prettyprint.cassandra.model.thrift.ThriftColumnQuery$1.doInKeyspace(ThriftColumnQuery.java:52) >>> at >>> me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20) >>> at >>> me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85) >>> at >>> me.prettyprint.cassandra.model.thrift.ThriftColumnQuery.execute(ThriftColumnQuery.java:51) >>> at >>> com.stocktouch.dao.StockDaoImpl.getHistorical(StockDaoImpl.java:365) >>> at >>> com.stocktouch.dao.StockDaoImpl.getHistoricalQuote(StockDaoImpl.java:433) >>> at >>> com.stocktouch.service.StockHistoryServiceImpl.getHistoricalQuote(StockHistoryServiceImpl.java:480) >>> at >>> com.stocktouch.service.AnalyzerServiceImpl.getClose(AnalyzerServiceImpl.java:180) >>> at >>> com.stocktouch.service.AnalyzerServiceImpl.calcClosingPrices(AnalyzerServiceImpl.java:90) >>> at >>> com.stocktouch.service.AnalyzerServiceImpl.nightlyRollup(AnalyzerServiceImpl.java:66) >>> at >>> com.stocktouch.service.AnalyzerServiceImpl$2.run(AnalyzerServiceImpl.java:55) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:662) >>> Caused by: org.apache.thrift.transport.TTransportException >>> at >>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) >>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) >>> at >>> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129) >>> at >>> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101) >>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) >>> at >>> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378) >>> at >>> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297) >>> at >>> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204) >>> at >>> org.apache.cassandra.thrift.Cassandra$Client.recv_get(Cassandra.java:509) >>> at >>> org.apache.cassandra.thrift.Cassandra$Client.get(Cassandra.java:492) >>> at >>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$23.execute(KeyspaceServiceImpl.java:846) >>> ... 20 more >>> >>> >>> thanks, >>> deno >>> >>> >> >
