My first guess would be read repair, are you seeing any increase in ReadRepairStage tasks ?
RR (in 1.X) is normally only enabled for 10% of the request. cheers ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 20/07/2012, at 5:17 AM, jmodha wrote: > Hi, > > We've observed a some strange behaviour after finishing streaming data into > our new cluster. > > Below are the steps we're doing: > > 1 - We stream data from an old cluster (1.0.3) to a new cluster (1.1.1). > This completes with no errors. > 2 - We then run some validation scripts (some Java code which uses hector) > to ensure what we've streamed in matches the data in our old cluster. > 3 - The validation script is a read-only operation, but when observing the > WriteCount JMX property on one of the column families its continuously > increasing across all nodes! > > I have verified that we only do reads and no writes, so why am I seeing the > WriteCount JMX property increasing on all of my nodes for this one column > family only? > > Some other info.. we're reading at LOCAL_QUORUM, its a two DC setup, 6 nodes > in each DC. All of our CF's have rows that contain a mixture of TTL and > non-TTL columns. > > Any ideas as to why this is happening would be greatly appreciated! > > P.S. I've attached some graphs which show reads vs. writes on the CF in > question. > > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/file/n7581333/reads.png > > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/file/n7581333/writes.png > > > -- > View this message in context: > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Seeing-writes-when-only-expecting-reads-tp7581333.html > Sent from the cassandra-u...@incubator.apache.org mailing list archive at > Nabble.com.
