Hi all, I have some troubles of cassandra in my production:
I build up a RPC server which using hector client to manipulate the cassandra. Wired things happen nowadays: the latency of RPC sometimes becames very high (10seconds~70seconds) in several minutes and reduce to normal level (30ms in average) after that time. I investigate the debug log of cassandra. During high latency time, the cassandra output lots of message like: "IncomingTcpConnection.java(116) Version is now 3. " Seems everything be blocked during that time. Our settings as following: The version of cassandra is 1.0.1 and hector version is 0.7.0 for the compatible of thrift version which we use (0.5.0) The cluster contains 4 nodes and all of them are seeds. The gcgraceseconds is 0 since we needn't delete the data p.s. It works well for a long time (3 months) but becames crazy these days after we push the new RPC server which supports bigger data saving (2mb in average). I'm not sure if these is the reason. Hope getting your replay~~ Thanks, Chao.
