Hello, We are experiencing an issue where nodes a temporarily slow due to I/O contention anywhere from 10 minutes to 2 hours. I don't believe this slowdown is Cassandra related, but factors outside of Cassandra. We run Cassandra 1.1.9. We run a 12 node cluster, with a replication factor of 3, and all queries use LOCAL_QUORUM consistency.
Our problem is (other than the contention issue, which we are working on), when this one node slows down, the whole system performance appears to slow down. Is there a way in Cassandra to accommodate or mitigate slower nodes? Shutting down the node in question during the period of contention does "resolve" the performance problem, but is there anything in cassandra that can assist this situation while we resolve the hardware problem? Thanks, -Mike