Our test setup 4 nodes, RF=3, reads at CL=QUOROM and we tried CL=TWO Tell the network card to slow down every packet on node 2 After fixing astyanax to not go to node 2 anymore, we are still seeing cassandra have issues as it seems to be involving node 2 somehow. If we take node 2 down, it all speeds back up.
We are trying to get this working such that a slow node in cassandra does not impact our customers. We are in 1.2.2 and added the following properties….(our properties show PropertyFileSnitch though I see the keyspace has org.apache.cassandra.locator.SimpleStrategy set probably because it was created through a tool instead of CLI…shucks)….anyways, I still expected dynamic snitch to work…. # controls how often to perform the more expensive part of host score # calculation dynamic_snitch: true dynamic_snitch_update_interval_in_ms: 100 # controls how often to reset all host scores, allowing a bad host to # possibly recover dynamic_snitch_reset_interval_in_ms: 600000 # if set greater than zero and read_repair_chance is < 1.0, this will allow # 'pinning' of replicas to hosts in order to increase cache capacity. # The badness threshold will control how much worse the pinned host has to be # before the dynamic snitch will prefer other replicas over it. This is # expressed as a double which represents a percentage. Thus, a value of # 0.2 means Cassandra would continue to prefer the static snitch values # until the pinned host was 20% worse than the fastest. dynamic_snitch_badness_threshold: 0.1 Any help appreciated, Thanks, Dean
