Well, we finally have dynamic switch working. It seems to switch nodes to a Remote node and SimpleSTrategy can not deal with that well. Also, we had to move to CL=QUOROM_LOCAL instead of QUOROM.
So for those of you that want cassandra to keep performing well when one node starts to get really slow, make sure you have 1. NetworkTopologyStrategy set 2. Have CL=QUOROM_LOCAL We had to dive into the code to figure all that out. I hope it helps someone else. Later, Dean From: aaron morton <aa...@thelastpickle.com<mailto:aa...@thelastpickle.com>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Date: Monday, March 11, 2013 6:48 AM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: Re: has anyone used dynamic snitch at all Check that read_repair_chance on the CF's is 0.1, not the old 1.0 Wait at least 10 minutes for the DynamicSnitch to re-calculate. Use the org.apache.cassandra.db:type=DynamicEndpointSnitch MBean to see what scores it has given the nodes. Cheers ----------------- Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 8/03/2013, at 11:40 AM, Edward Capriolo <edlinuxg...@gmail.com<mailto:edlinuxg...@gmail.com>> wrote: dynamic_snitch=true is the default. So it is usually on "wrapping" other snitches. I have found several scenarios where it does not work exactly as your would expect. On Fri, Mar 8, 2013 at 2:26 PM, Hiller, Dean <dean.hil...@nrel.gov<mailto:dean.hil...@nrel.gov>> wrote: Our test setup 4 nodes, RF=3, reads at CL=QUOROM and we tried CL=TWO Tell the network card to slow down every packet on node 2 After fixing astyanax to not go to node 2 anymore, we are still seeing cassandra have issues as it seems to be involving node 2 somehow. If we take node 2 down, it all speeds back up. We are trying to get this working such that a slow node in cassandra does not impact our customers. We are in 1.2.2 and added the following properties….(our properties show PropertyFileSnitch though I see the keyspace has org.apache.cassandra.locator.SimpleStrategy set probably because it was created through a tool instead of CLI…shucks)….anyways, I still expected dynamic snitch to work…. # controls how often to perform the more expensive part of host score # calculation dynamic_snitch: true dynamic_snitch_update_interval_in_ms: 100 # controls how often to reset all host scores, allowing a bad host to # possibly recover dynamic_snitch_reset_interval_in_ms: 600000 # if set greater than zero and read_repair_chance is < 1.0, this will allow # 'pinning' of replicas to hosts in order to increase cache capacity. # The badness threshold will control how much worse the pinned host has to be # before the dynamic snitch will prefer other replicas over it. This is # expressed as a double which represents a percentage. Thus, a value of # 0.2 means Cassandra would continue to prefer the static snitch values # until the pinned host was 20% worse than the fastest. dynamic_snitch_badness_threshold: 0.1 Any help appreciated, Thanks, Dean
