Re: Simulating a failed node

2012-10-29 Thread Andrew Bialecki
Thanks, extremely helpful. The key bit was I wasn't flushing the old Keyspace before re-running the stress test, so I was stuck at RF = 1 from a previous run despite passing RF = 2 to the stress tool. On Sun, Oct 28, 2012 at 2:49 AM, Peter Schuller wrote: > > Operation [158320] retried 10 times

Re: Simulating a failed node

2012-10-27 Thread Peter Schuller
> Operation [158320] retried 10 times - error inserting key 0158320 > ((UnavailableException)) This means that at the point where the thrift request to write data was handled, the co-ordinator node (the one your client is connected to) believed that, among the replicas responsible for the key, to

Re: Simulating a failed node

2012-10-27 Thread Andrew Bialecki
The default replication factor and consistency level for the stress tool is one, so that's what I'm using. I've also experimented and seen the same behavior with RF=2, but I haven't tried a different CL. On Sun, Oct 28, 2012 at 12:36 AM, Watanabe Maki wrote: > What RF and CL are you using? > > >

Re: Simulating a failed node

2012-10-27 Thread Watanabe Maki
What RF and CL are you using? On 2012/10/28, at 13:13, Andrew Bialecki wrote: > Hey everyone, > > I'm trying to simulate what happens when a node goes down to make sure my > cluster can gracefully handle node failures. For my setup I have a 3 node > cluster running 1.1.5. I'm then using the

Simulating a failed node

2012-10-27 Thread Andrew Bialecki
Hey everyone, I'm trying to simulate what happens when a node goes down to make sure my cluster can gracefully handle node failures. For my setup I have a 3 node cluster running 1.1.5. I'm then using the stress tool included in 1.1.5 coming from an external server and running it with the following