I have a 3 nodes cluster with replication factor = 3. I'm testing a complete node failure scenario. Here is my work flow.
1. Bring down node #3. We'll simulate that this node has failed. 2. Bring up a new node #4 with different IP address. Set bootstrap = True. Start cassandra. The 4th did not complete the bootstrap. It got stuck at this log message INFO [main] 2010-12-23 16:37:24,527 StorageService.java (line 475) Joining: getting bootstrap token I tested the same scenario with replication factor=3 using 4 nodes cluster. I simulated the 4th node failure while there 3 nodes were still alive. I was able to bootstrap the 5th node without any problem. I suspect that the number of live nodes before the bootstraping has to be more than replication factor. Am I wrong ? Is there anyway around this ? Thanks, -- Nick K
