Here is a bit more detail of the whole situation. I am hoping someone can help me out here.
We have a seven node cluster. One the nodes started to have issues but it was running. We decided to add a new node, and remove the problematic node after the new node joins. However, the new node did not join the cluster even after three days. Hence, we decided to go with the replacement option. We shutdown the problematic node. After that, we stopped cassandra on the bootstraping node, deleted all the data, and restarted that node as the replacement node for the problematic node. Since, we reused the bootstrapping node as the replacement node, I am wondering whether that is causing any issue. Any insights are appreciated. This is the output of nodetool describecluster from the replacement node, and two other nodes. mhossain@cassandra-24:~$ nodetool describecluster Cluster Information: Name: App Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Schema versions: 80649e67-8ed9-38a4-8afa-560be7c694f4: [10.0.7.80, 10.0.7.4, 10.0.7.190, 10.0.7.100, 10.0.7.195, 10.0.7.160, 10.0.7.176] mhossain@cassandra-13:~$ nodetool describecluster Cluster Information: Name: App Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Schema versions: 80649e67-8ed9-38a4-8afa-560be7c694f4: [10.0.7.80, 10.0.7.190, 10.0.7.100, 10.0.7.195, 10.0.7.160, 10.0.7.176] UNREACHABLE: [10.0.7.91, 10.0.7.4] mhossain@cassandra-09:~$ nodetool describecluster Cluster Information: Name: App Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Schema versions: 80649e67-8ed9-38a4-8afa-560be7c694f4: [10.0.7.80, 10.0.7.190, 10.0.7.100, 10.0.7.195, 10.0.7.160, 10.0.7.176] UNREACHABLE: [10.0.7.91, 10.0.7.4] cassandra-24 (10.0.7.4) is the replacement node. 10.0.7.91 is the ip address of the dead node. -Mir On Thu, Apr 21, 2016 at 10:02 AM, Mir Tanvir Hossain < mir.tanvir.hoss...@gmail.com> wrote: > Hi, I am trying to replace a dead node with by following > https://docs.datastax.com/en/cassandra/2.0/cassandra/operations/ops_replace_node_t.html. > It's been 3 full days since the replacement node started, and the node is > still not showing up as part of the cluster on OpsCenter. I was wondering > whether the delay is due to the fact that I have a test keyspace with > replication factor of one? If I delete that keyspace, would the new node > successfully replace the dead node? Any general insight will be hugely > appreciated. > > Thanks, > Mir > > >
