Hmm, we considered that option but if the old node is assassinated, his range will be assigned to a neighbor that doesn't have the data, what will cause empty reads. What we did to solve the problem was to do a safe removal via "nodetool removenode <deadNodeId>", wait some hours for neighbors to stream that node's data, and then bootstrap the replacement node. However this procedure takes double the time, because data needs to be streamed twice, which is not really optimal.
It would be really nice to know if this is expected behavior of if I should fill a bug request. On Fri, Mar 14, 2014 at 11:59 AM, Rahul Menon <ra...@apigee.com> wrote: > Since the older node is not available i would ask you to assassinate the > old node and then get the node new node to bootstrap. > > > On Thu, Mar 13, 2014 at 10:56 PM, Paulo Ricardo Motta Gomes < > paulo.mo...@chaordicsystems.com> wrote: > >> Yes, exactly. >> >> >> On Thu, Mar 13, 2014 at 1:27 PM, Rahul Menon <ra...@apigee.com> wrote: >> >>> And the token value as suggested is tokenvalueoddeadnode-1 ? >>> >>> >>> On Thu, Mar 13, 2014 at 9:29 PM, Paulo Ricardo Motta Gomes < >>> paulo.mo...@chaordicsystems.com> wrote: >>> >>>> Nope, they have different IPs. I'm using the procedure described here >>>> to replace a dead node: >>>> http://www.datastax.com/docs/1.1/cluster_management#replacing-a-dead-node >>>> >>>> Dead node token: X (IP: Y) >>>> Replacement node token: X-1 (IP: Z) >>>> >>>> So, as soon as the replacement node (Z) is started, it sees the dead >>>> node (Y) as UP, and tries to stream data from it during the join process. >>>> About 10 minutes later, the failure detector of Z detects Y as down, but >>>> since it was trying to fetch data from him, it fails the join/bootstrap >>>> process altogether. >>>> >>>> >>> >> >> >> -- >> *Paulo Motta* >> >> Chaordic | *Platform* >> *www.chaordic.com.br <http://www.chaordic.com.br/>* >> +55 48 3232.3200 >> +55 83 9690-1314 >> > > -- *Paulo Motta* Chaordic | *Platform* *www.chaordic.com.br <http://www.chaordic.com.br/>* +55 48 3232.3200 +55 83 9690-1314
