I got that a couple of time (due to DNS issues in our infra) what you could try - check in cassandra log. It is possible you see a log line telling you 10.56.62.211 and 10.59.21.241 o 10.58.83.109 share the same token - if 10.56.62.211 is up, try decommission (via nodetool) - if not, move 10.59.21.241 or 10.58.83.109 to current token + 1 - use removetoken (via nodetool) to remove the token associated with 10.56.62.211. in case of failure, you can use removetoken -f instead.
then, the unreachable IP should have disappeared. HTH On Thu, Jul 19, 2012 at 10:38 AM, Alain RODRIGUEZ <arodr...@gmail.com>wrote: > Hi, > > I tried to add a node a few days ago and it failed. I finally made it > work with an other node but now when I describe cluster on cli I got > this : > > Cluster Information: > Snitch: org.apache.cassandra.locator.Ec2Snitch > Partitioner: org.apache.cassandra.dht.RandomPartitioner > Schema versions: > UNREACHABLE: [10.56.62.211] > e7e0ec6c-616e-32e7-ae29-40eae2b82ca8: [10.59.21.241, 10.58.83.109] > > And nodetool ring gives me : > > Address DC Rack Status State Load > Owns Token > > 85070591730234615865843651857942052864 > 10.59.21.241 eu-west 1b Up Normal 101.17 GB > 50.00% 0 > 10.58.83.109 eu-west 1b Up Normal 55.27 GB > 50.00% 85070591730234615865843651857942052864 > > The point, as you can see, is that one of my node has twice the > information of the second one. I have a RF = 2 defined. > > My guess is that the token 0 node keep data for the unreachable node. > > The IP of the unreachable node doesn't belong to me anymore, I have no > access to this ghost node. > > Does someone know how to completely remove this ghost node from my cluster > ? > > Thank you. > > Alain > > INFO : > > On ubuntu (AMI Datastax 2.1 and 2.2) > Cassandra 1.1.2 (upgraded from 1.0.9) > 2 node cluster (+ the ghost one) > RF = 2 > -- ............................................................ Olivier Mallassi OCTO Technology ............................................................ 50, Avenue des Champs-Elysées 75008 Paris Mobile: (33) 6 28 70 26 61 Tél: (33) 1 58 56 10 00 Fax: (33) 1 58 56 10 01 http://www.octo.com Octo Talks! http://blog.octo.com
