Hi When I sent the mail I'd had the new node on for about an hour, the old node died about an hour before that. The weirdness in the log files stopped yesterday afternoon, about 4 or 5 hours after I replaced the node so it seems to have resolved itself. Seeing as there's no error to look at in the log files not sure if you still want the output of my gossipinfo but I've pasted it below anyway. Thanks!
/10.16.96.212 LOAD:7.8018521345E10 RPC_ADDRESS:0.0.0.0 RELEASE_VERSION:1.0.11 STATUS:NORMAL,113427455640312821154458202477256070484 SCHEMA:9b152e00-fd90-11e1-0000-2d22988ca597 /10.16.128.211 LOAD:7.8416250275E10 RPC_ADDRESS:0.0.0.0 RELEASE_VERSION:1.0.11 STATUS:NORMAL,85070591730234615865843651857942052863 SCHEMA:9b152e00-fd90-11e1-0000-2d22988ca597 /10.16.32.210 LOAD:1.29054735121E11 RPC_ADDRESS:0.0.0.0 RELEASE_VERSION:1.0.11 STATUS:NORMAL,56713727820156407428984779325531226112 SCHEMA:9b152e00-fd90-11e1-0000-2d22988ca597 /10.16.32.211 LOAD:7.2937725831E10 RPC_ADDRESS:0.0.0.0 RELEASE_VERSION:1.0.11 STATUS:NORMAL,141784319550391032739561396922763706368 SCHEMA:9b152e00-fd90-11e1-0000-2d22988ca597 ip-10-16-128-197.localdomain/10.16.128.197 LOAD:6.5571879526E10 RPC_ADDRESS:0.0.0.0 RELEASE_VERSION:1.0.11 STATUS:NORMAL,0 SCHEMA:9b152e00-fd90-11e1-0000-2d22988ca597 /10.16.96.211 LOAD:1.0633383453E11 RPC_ADDRESS:0.0.0.0 RELEASE_VERSION:1.0.11 STATUS:NORMAL,28356863910078203714492389662765613056 SCHEMA:9b152e00-fd90-11e1-0000-2d22988ca597 On Fri, Oct 19, 2012 at 2:56 AM, aaron morton <aa...@thelastpickle.com>wrote: > I replaced it with a new node, IP 10.16.128.197 and again token 0 with a > "-Dcassandra.replace_token=0" at startup > > Good Good. > > How long ago did you bring the new node on ? There is a fail safe to > remove 128.210 after 3 days if it does not gossip to other nodes. > > I *thought* that remove_token would remove the old IP from the ring. Can > you post the output from nodetool gossipinfo from the 128.197 node ? > > Thanks > > ----------------- > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 19/10/2012, at 2:44 AM, Thomas van Neerijnen <t...@bossastudios.com> > wrote: > > Hi all > > I'm running Cassandra 1.0.11 on Ubuntu 11.10. > > I've got a ghost node which keeps showing up on my ring. > > A node living on IP 10.16.128.210 and token 0 died and had to be replaced. > I replaced it with a new node, IP 10.16.128.197 and again token 0 with a > "-Dcassandra.replace_token=0" at startup. This all went well but now I'm > seeing the following weirdness constantly reported in the log files around > the ring: > > INFO [GossipTasks:1] 2012-10-18 13:39:22,441 Gossiper.java (line 632) > FatClient /10.16.128.210 has been silent for 30000ms, removing from gossip > INFO [GossipStage:1] 2012-10-18 13:40:25,933 Gossiper.java (line 838) > Node /10.16.128.210 is now part of the cluster > INFO [GossipStage:1] 2012-10-18 13:40:25,934 Gossiper.java (line 804) > InetAddress /10.16.128.210 is now UP > INFO [GossipStage:1] 2012-10-18 13:40:25,937 StorageService.java (line > 1017) Nodes /10.16.128.210 and /10.16.128.197 have the same token 0. > Ignoring /10.16.128.210 > INFO [GossipTasks:1] 2012-10-18 13:40:37,509 Gossiper.java (line 818) > InetAddress /10.16.128.210 is now dead. > INFO [GossipTasks:1] 2012-10-18 13:40:56,526 Gossiper.java (line 632) > FatClient /10.16.128.210 has been silent for 30000ms, removing from gossip > > >
