thank you Ben. We are using cassandra 2.1.12 version. We did face the bug mentioned https://issues.apache.org/jira/browse/CASSANDRA-10371 in DSE 4.6.7, in another cluster. It's strange we are seeing that even in cassandra 2.1.12 version.
The "nodetool describecluster" showing decommissioned nodes as UNREACHABLE is something we are seeing for the first time. thanks Sai On Wed, Feb 17, 2016 at 12:36 PM, Ben Bromhead <b...@instaclustr.com> wrote: > I'm not sure what version of Cassandra you are running so here is some > general advice: > > - Gossip entries for decommissioned nodes will hang around for a few > days to help catch up nodes in the case of a partition. This is why you see > the decommissioned nodes listed as LEFT. This is intentional > - If you keep seeing those entries in your logs and you are on 2.0.x, > you might be impacted by > https://issues.apache.org/jira/browse/CASSANDRA-10371. In this case > upgrade to 2.1 or you can try the work arounds listed in the ticket. > > Ben > > On Tue, 16 Feb 2016 at 11:09 sai krishnam raju potturi < > pskraj...@gmail.com> wrote: > >> hi; >> we have a 12 node cluster across 2 datacenters. We are currently >> using cassandra 2.1.12 version. >> >> SNITCH : GossipingPropertyFileSnitch >> >> When we decommissioned few nodes in a particular datacenter and observed >> the following : >> >> nodetool status shows only the live nodes in the cluster. >> >> nodetool describecluster shows the decommissioned nodes as UNREACHABLE. >> >> nodetool gossipinfo shows the decommissioned nodes as "LEFT" >> >> >> When the live nodes were restarted, "nodetool describecluster" shows >> only the live nodes, which is expected. >> >> Purging the gossip info too did not help. >> >> INFO 17:27:07 InetAddress /X.X.X.X is now DOWN >> INFO 17:27:07 Removing tokens [125897680671740685543105407593050165202, >> 140213388002871593911508364312533329916, >> 98576967436431350637134234839492449485] for /X.X.X.X >> INFO 17:27:07 InetAddress /X.X.X.X is now DOWN >> INFO 17:27:07 Removing tokens [11116977666116265389022494863106850615, >> 111270759969411259938117902792984586225, >> 138611464975439236357814418845450428175] for /X.X.X.X >> >> Has anybody experienced similar behaviour. Restarting the entire cluster, >> everytime a node is decommissioned does not seem right. Thanks in advance >> for the help. >> >> >> thanks >> Sai >> >> >> -- > Ben Bromhead > CTO | Instaclustr <https://www.instaclustr.com/> > +1 650 284 9692 > Managed Cassandra / Spark on AWS, Azure and Softlayer >
