Order is decommission, remove, assassinate. Which have you tried? On Sep 21, 2015 10:47 AM, "Dikang Gu" <dikan...@gmail.com> wrote:
> Hi there, > > I have a dead node in our cluster, which is a wired state right now, and > can not be removed from cluster. > > The nodestatus shows: > Datacenter: DC1 > =============== > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- Address Load Tokens Owns Host ID > Rack > DN 10.210.165.55 ? 256 ? null > r1 > > I tried the unsafeAssassinateEndpoint, but got exception like: > 2015-09-18_23:21:40.79760 INFO 23:21:40 InetAddress /10.210.165.55 is > now DOWN > 2015-09-18_23:21:40.80667 ERROR 23:21:40 Exception in thread > Thread[GossipStage:1,5,main] > 2015-09-18_23:21:40.80668 java.lang.NullPointerException: null > 2015-09-18_23:21:40.80669 at > org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1584) > ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] > 2015-09-18_23:21:40.80669 at > org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1592) > ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] > 2015-09-18_23:21:40.80670 at > org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1822) > ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] > 2015-09-18_23:21:40.80671 at > org.apache.cassandra.service.StorageService.onChange(StorageService.java:1495) > ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] > 2015-09-18_23:21:40.80671 at > org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2121) > ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] > 2015-09-18_23:21:40.80672 at > org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1009) > ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] > 2015-09-18_23:21:40.80673 at > org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1113) > ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] > 2015-09-18_23:21:40.80673 at > org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49) > ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] > 2015-09-18_23:21:40.80673 at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62) > ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] > 2015-09-18_23:21:40.80674 at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > ~[na:1.7.0_45] > 2015-09-18_23:21:40.80674 at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > ~[na:1.7.0_45] > 2015-09-18_23:21:40.80674 at java.lang.Thread.run(Thread.java:744) > ~[na:1.7.0_45] > 2015-09-18_23:21:40.85812 WARN 23:21:40 Not marking nodes down due to > local pause of 10852378435 > 5000000000 > > Any suggestions about how to remove it? > Thanks. > > -- > Dikang > >
