Hi,
Why did the host ID change?
probably this node data folder (at least system keyspace) was erased. Or nodes
changed their IP, do you use dynamic IPs?
Best regards, Vladimir Yudovin,
Winguzone - Cloud Cassandra Hosting
---- On Mon, 06 Mar 2017 22:44:50 -0500 Joe Olson
<technol...@nododos.com> wrote ----
I have a 9 node cluster I had shut down (cassandra stopped on all nodes, all
nodes shutdown) that I just tried to start back up. I have done this several
times successfully. However, on this attempt, one of the nodes failed to join
the cluster. Upon inspection of /var/log/cassandra/system.log, I found the
following:
WARN [GossipStage:1] 2017-03-06 21:06:36,648 TokenMetadata.java:252 - Changing
/192.168.211.82's host ID from cff3ef25-9a47-4ea4-9519-b85d20bef3ee to
59f2da9f-0b85-452f-b61a-fa990de53e4b
further down:
ERROR [main] 2017-03-06 21:20:14,718 CassandraDaemon.java:747 - Exception
encountered during startup
java.lang.RuntimeException: A node with address /192.168.211.82 already exists,
cancelling join. Use cassandra.replace_address if you want to replace this node.
at
org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:491)
~[apache-cassandra-3.9.0.jar:3.9.0]
at
org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)
~[apache-cassandra-3.9.0.jar:3.9.0]
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)
~[apache-cassandra-3.9.0.jar:3.9.0]
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)
~[apache-cassandra-3.9.0.jar:3.9.0]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)
[apache-cassandra-3.9.0.jar:3.9.0]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)
[apache-cassandra-3.9.0.jar:3.9.0]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)
[apache-cassandra-3.9.0.jar:3.9.0]
nodetool status:
UN 192.168.211.88 2.58 TiB 256 32.0%
9de2d3ef-5ae1-4c7f-8560-730757a6d1ae rack1
UN 192.168.211.80 2.26 TiB 256 33.9%
d83829d3-a1d3-4e6c-b014-7cfe45e22d67 rack1
UN 192.168.211.81 2.91 TiB 256 34.1%
0cafd24e-d3ed-4e51-b586-0b496835a931 rack1
DN 192.168.211.82 551.45 KiB 256 31.9%
59f2da9f-0b85-452f-b61a-fa990de53e4b rack1
UN 192.168.211.83 2.32 TiB 256 32.7%
db006e31-03fa-486a-8512-f88eb583bd0c rack1
UN 192.168.211.84 2.54 TiB 256 34.3%
a9a50a74-2fc2-4866-a03a-ec95a7866183 rack1
UN 192.168.211.85 2.4 TiB 256 35.9%
733e6703-c18f-432f-a787-3731f80ba42d rack1
UN 192.168.211.86 2.34 TiB 256 32.1%
0daa06fa-708f-4ff8-a15e-861f1a53113a rack1
UN 192.168.211.87 4.07 TiB 256 33.1%
2aa578c6-1332-4b94-81c6-c3ce005a52ef rack1
My questions:
1. Why did the host ID change?
2. If I modify cassandra-env.sh to include
JVM_OPTS="$JVM_OPTS -Dcassandra.replace_address=192.168.211.82", will I recover
the data on the original node? It is still on the node's hard drive.I really
don't want to have to restream 2.6TB of data onto a "new" node.
