I'm running Cassandra 1.0.1 if that makes any difference.
/Henrik On Sun, Dec 11, 2011 at 13:16, Henrik Schröder <skro...@gmail.com> wrote: > I have an existing cluster of four Cassandra nodes. The machines have both > an internal and an external IP, and originally I set them up to use the > external network. A little while later I moved them to the internal network > by bringing all machines down, changing the config, and bringing them up > again. In the logs I found they all said "Changing ownership of token XXX", > and nodetool ring reported that the cluster consisted of those four > machines on their internal ips. After that, as part of a cleanup process, I > moved the tokens on all machines to make sure the cluster was balanced, and > it also worked perfectly. > > However, now I have to temporarily move the cluster back to the external > network for a little while. I tried doing the same thing as last time, > bringing all nodes down, changing the config (rpc address, gossip address, > list of seeds) and bringing them up again, but this resulted in a very > confused cluster. When I ran nodetool ring, it reported eight nodes, the > four internal ips were marked as down, and the four external were marked as > up, but with the token they had when they previously used that ip. Checking > the logs, there was no token ownership change, all nodes picked the saved > token they had when they last used the external ip, and not the token they > should have, the one I moved each server to when on the internal ip. > > I immediately moved all servers back to the internal IP, and then nodetool > reported the same as before, a cluster of four machines, all up, and all on > the token they're supposed to have. No mention of the external ips or the > old tokens they had there. > > How do I reset this data? Where is it stored? Why does it store all of > this when nodetool doesn't report it? Why does a node store several saved > tokens? How do I change their ip without losing any data and without having > to do removetoken or similar? > > One thought I have is to bring down one node, delete the system keyspace, > and bring it back up, at which point it would only use what's in the > config, but fetch the schema from the other nodes. Or would it also fetch > the old information of what token it had when it was on the external ip? Or > would something else go wrong? > > > /Henrik >
