Glad to report I fixed this problem.
1. I added the load_ring_state=false flag
2. I was able to arrange a time where I could take down the whole
cluster and bring it back up.
After that the "phantom" node disappeared.
On Fri, May 27, 2011 at 12:48 AM, wrote:
> Hi Aaron - Thanks alot for the gre
Hi Aaron - Thanks alot for the great feedback. I'll try your suggestion on
removing it as an endpoint with jmx.
On , aaron morton wrote:
Off the top of my head the simple way to stop invalid end point state
been passed around is a full cluster stop. Obviously thats not an option.
The probl
Off the top of my head the simple way to stop invalid end point state been
passed around is a full cluster stop. Obviously thats not an option. The
problem is if one node has the IP is will share it around with the others.
Out of interest take a look at the o.a.c.db.FailureDetector MBean
getA
@Aaron -
Unfortunately I'm still seeing message like: " is down",
removing from gossip, although with not the same frequency.
And repair/move jobs don't seem to try to stream data to the removed node
anymore.
Anyone know how to totally purge any stored gossip/endpoint data on nodes that
we
cool. I was going to suggest that but as you already had the move running I
thought it may be a little drastic.
Did it show any progress ? If the IP address is not responding there should
have been some sort of error.
Cheers
-
Aaron Morton
Freelance Cassandra Developer
@aaro
Seems like it had something to do with stale endpoint information. I did a
rolling restart of the whole cluster and that seemed to trigger the nodes
to remove the node that was decommissioned.
On , aaron morton wrote:
Is it showing progress ? It may just be a problem with the information
p
Is it showing progress ? It may just be a problem with the information printed
out.
Can you check from the other nodes in the cluster to see if they are receiving
the stream ?
cheers
-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 26
I recently removed a node (with decommission) from our cluster.
I added a couple new nodes and am now trying to rebalance the cluster using
nodetool move.
However, netstats shows that the node being "moved" is trying to stream data
to the node that I already decommissioned yesterday.
The remo
