No. I stopped the stress test before issuing decommission command. So, it was
not under ANY load.
I waited for over an hour and nothing changed.
Then , I turned on DEBUG in the log4j-server.properties and then restarted the
Cassandra process .
As soon as I restarted, the decommissioned node left the cluster and everything
was back to normal.
Have you seen this behaviour before?
From: Jonathan Colby
Sent: Tuesday, April 12, 2011 3:15 PM
To: user@cassandra.apache.org
Subject: Re: Help on decommission
how long as it been in Leaving status? Is the cluster under stress test load
while you are doing the decommission?
On Apr 12, 2011, at 6:53 PM, Baskar Duraikannu wrote:
I have setup a 4 node cluster for testing. When I setup the cluster, I have
setup initial tokens in such a way that each gets 25% of load and then started
the node with autobootstrap=false.
After all nodes are up, I loaded data using the stress test tool with
replication factor of 3. As per of my testing, I am trying to remove one of
the node using "nodetool decomission" but the node seems to be stuck in
"leaving" status.
How do I check whether it is doing any work at all? Please help
[root@localhost bin]# ./nodetool -h 10.140.22.25 ring
Address Status State Load Owns Token
127605887595351923798765477786913079296
10.140.22.66 Up Leaving 119.41 MB 25.00% 0
10.140.22.42 Up Normal 116.23 MB 25.00%
42535295865117307932921825928971026432
10.140.22.28 Up Normal 119.93 MB 25.00%
85070591730234615865843651857942052864
10.140.22.25 Up Normal 116.21 MB 25.00%
127605887595351923798765477786913079296
[root@localhost bin]# ./nodetool -h 10.140.22.66 netstats
Mode: Leaving: streaming data to other nodes
Streaming to: /10.140.22.42
/var/lib/cassandra/data/Keyspace1/Standard1-f-1-Data.db/(0,120929157)
progress=120929157/120929157 - 100%
/var/lib/cassandra/data/Keyspace1/Standard1-f-2-Data.db/(0,3361291)
progress=0/3361291 - 0%
Not receiving any streams.
Pool Name Active Pending Completed
Commands n/a 0 17
Responses n/a 0 108109
[root@usnynyc1cass02 bin]# ./nodetool -h 10.140.22.42 netstats
Mode: Normal
Not sending any streams.
Streaming from: /10.140.22.66
Keyspace1:
/var/lib/cassandra/data/Keyspace1/Standard1-f-2-Data.db/(0,3361291)
progress=0/3361291 - 0%
Pool Name Active Pending Completed
Commands n/a 0 11
Responses n/a 0 107879
Regards,
Baskar
