Those hosts are likely sending streams. If you do `nodetool netstats` on the replicas of the node you're removing, you should see byte counters and file counters - they should all be incrementing. If one of them isnt incremening, that one is probably stuck.
There's at least one bug in 4.1 that can cause (I think? rate limiters) to interact in a way that can cause this. https://issues.apache.org/jira/browse/CASSANDRA-18110 describes it and has a workaround. On Mon, Jan 23, 2023 at 9:41 AM Joe Obernberger < joseph.obernber...@gmail.com> wrote: > I had a drive fail (first drive in the list) on a Cassandra cluster. > I've stopped the node (as it no longer starts), and am trying to remove > it from the cluster, but the removenode command is hung (been running > for 3 hours so far): > nodetool removenode status is always reporting the same token as being > removed. Help? > > nodetool removenode status > RemovalStatus: Removing token (-9196617215347134065). Waiting for > replication confirmation from > [/172.16.100.248,/172.16.100.249,/172.16.100.251,/172.16.100.252,/ > 172.16.100.34,/172.16.100.35,/172.16.100.36,/172.16.100.37,/172.16.100.38 > ,/172.16.100.42,/172.16.100.44,/172.16.100.45]. > > Thanks. > > -Joe > > > -- > This email has been checked for viruses by AVG antivirus software. > www.avg.com >
