Re: unreachable nodes mystery in describecluster output

2016-08-03 Thread Aleksandr Ivanov
there a VPN between DCs? Is there room for improvement at the network > level? TCP tuning, etc. I'm not saying you won't have unreachable nodes but > it's worth it if you can. > > Romain > > Le Mercredi 3 août 2016 15h02, Aleksandr Ivanov a > écrit : >

Re: unreachable nodes mystery in describecluster output

2016-08-03 Thread Romain Hardouin
That's a good news if describecluster shows the same version on each node. Try with a high timeout like 120 seconds to see if it works. Is there a VPN between DCs? Is there room for improvement at the network level? TCP tuning, etc. I'm not saying you won't have unreachable nodes

Re: unreachable nodes mystery in describecluster output

2016-08-03 Thread Aleksandr Ivanov
> I use 25sec timeout (--request-timeout 25) Apart the unreachable nodes, do you know if all nodes have the same schema > version? > "nodetool gossipinfo" shows same scheme version on all nodes. Best, > > Romain >

Re: unreachable nodes mystery in describecluster output

2016-08-03 Thread Romain Hardouin
Hi, The latency is high... Regarding the ALTER, did you try to increase the timeout with "cqlsh --request-timeout=REQUEST_TIMEOUT"? Because the default is 10 seconds. Apart the unreachable nodes, do you know if all nodes have the same schema version?  Best, Romain

unreachable nodes mystery in describecluster output

2016-08-03 Thread Aleksandr Ivanov
Hello, I'm running v3.0.8 in multi-data center deployment (6 DCs, 6 nodes per DC, maximum latency between some nodes ~200ms). After clean cluster start I run into issue when "nodetool descibecluster" shows that some random nodes from deployment are UNREACHABLE however in "nodetool status" or "node

Re: Unreachable Nodes

2013-05-22 Thread Alain RODRIGUEZ
I had to face this too, but precisely the "unsafeAssassinateEndpoint" removed the "UNREACHABLE" nodes (from describe cluster - CLI). After that, I had these ghost host marked as "STATUS:LEFT" on gossipinfo (nodetool) and my truncate could run properly. But this is o

Re: Unreachable Nodes

2013-05-22 Thread Brian Tarbox
hat's a good thing. Gossiper just keep this information for a while (7 >> or 10 days by default off the top off my head), but this doesn't harm your >> cluster in any ways, but having "UNREACHABLE" nodes could have been >> annoying. By the way gossipinfo shows

Re: Unreachable Nodes

2013-05-22 Thread Vasileios Vlachos
ood thing. Gossiper just keep this information for a while (7 or > 10 days by default off the top off my head), but this doesn't harm your > cluster in any ways, but having "UNREACHABLE" nodes could have been > annoying. By the way gossipinfo shows you t

Re: Unreachable Nodes

2013-05-22 Thread Alain RODRIGUEZ
when we run "describe cluster" from the CLI?" That's a good thing. Gossiper just keep this information for a while (7 or 10 days by default off the top off my head), but this doesn't harm your cluster in any ways, but having "UNREACHABLE" nodes could have been anno

Unreachable Nodes

2013-05-22 Thread Vasileios Vlachos
Hello All, A while ago we had 3 cassandra nodes on Amazon. At some point we decided to buy some servers and deploy cassandra there. The problem is that since then we have a list of dead IPs listed as UNREACHABLE nodes when we run describe cluster on cassandra-cli. I have seen other posts which