Re: Node after restart sees other nodes down for 10 minutes

2016-07-27 Thread Paulo Motta
> This sounds like an issue that can potentially affect many users. Is it not the case? This seems to affect only some configurations, specially EC2, but not all for some reason (it might be related to default tcp timeout configuration). > Do we have a solution for this? Watch https://issues.apa

Re: Node after restart sees other nodes down for 10 minutes

2016-07-27 Thread Farzad Panahi
Paulo, I can confirm that the problem is as you stated. Some or all of the other nodes are keeping a connection in CLOSE_WAIT state. Those nodes are seen as DN from the point of the node I have restarted the Cassandra service on. But nodetool disablegossip did not fix the problem. This sounds lik

Re: Node after restart sees other nodes down for 10 minutes

2016-07-27 Thread Farzad Panahi
Thanks Paulo for the reply. Cassandra version is 3.0.8. I will test what you said and share the results. On Wed, Jul 27, 2016 at 2:01 PM, Paulo Motta wrote: > This looks somewhat related to CASSANDRA-9630. What is the C* version? > > Can you check with netstats if other nodes keep connections w

Re: Node after restart sees other nodes down for 10 minutes

2016-07-27 Thread Paulo Motta
This looks somewhat related to CASSANDRA-9630. What is the C* version? Can you check with netstats if other nodes keep connections with the stopped node in the CLOSE_WAIT state? And also if the problem disappears if you run nodetool disablegossip before stopping the node? 2016-07-26 16:54 GMT-03:

Node after restart sees other nodes down for 10 minutes

2016-07-26 Thread Farzad Panahi
I am new to Cassandra and trying to figure out how the cluster behaves when things go south. I have a 6-node cluster, RF=3. I stop Cassandra service on a node for a while. All nodes see the node as DN. After a while I start the Cassandra service on DN. Interesting point is that all other nodes se