Starting the node with the JVM option -Dcassandra.load_ring_state=false in cassandra-env.sh sometimes works.
If not post the output from nodetool gossipinfo Cheers ----------------- Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 5/04/2013, at 9:38 AM, S C <as...@outlook.com> wrote: > Is there a way to force gossip among the nodes? > > From: as...@outlook.com > To: user@cassandra.apache.org > Subject: RE: gossip not working > Date: Thu, 4 Apr 2013 19:59:45 -0500 > > I am not seeing anything in the logs other than "Starting up server gossip" > and there is no firewall between the nodes. > From: paulsu...@gmail.com > Subject: Re: gossip not working > Date: Thu, 4 Apr 2013 18:49:29 -0500 > To: user@cassandra.apache.org > > What errors are you seeing in the log files of the down nodes? Did you run > upgradesstables? You need to upgradesstables when moving from < 1.1.7 to 1.1.9 > > On Apr 4, 2013, at 6:11 PM, S C <as...@outlook.com> wrote: > > I was in the middle of upgrade to 1.1.9. I brought one node with 1.1.9 while > the other were running on 1.1.5. Once one of the node was on 1.1.9 it is no > longer recognizing other nodes in the ring. > > On 192.168.56.10 and 11 > > 192.168.56.10 DC1-Cass RAC1 Up Normal 28.06 GB 50.00% > 0 > 192.168.56.11 DC1-Cass RAC1 Up Normal 31.59 GB 25.00% > 42535295865117307932921825928971026432 > 192.168.56.12 DC1-Cass RAC1 Down Normal 29.02 GB 25.00% > 85070591730234615865843651857942052864 > > > On 192.168.56.12 > > 192.168.56.10 DC1-Cass RAC1 Down Normal 28.06 GB > 50.00% 0 > 192.168.56.11 DC1-Cass RAC1 Down Normal 31.59 GB > 25.00% 42535295865117307932921825928971026432 > 192.168.56.12 DC1-Cass RAC1 Up Normal 29.02 GB 25.00% > 85070591730234615865843651857942052864 > > > I do not see anything in the logs that tells me that there is a gossip issue. > > nodetool info > Token : 85070591730234615865843651857942052864 > Gossip active : true > Thrift active : true > Load : 29.05 GB > Generation No : 1365114563 > Uptime (seconds) : 2127 > Heap Memory (MB) : 848.71 / 7945.94 > Exceptions : 0 > Key Cache : size 2208 (bytes), capacity 104857584 (bytes), 1056 hits, > 1099 requests, 0.961 recent hit rate, 14400 save period in seconds > Row Cache : size 0 (bytes), capacity 0 (bytes), 0 hits, 0 requests, > NaN recent hit rate, 0 save period in seconds > > nodetool info > Token : 42535295865117307932921825928971026432 > Gossip active : true > Thrift active : true > Load : 31.59 GB > Generation No : 1364413038 > Uptime (seconds) : 703904 > Heap Memory (MB) : 733.02 / 7945.94 > Exceptions : 1 > Key Cache : size 3693312 (bytes), capacity 104857584 (bytes), 26071678 > hits, 26616282 requests, 0.980 recent hit rate, 14400 save period in seconds > Row Cache : size 0 (bytes), capacity 0 (bytes), 0 hits, 0 requests, > NaN recent hit rate, 0 save period in seconds > > > > There is no firewall between the nodes and I can reach each other on storage > port. > What else should I be looking at to find root cause? Appreciate your inputs.
