I did try this option and everything is working fine. Thank you Aaron. From: aa...@thelastpickle.com Subject: Re: gossip not working Date: Fri, 5 Apr 2013 23:02:58 +0530 To: user@cassandra.apache.org
Starting the node with the JVM option -Dcassandra.load_ring_state=false in cassandra-env.sh sometimes works. If not post the output from nodetool gossipinfo Cheers -----------------Aaron MortonFreelance Cassandra ConsultantNew Zealand @aaronmortonhttp://www.thelastpickle.com On 5/04/2013, at 9:38 AM, S C <as...@outlook.com> wrote:Is there a way to force gossip among the nodes? From: as...@outlook.com To: user@cassandra.apache.org Subject: RE: gossip not working Date: Thu, 4 Apr 2013 19:59:45 -0500 I am not seeing anything in the logs other than "Starting up server gossip" and there is no firewall between the nodes. From: paulsu...@gmail.com Subject: Re: gossip not working Date: Thu, 4 Apr 2013 18:49:29 -0500 To: user@cassandra.apache.org What errors are you seeing in the log files of the down nodes? Did you run upgradesstables? You need to upgradesstables when moving from < 1.1.7 to 1.1.9 On Apr 4, 2013, at 6:11 PM, S C <as...@outlook.com> wrote:I was in the middle of upgrade to 1.1.9. I brought one node with 1.1.9 while the other were running on 1.1.5. Once one of the node was on 1.1.9 it is no longer recognizing other nodes in the ring. On 192.168.56.10 and 11 192.168.56.10 DC1-Cass RAC1 Up Normal 28.06 GB 50.00% 0 192.168.56.11 DC1-Cass RAC1 Up Normal 31.59 GB 25.00% 42535295865117307932921825928971026432 192.168.56.12 DC1-Cass RAC1 Down Normal 29.02 GB 25.00% 85070591730234615865843651857942052864 On 192.168.56.12 192.168.56.10 DC1-Cass RAC1 Down Normal 28.06 GB 50.00% 0 192.168.56.11 DC1-Cass RAC1 Down Normal 31.59 GB 25.00% 42535295865117307932921825928971026432 192.168.56.12 DC1-Cass RAC1 Up Normal 29.02 GB 25.00% 85070591730234615865843651857942052864 I do not see anything in the logs that tells me that there is a gossip issue. nodetool infoToken : 85070591730234615865843651857942052864Gossip active : trueThrift active : trueLoad : 29.05 GBGeneration No : 1365114563Uptime (seconds) : 2127Heap Memory (MB) : 848.71 / 7945.94Exceptions : 0Key Cache : size 2208 (bytes), capacity 104857584 (bytes), 1056 hits, 1099 requests, 0.961 recent hit rate, 14400 save period in secondsRow Cache : size 0 (bytes), capacity 0 (bytes), 0 hits, 0 requests, NaN recent hit rate, 0 save period in seconds nodetool infoToken : 42535295865117307932921825928971026432Gossip active : trueThrift active : trueLoad : 31.59 GBGeneration No : 1364413038Uptime (seconds) : 703904Heap Memory (MB) : 733.02 / 7945.94Exceptions : 1Key Cache : size 3693312 (bytes), capacity 104857584 (bytes), 26071678 hits, 26616282 requests, 0.980 recent hit rate, 14400 save period in secondsRow Cache : size 0 (bytes), capacity 0 (bytes), 0 hits, 0 requests, NaN recent hit rate, 0 save period in seconds There is no firewall between the nodes and I can reach each other on storage port. What else should I be looking at to find root cause? Appreciate your inputs.
