Using Cassandra 2.2.4 on Ubuntu.
We have a cluster with two nodes that during several hours failed to connect
with each other due to network problems. The database continued to be used in
one of the nodes with writes being stored in the Hints file as supposed.
But now that the network is OK again and each machine can communicate we see
that each node indicates the other is DOWN and does not replicates.
When the network came up we started to see in log files "Convicting
/192.168.1.102 with status NORMAL - alive false"
It seems each node evictions each other and later failing to reconnect.
Is there some configuration that we might be missing ? Any help would be much
appreciated.
- NODE 192.168.1.10 - "nodetool status”
Datacenter: DC1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID
Rack
DN 192.168.1.102 12.02 MB 256 ?
ff906882-8224-40ac-8cdb-98f5e725814d rack1
Datacenter: DC2
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID
Rack
UN 192.168.1.10 41.87 MB 256 ?
51650afd-84dd-4e25-a6f0-13627858d5dc rack1
- NODE 192.168.1.102 - “nodetool status"
Datacenter: DC1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID
Rack
UN 192.168.1.102 12.4 MB 256 ?
ff906882-8224-40ac-8cdb-98f5e725814d rack1
Datacenter: DC2
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID
Rack
DN 192.168.1.10 26.31 MB 256 ?
51650afd-84dd-4e25-a6f0-13627858d5dc rack1
