We have cassandra cluster in three different datacenters (DC1, DC2 and DC3)
and we have 10 machines in each datacenter. We have few tables in cassandra
in which we have less than 100 records.
What we are seeing - some tables are out of sync between machines in DC3 as
compared to DC1 or DC2 when we do select count(*) on it.
As an example we did select count(*) while connecting to one cassandra
machine in dc3 datacenter as compared to one cassandra machine in dc1
datacenter and the results were different.
root@machineA:/home/david/apache-cassandra/bin# python cqlsh
dc3114.dc3.host.com
Connected to TestCluster at dc3114.dc3.host.com:9160.
[cqlsh 2.3.0 | Cassandra 1.2.9 | CQL spec 3.0.0 | Thrift protocol
19.36.0]
Use HELP for help.
cqlsh> use testingkeyspace ;
cqlsh:testingkeyspace> select count(*) from test_metadata ;
count
-------
12
cqlsh:testingkeyspace> exit
root@machineA:/home/david/apache-cassandra/bin# python cqlsh
dc18b0c.dc1.host.com
Connected to TestCluster at dc18b0c.dc1.host.com:9160.
[cqlsh 2.3.0 | Cassandra 1.2.9 | CQL spec 3.0.0 | Thrift protocol
19.36.0]
Use HELP for help.
cqlsh> use testingkeyspace ;
cqlsh:testingkeyspace> select count(*) from test_metadata ;
count
-------
16
What could be the reason for this sync issue? Can anyone shed some light
on this?
Since our java driver code and datastax c++ driver code are using these
tables with CONSISTENCY LEVEL ONE.
