On a 2-node cluster with replication factor 2, I have a column family with an index on one of the columns.
Every now and then, I notice that a lookup of the record through the index on node 1 produces the record, but the same lookup on node 2 does not! If I do a lookup by row key, the record is found, and the indexed value is there. So as far as I can tell, the index on one of the nodes looses values, and is no longer in sync with the other node, even though the replication factor requires it. I typically repair these issues by storing the indexed column value again. The indexed data is static data; it doesn't change. I'm running cassandra 1.2.3. I'm running a nodetool repair on each node every day (although this does not fix this problem). This problem worries me a lot. I don't have a clue about the cause of it. Any help would be greatly appreciated. Tom