Hi,
I Have a 3 node cluster running Cassandra 1.0.3 and using replication
factor=3.
Recently I've noticed that some previously deleted rows have started to
reappear for some reason. And now I wonder if this is a known issue with
1.0.3?
Repairs have been running every weekend (gc_grace is 10 days) and always
completed successfully. But while looking at the logs I noticed that a
fair number of ranges (around 10% of the total number of keys) have been
streamed between these nodes during the repair sessions. This seems a
bit high to me given that everything is written using quorum and all
nodes have been up all the time.
For me this looks suspiciously like some already deleted keys are
streamed to other nodes during repair.
Some more details about the data:
All keys are written to only once and most of them are deleted a couple
of days/weeks later. Some keys are large enough to require incremental
compaction.
Could this bug cause this?
https://issues.apache.org/jira/browse/CASSANDRA-3510
Regards,
Jonas