I have a 3 nodes cluster, replicate_factor is 3 also. Consistency level is
Write quorum, Read quorum.
Traffic has three major steps
Create:
Rowkey: xxxx
Column: status=new, requests="xxxxx"
Update:
Rowkey: xxxx
Column: status=executing, requests="xxxxx"
Delete:
Rowkey: xxxx
When one node down, it can work according to consistency configuration, and
the final status is all requests are finished and delete.
So if running cassandra client to list the result (also set consistency
quorum). It shows empty (only rowkey left)， which is correct.
But if we start the dead node, the hinted handoff model will write back the
data to this node. So there are lots of create, update, delete.
I don't know due to GC or compaction, the delete records on other two nodes
seems not work, and if using cassandra client to list the data (also
consistency quorum), the deleted row show again with column value.
And if using client to check the data several times, you can find the data
is changed, seems hinted handoff replay operation, the deleted data show up
and then disappear.
So the hinted handoff mechanism will faster the repair, but the temporary
data will be seen from external (if data is deleted).
Is there a way to have this procedure invisible from external, until the
hinted handoff finished?
What I want is final status synchronization, the temporary status is out of
date and also incorrect, should never been seen from external.
Is it due to row delete instead of column delete? Or compaction?
