> above. From looking at the data I'm guessing that the results from the 3 > nodes are correct and the results from the 2 nodes are old (the diff between > the result sets is that the 54 is a subset of the 68).
If I interpret the thread correctly, those 2 that you say you believe are old are the ones that are returning extra results. So, returning results that should no longer be there. That suggests to me that you are seeing old values that have since been deleted (unless you're seeing some kind of arbitrary random data popping up). Is it possible that you're not running 'nodetool repair' frequently enough with respect to GCGraceSeconds? The negative results of that would be that deletions are essentially forgotten if they don't reach all nodes in time. Have you so far only seen incorrect results in the form of additional data that should be deleted, or have you also seen old versions of columns? (That refuse to self-heal, that is.) Now, GCGraceSeconds/repair issues would not explain, to me at least, why read-repair is not fixing the discrepancy. I don't think I saw you explicitly confirm it, so I'll ask: Are you indeed running with read repair turned on? If it is turned off, and if all your discrepancies are in the form of forgotten deletes, then GCGraceSeconds/repair seems like a likely candidate cause. -- / Peter Schuller
