So, if I have a 6 node cluster in the token ring, A->B->C->D-E->F, replication factor 3, and I run repair (without -pr) on A, is the flow of information:
A synchronizes information it is responsible for with B and C (because B and C are replicas of A). A, as a replica of E and F, synchronizes E and F's information to itself. If I re-ran repair of E, it would actually be redoing the work of synchronizing with A, but will still be doing the worthwhile work of synchronizing with F. Running repair with "-pr" will prevent this duplicate work, if you ran it on each node. If I ran repair on A with "-pr", A synchronizes its primary range with B and C, but does not perform synchronization work with E and F. Does this sound correct? -Mike On Jul 15, 2012, at 9:47 AM, Edward Capriolo wrote: > Great job sleuthing. > > Originally repair did not have a -pr. When you run the standard repair > the node compares it's data with its neighbours and vice versa. They > also send each other updates. Since you are supposed to repair every > node < gc_grace submitting a full repair to each node would create > duplicated work, since a repair on node A has an effect on node B and > node C. > > If you want to understand this some more you should run > compactionstats and netstats across your cluster while a repair is > going on, then you can see what effect the commands have on other > nodes. > > I will try to write up some documentation on it as well because -pr is > a nice feature. Many may not even be expressly aware of it. > > On Sat, Jul 14, 2012 at 2:00 PM, Michael Theroux <mthero...@yahoo.com> wrote: >> Hello, >> >> I'm looking at nodetool repair with the "-pr", vs. non "-pr" option. >> Looking around, I'm seeing a lot of conflicting information out there. >> Almost universally, the recommendation is to run nodetool repair with the >> "-pr" for any day-to-day maintenance. >> >> This is my understanding of how it works. I appreciate any corrections to >> my misinformation. >> >> nodetool repair -pr >> >> - This performs a repair on the "primary range" of the node. The primary >> range is essentially the part of the ring that the node is responsible for. >> When this command is run, synchronization of replicas will occur for the >> rows that this node is responsible for. If replicas are missing from that >> node's neighbors for those rows, they will be replicated. >> >> nodetool repair >> >> - This is where I see a lot of conflicting information. I see a lot of >> answers in which there is a suggestion that this command will perform a >> repair across the entire cluster. However, I don't believe this is true >> from my observations (and some of the items I read seems to agree with >> this). Instead, this command performs synchronization of your primary >> range, but also for other ranges that this node maybe responsible for in a >> replica capacity. The way I'm thinking about it is that the -pr option >> causes repairs to push information from its primary range to replicas. >> Without -pr, nodetool replair does a push, and pull for its neighbors that >> this node maybe a replica for. This makes sense to me, as people recommend >> running nodetool repair after a node has been down. This is to allow the >> downed node to get any missed information that should have been replicated >> to it while it was down. >> >> I'm sure there lots of flaws to the above understanding as I'm cobbling it >> together. I appreciate the feedback, >> >> -Mike
