Hi Ben, Just for my understanding :
When you said : What we really want to do is apply: LeaderThrottle [104,107,109] FollowerThrottle [105,113] ---> This means that we want to apply leader throttle to only replica that is the leader out of 104, 107, 109 right? Also when you said : [104,107,109,105,113] which will mean the regular replication traffic between (say 107 is leader) 107->104 and 107->109 will be throttled also. --- > By regular traffic you mean traffic for partition 0, right? Also, the only reason 107->104 and 107->109 would get throttled is because of follower throttle right? Thanks, Mayuresh On Mon, Sep 26, 2016 at 7:13 PM, Neha Narkhede <n...@confluent.io> wrote: > Makes sense to me. Thanks Ben! > > On Mon, Sep 26, 2016 at 9:39 AM Jun Rao <j...@confluent.io> wrote: > > > Yes, this change makes sense to me since it gives the admin better > control. > > > > Thanks, > > > > Jun > > > > On Sun, Sep 25, 2016 at 7:35 AM, Ben Stopford <b...@confluent.io> wrote: > > > > > Hi All > > > > > > We’ve made an adjustment to KIP-73: Replication Quotas which I’d like > to > > > open up to the community for approval. > > > > > > Previously the admin passed a list of replicas to be throttled: > > > > > > quota.replication.throttled.replicas = > > > [partId]:[replica],[partId]:[replica],[partId]:[replica] etc > > > > > > The change is to split this into two properties. One that corresponds > to > > > the leader-side throttle, and the other that corresponds to the > > > follower-side throttle. > > > > > > quota.leader.replication.throttled.replicas = > > > [partId]:[replica],[partId]:[replica],[partId]:[replica] > > > quota.follower.replication.throttled.replicas = > > > [partId]:[replica],[partId]:[replica],[partId]:[replica] > > > > > > This provides more control over the throttling process. It also helps > us > > > with a common use case which I’ve described below, for those > interested. > > > > > > Please reply if you have any comments / issues / suggestions. > > > > > > Thanks as ever. > > > > > > Ben > > > > > > > > > Sample Use Case: > > > > > > Say we are moving partition 0. It has replicas [104,107,109] moving to > > > [105,107,113] > > > > > > So the leader could be any of [104,107,109] and we know we will be > > > creating new replicas on 105 & 113. > > > > > > In the current mechanism, all we can do is apply both leader and > follower > > > throttles to all 5: [104,107,109,105,113] which will mean the regular > > > replication traffic between (say 107 is leader) 107->104 and 107->109 > > will > > > be throttled also. This is potentially problematic. > > > > > > What we really want to do is apply: > > > > > > LeaderThrottle [104,107,109] > > > FollowerThrottle [105,113] > > > > > > This way, during a rebalance, we that standard replication traffic will > > > not be throttled, but the rebalance will perform correctly if leaders > > > move. One subtlety is that, should the leader move to the “move > > > destination” node, it would no longer be throttled. But this is > actually > > to > > > our benefit in the normal case. > > > > > > > > > -- > Thanks, > Neha > -- -Regards, Mayuresh R. Gharat (862) 250-7125
