Hi Harry Fallows, Thanks for the KIP!
I went over both the KIP-1051 and KIP-1009. Assuming that the leader.replication.throttled.replicas and follower.replication.throttled.replicas are set to Wildcard (*) to apply for all the partitions in the broker. If we set a static value for leader and follower replication throttled rate, then it might impact the normal replication traffic. Throttling rate depends on the number of brokers in the cluster. If the cluster contains 100+ brokers, then the leader.replication.throttled.rate is shared across all the followers. The number of followers reading data from the leader depends on the partition assignment strategy. If the leader replication throttle is breached, then the follower might fail to catch-up with the leader. If there are sudden spikes in a specific set of topics/partitions in the cluster, then the replicas might fail to join the isr and can impact the cluster reliability. If we are going with this proposal, then we may also have to emit a metric to inform the administrator that the leader/follower replication quota is breached. -- Kamal On Thu, Jul 4, 2024 at 8:10 PM Harry Fallows <harryfall...@protonmail.com.invalid> wrote: > Hi everyone, > > Bumping this one last time before I call a vote. Please take a look if > you're interested in replication throttling and/or static/dynamic config. > > Kind regards, > Harry > > On Thursday, 13 June 2024 at 19:39, Harry Fallows < > harryfall...@protonmail.com.INVALID> wrote: > > > Hi Hector, > > > > I did see your colleague's KIP, and I actually mentioned it in the KIP > that I have written. As I see it, both of these KIPs move towards more > easily configurable replication throttling and both should be implemented. > KIP-1009 makes it easier to enable throttling and KIP-1051 makes it easier > to apply a throttle rate. I did try to look at supporting KIP-1009 in the > discussion thread, however, I only subscribed to the mailing list after it > was published and I couldn't figure out how to respond to it in Pony mail. > I would be definitely be interested in partnering up to get both changes > across the line, whether that be by combining them or supporting both > individually (I'm not sure which is best, this is my first contribution!). > > > > I also see that KAFKA-10190 is mentioned in KIP-1009 as a related > ticket. Coincidentally, I raised a PR to address this bug a couple of days > ago (https://github.com/apache/kafka/pull/16280). I think this is also a > change that will move towards more easily configurable replication > throttling as it allows configuring the throttle rate across the whole > cluster via a default value. As far as I understand, this change does not > need a KIP though because it is a bugfix (the current behaviour of ignoring > the default is unintentional). > > > > Let me know what you think. > > > > Kind regards, > > Harry > > > > > > -------- Original Message -------- > > On 6/13/24 19:08, Hector Geraldino (BLOOMBERG/ 919 3RD A) > hgerald...@bloomberg.net wrote: > > > > > Hi Harry, > > > > > > A colleague of mine opened KIP-1009: Add Broker-level Throttle > Configurations, which aims to achieve the same goal (although from a > different angle). > > > > > > Can you please take a look and see if this would work for the things > you have in mind? Maybe we can partner and coalesce around either KIP and > try to push it to the end line. > > > > > > KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1009%3A+Add+Broker-level+Throttle+Configurations > > > > > > From: dev@kafka.apache.org At: 06/13/24 09:22:40 UTC-4:00To: > dev@kafka.apache.org > > > Subject: Re: [DISCUSS] KIP-1051 Statically configured log replication > throttling > > > > > > Hi everyone, > > > > > > Bumping this thread, as I haven't yet had any replies. > > > > > > Kind regards, > > > Harry > > > > > > On Thursday, 6 June 2024 at 17:59, Harry Fallows > > > harryfall...@protonmail.com.INVALID wrote: > > > > > > > Hi everyone, > > > > > > > > I would like to propose a change to allow the static configuration > of leader > > > > and follower replication throttling rates. > > > > > > > > These configurations are very useful for preventing client traffic > from > > > > getting throttled by replication traffic during events that cause a > spike in > > > > replication. Currently they are only configurable dynamically, which > means they > > > > are only really useful for throttling replication traffic during > planned > > > > events. By allowing these configurations to be set statically, they > can be used > > > > to prevent client traffic throttling during unplanned events. > > > > > > > > KIP: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1051%3A+Statically+configu > > > > red+log+replication+throttling > > > > > > > > Best regards, > > > > Harry Fallows >
