Hi everyone, Bumping this one last time before I call a vote. Please take a look if you're interested in replication throttling and/or static/dynamic config.
Kind regards, Harry On Thursday, 13 June 2024 at 19:39, Harry Fallows <harryfall...@protonmail.com.INVALID> wrote: > Hi Hector, > > I did see your colleague's KIP, and I actually mentioned it in the KIP that I > have written. As I see it, both of these KIPs move towards more easily > configurable replication throttling and both should be implemented. KIP-1009 > makes it easier to enable throttling and KIP-1051 makes it easier to apply a > throttle rate. I did try to look at supporting KIP-1009 in the discussion > thread, however, I only subscribed to the mailing list after it was published > and I couldn't figure out how to respond to it in Pony mail. I would be > definitely be interested in partnering up to get both changes across the > line, whether that be by combining them or supporting both individually (I'm > not sure which is best, this is my first contribution!). > > I also see that KAFKA-10190 is mentioned in KIP-1009 as a related ticket. > Coincidentally, I raised a PR to address this bug a couple of days ago > (https://github.com/apache/kafka/pull/16280). I think this is also a change > that will move towards more easily configurable replication throttling as it > allows configuring the throttle rate across the whole cluster via a default > value. As far as I understand, this change does not need a KIP though because > it is a bugfix (the current behaviour of ignoring the default is > unintentional). > > Let me know what you think. > > Kind regards, > Harry > > > -------- Original Message -------- > On 6/13/24 19:08, Hector Geraldino (BLOOMBERG/ 919 3RD A) > hgerald...@bloomberg.net wrote: > > > Hi Harry, > > > > A colleague of mine opened KIP-1009: Add Broker-level Throttle > > Configurations, which aims to achieve the same goal (although from a > > different angle). > > > > Can you please take a look and see if this would work for the things you > > have in mind? Maybe we can partner and coalesce around either KIP and try > > to push it to the end line. > > > > KIP: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1009%3A+Add+Broker-level+Throttle+Configurations > > > > From: dev@kafka.apache.org At: 06/13/24 09:22:40 UTC-4:00To: > > dev@kafka.apache.org > > Subject: Re: [DISCUSS] KIP-1051 Statically configured log replication > > throttling > > > > Hi everyone, > > > > Bumping this thread, as I haven't yet had any replies. > > > > Kind regards, > > Harry > > > > On Thursday, 6 June 2024 at 17:59, Harry Fallows > > harryfall...@protonmail.com.INVALID wrote: > > > > > Hi everyone, > > > > > > I would like to propose a change to allow the static configuration of > > > leader > > > and follower replication throttling rates. > > > > > > These configurations are very useful for preventing client traffic from > > > getting throttled by replication traffic during events that cause a spike > > > in > > > replication. Currently they are only configurable dynamically, which > > > means they > > > are only really useful for throttling replication traffic during planned > > > events. By allowing these configurations to be set statically, they can > > > be used > > > to prevent client traffic throttling during unplanned events. > > > > > > KIP: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1051%3A+Statically+configu > > > red+log+replication+throttling > > > > > > Best regards, > > > Harry Fallows
