Hi all, I updated the KIP with more details on per-IP connection rate limiting. Notable changes are the addition of metrics tracking per-IP connection rate acceptance and IP connection rate limiting throttling. In addition, I fleshed out details around the API we will use to describe and alter per-IP quotas. Let me know if there are any questions/concerns.
Thanks David (Mao) On 2020/11/03 21:04:37, Anna Povzner <a...@confluent.io> wrote: > Hi David,> > > You are right, using a token bucket exposes the metric for the number of> > remaining tokens in the bucket. Since we also want to evaluate using a> > token bucket for bandwidth & request throttling, it would be better to have> > this discussion separately in a separate KIP. This KIP does not implement> > token bucket throttling, and I made sure that KIP wiki does not mention it> > as well. Hopefully, I haven't added any confusion.> > > As David Mao is implementing per-IP part of this KIP, he will update the> > KIP wiki and notify this thread with a couple of more details that were not> > mentioned in the KIP wiki (e.g., per-IP rate metrics exposed when> > throttling).> > > Thanks,> > Anna> > > On Wed, Sep 2, 2020 at 1:06 PM David Jacot <dj...@confluent.io> wrote:> > > > Hi Anna,> > >> > > Thanks for the update.> > >> > > If you use Token Bucket, it will expose another metric which reports the> > > number of remaining tokens in the bucket, in addition to the current rate> > > metric. It would be great to add it in the metrics section of the KIP as> > > well> > > for completeness.> > >> > > Best,> > > David> > >> > > On Tue, Aug 11, 2020 at 4:28 AM Anna Povzner <an...@confluent.io> wrote:> > >> > > > Hi All,> > > >> > > > I wanted to let everyone know that we would like to make the following> > > > changes to the KIP:> > > >> > > > 1.> > > >> > > > Expose connection acceptance rate metrics (broker-wide and> > > per-listener)> > > > and per-listener average throttle time metrics for better> > > observability> > > > and> > > > debugging.> > > > 2.> > > >> > > > KIP-599 introduced a new implementation of MeasurableStat that> > > > implements a token bucket, which improves rate throttling for bursty> > > > workloads (KAFKA-10162). We would like to use this same mechanism for> > > > connection accept rate throttling.> > > >> > > >> > > > I updated the KIP to reflect these changes.> > > >> > > > Let me know if you have any concerns.> > > >> > > > Thanks,> > > >> > > > Anna> > > >> > > >> > > > On Thu, May 21, 2020 at 5:42 PM Anna Povzner <an...@confluent.io> wrote:> > > >> > > > > The vote for KIP-612 has passed with 3 binding and 3 non-binding +1s,> > > and> > > > > no objections.> > > > >> > > > >> > > > > Thanks everyone for reviews and feedback,> > > > >> > > > > Anna> > > > >> > > > > On Tue, May 19, 2020 at 2:41 AM Rajini Sivaram <> > > rajinisiva...@gmail.com>> > > > > wrote:> > > > >> > > > >> +1 (binding)> > > > >>> > > > >> Thanks for the KIP, Anna!> > > > >>> > > > >> Regards,> > > > >>> > > > >> Rajini> > > > >>> > > > >>> > > > >> On Tue, May 19, 2020 at 9:32 AM Alexandre Dupriez <> > > > >> alexandre.dupr...@gmail.com> wrote:> > > > >>> > > > >> > +1 (non-binding)> > > > >> >> > > > >> > Thank you for the KIP!> > > > >> >> > > > >> >> > > > >> > Le mar. 19 mai 2020 à 07:57, David Jacot <dj...@confluent.io> a> > > > écrit> > > > >> :> > > > >> > >> > > > >> > > +1 (non-binding)> > > > >> > >> > > > >> > > Thanks for the KIP, Anna!> > > > >> > >> > > > >> > > On Tue, May 19, 2020 at 7:12 AM Satish Duggana <> > > > >> satish.dugg...@gmail.com> > > > >> > >> > > > >> > > wrote:> > > > >> > >> > > > >> > > > +1 (non-binding)> > > > >> > > > Thanks Anna for the nice feature to control the connection> > > > creation> > > > >> > rate> > > > >> > > > from the clients.> > > > >> > > >> > > > >> > > > On Tue, May 19, 2020 at 8:16 AM Gwen Shapira <g...@confluent.io> > > >> > > > >> > wrote:> > > > >> > > >> > > > >> > > > > +1 (binding)> > > > >> > > > >> > > > >> > > > > Thank you for driving this, Anna> > > > >> > > > >> > > > >> > > > > On Mon, May 18, 2020 at 4:55 PM Anna Povzner <> > > a...@confluent.io> > > > >> > > > >> > wrote:> > > > >> > > > >> > > > >> > > > > > Hi All,> > > > >> > > > > >> > > > >> > > > > > I would like to start the vote on KIP-612: Ability to limit> > > > >> > connection> > > > >> > > > > > creation rate on brokers.> > > > >> > > > > >> > > > >> > > > > > For reference, here is the KIP wiki:> > > > >> > > > > >> > > > >> > > > > >> > > > >> > > > >> > > > >> > > >> > > > >> >> > > > >>> > > >> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-612%3A+Ability+to+Limit+Connection+Creation+Rate+on+Brokers> > > > > > >> > > > > >> > > > >> > > > > > And discussion thread:> > > > >> > > > > >> > > > >> > > > > >> > > > >> > > > >> > > > >> > > >> > > > >> >> > > > >>> > > >> > > https://lists.apache.org/thread.html/r61162661fa307d0bc5c8326818bf223a689c49e1c828c9928ee26969%40%3Cdev.kafka.apache.org%3E> > > > > > >> > > > > >> > > > >> > > > > > Thanks,> > > > >> > > > > >> > > > >> > > > > > Anna> > > > >> > > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > > --> > > > >> > > > > Gwen Shapira> > > > >> > > > > Engineering Manager | Confluent> > > > >> > > > > 650.450.2760 | @gwenshap> > > > >> > > > > Follow us: Twitter | blog> > > > >> > > > >> > > > >> > > >> > > > >> >> > > > >>> > > > >> > > >> > >> >
