Re: [DISCUSS] PIP-310: Support custom publish rate limiters

2023-11-04 Thread Girish Sharma
On Sat, Nov 4, 2023 at 9:21 PM Lari Hotari wrote: > One additional question: > > In your use case, do you have multiple producers concurrently producing to > the same topic from different clients? > > The use case is challenging in the current implementation when using topic > producing rate limi

Re: [DISCUSS] PIP-310: Support custom publish rate limiters

2023-11-04 Thread Girish Sharma
Replies inline. On Sat, Nov 4, 2023 at 8:55 PM Lari Hotari wrote: > > One possibility would be to improve the existing rate limiter to allow > bursting. > I think that Pulsar's out-of-the-box rate limiter should cover 99% of the > use cases instead of having one implementing their own rate limit

Re: [DISCUSS] PIP-310: Support custom publish rate limiters

2023-11-04 Thread Lari Hotari
One additional question: In your use case, do you have multiple producers concurrently producing to the same topic from different clients? The use case is challenging in the current implementation when using topic producing rate limiting. The problem is that the different producers will be able t

Re: [DISCUSS] PIP-310: Support custom publish rate limiters

2023-11-04 Thread Lari Hotari
Replies inline On Fri, 3 Nov 2023 at 20:48, Girish Sharma wrote: > Could you please elaborate more on these details? Here are some questions: > > 1. What do you mean that it is too strict? > > - Should the rate limiting allow bursting over the limit for some > time? > > > > That's one of the