Re: [DISCUSS] PIP-385 Add rate limit semantics to pulsar protocol and Java client

2024-11-07 Thread Girish Sharma
Hello everything! Thank you for participating in the discussion. It's been over a month now. I will be starting a VOTE thread tomorrow. Regards On Thu, Oct 31, 2024 at 2:28 AM Rajan Dhabalia wrote: > Hi Girish, > > I apologize for the delayed response. I have added my comments on the > proposa

Re: [DISCUSS] PIP-385 Add rate limit semantics to pulsar protocol and Java client

2024-10-30 Thread Rajan Dhabalia
Hi Girish, I apologize for the delayed response. I have added my comments on the proposal with few questions and suggestions. Thank, Rajan On Tue, Oct 29, 2024 at 10:48 PM Girish Sharma wrote: > Hello everyone, gentle reminder. > > If there are no further comments, then please close the github

Re: [DISCUSS] PIP-385 Add rate limit semantics to pulsar protocol and Java client

2024-10-29 Thread Girish Sharma
Hello everyone, gentle reminder. If there are no further comments, then please close the github comments as resolved so that the PR can be merged after voting. Regards On Sat, Oct 26, 2024 at 1:52 AM Lari Hotari wrote: > Thanks for the great progress, Girish. > I apologize for the delayed feed

Re: [DISCUSS] PIP-385 Add rate limit semantics to pulsar protocol and Java client

2024-10-25 Thread Lari Hotari
Thanks for the great progress, Girish. I apologize for the delayed feedback due to Pulsar 4.0 release activities. I'll follow up in more detail next week. The Pulsar 4.0 blog post mentions: "While Pulsar already supports producer rate limiting, the community is building on this foundation with P

Re: [DISCUSS] PIP-385 Add rate limit semantics to pulsar protocol and Java client

2024-10-10 Thread Girish Sharma
I've updated the proposal with suggestions from Lari about utilization based rate limit exceptions on clients, along with a minor change in the blocking section to ensure ordering is maintained. Please have a look again. Regarding this comment: > Well, even if we have throttle producer protocol,

Re: [DISCUSS] PIP-385 Add rate limit semantics to pulsar protocol and Java client

2024-10-05 Thread Girish Sharma
Hi Rajan, Thanks for taking the time and going through the PIP. >>> Well, even if we have throttle producer protocol, if client app is keep >>> producing messages then client app will see high timeout and to fast fail >>> this issue, Pulsar Client has internal producer queue and client can always

Re: [DISCUSS] PIP-385 Add rate limit semantics to pulsar protocol and Java client

2024-10-04 Thread Rajan Dhabalia
Hi Girish, I have gone through the proposal and you mentioned few problems as a motivation of this improvements >> Noisy neighbors - Even if one topic is exceeding the quota, since the entire channel read is paused, all topics sharing the same connect (for example - using the same java client obj

Re: [DISCUSS] PIP-385 Add rate limit semantics to pulsar protocol and Java client

2024-10-04 Thread Lari Hotari
Great work on this proposal, Girish! This improvement addresses a crucial aspect of Pulsar's functionality. You're effectively bridging an important gap in Pulsar's producer flow control. This addition will improve the ability to set and meet SLAs across various Pulsar use cases, which is inval

[DISCUSS] PIP-385 Add rate limit semantics to pulsar protocol and Java client

2024-10-04 Thread Girish Sharma
Hello Pulsar Community, I would like to propose a new improvement for Pulsar protocol related to rate limiting that the broker imposes to maintain quality of service. This proposal adds a new binary protocol command pair and corresponding server and java client changes. With the new protocol comma