Hi Rajan, Thanks for taking the time and going through the PIP. >>> Well, even if we have throttle producer protocol, if client app is keep >>> producing messages then client app will see high timeout and to fast fail >>> this issue, Pulsar Client has internal producer queue and client can always >>> tune that queue. once that queue is fail, client can configure to fast fail >>> or wait by blocking client thread but in both ways, client application will >>> be aware that publish is taking longer time and client app can always do >>> backoff if needed, So, this is very well known issue and it's already >>> solved in Pulsar. Your github comments are missing this point about the client timeout, producer queue etc. Could you please paste it there itself so that we can keep the discussion contained at one place?
Regards On Sat, Oct 5, 2024 at 4:58 AM Rajan Dhabalia <rdhaba...@apache.org> wrote: > Hi Girish, > > I have gone through the proposal and you mentioned few problems as a > motivation of this improvements > > >> Noisy neighbors - Even if one topic is exceeding the quota, since the > entire channel read is paused, all topics sharing the same connect (for > example - using the same java client object) get rate limited. > > I don't think it's a noisy neighbor issue. There are many ways: clients can > use a separate connection for different topics by increasing the number of > connections and more specifically create Cache of PulsarClient objects to > manage topics belonging to different usecases. If you use one channel for > different tenants/usecases and if they get impacted then it's not a noisy > neighbor but the application might need design improvement. > For example: If client app use the same topic for different usecases then > all usecases can be impacted by each other, and that doesn't mean Pulsar > has a noisy neighbor issue but it needs a design change to use separate > topics for each usecase. So, this challenge is easily achievable. > > >> Unaware clients - clients are completely unaware that they are being > rate limited. This leads to all send calls taking super long time or simply > timing out... they can either fail fast or induce back-pressure to their > upstream. > > Well, even if we have throttle producer protocol, if client app is keep > producing messages then client app will see high timeout and to fast fail > this issue, Pulsar Client has internal producer queue and client can always > tune that queue. once that queue is fail, client can configure to fast fail > or wait by blocking client thread but in both ways, client application will > be aware that publish is taking longer time and client app can always do > backoff if needed, So, this is very well known issue and it's already > solved in Pulsar. > > and we should have server side metrics for topic throttling which should > give a clear picture of msgRate and throttling for any further debugging. > > So, I think every issue is already addressed and I don't see any specific > need for these issue. > > Thanks, > Rajan > > > > On Fri, Oct 4, 2024 at 3:45 PM Lari Hotari <lhot...@apache.org> wrote: > > > Great work on this proposal, Girish! > > > > This improvement addresses a crucial aspect of Pulsar's functionality. > > You're effectively bridging an important gap in Pulsar's producer flow > > control. This addition will improve the ability to set and meet SLAs > across > > various Pulsar use cases, which is invaluable for many of our users. > > > > Thank you for driving this important improvement. It's contributions like > > these that continue to enhance Pulsar's robustness and flexibility. > > > > Looking forward to seeing this develop further. > > > > -Lari > > > > On 2024/10/04 14:48:09 Girish Sharma wrote: > > > Hello Pulsar Community, > > > > > > I would like to propose a new improvement for Pulsar protocol related > to > > > rate limiting that the broker imposes to maintain quality of service. > > This > > > proposal adds a new binary protocol command pair and corresponding > server > > > and java client changes. With the new protocol command, clients would > be > > > able to understand that they are breaching the quota for a topic and > take > > > action accordingly. > > > > > > The full proposal can be found at > > > https://github.com/apache/pulsar/pull/23398 > > > Direct link to rendered markdown with mermaid flowcharts - > > > https://github.com/grssam/pulsar/blob/rl-protocol/pip/pip-385.md > > > > > > Please share your thoughts on this proposal along with any concerns or > > > suggestions. > > > > > > Regards > > > -- > > > Girish Sharma > > > > > > -- Girish Sharma
