Hi Xiaolong, > Similar to triggering the backlog threshold full processing method: AFAIK, this is handled in creating producers. But with produce throttling, it's more like a transient state. One second is throttling, the next is not.
IMO, throttling on connection brings more benefit for protecting broker. As for most clients, they will do a retry when it's rate-limited. This amplifies the risk of crushing the broker or make it morse for heavy loaded brokers. So I suggest that if we want to introduce this feature. It have to come with some mandatory backoff policy to avoid the case mentioned above. Thanks, Haiting On 2022/04/13 03:01:09 "r...@apache.org" wrote: > Similar to triggering the backlog threshold full processing method: > > When the backlog hits the threshold, it is assumed that the user has set > the producer_exception policy, and the Error of > ProducerBlockedQuotaExceededError is returned to the client-side, so that > the user can perceive this behavior. > > r...@apache.org <ranxiaolong...@gmail.com> 于2022年4月13日周三 10:59写道： > > > We also encountered the same problem. When the current limit is triggered, > > fast failure is an ideal way to deal with it. In this way, the behavior of > > the current limit itself can be notified to the client-side in time, and > > the user can feel the behavior of the current limit, instead of letting the > > user Send or receive a timeout. > > > > But for pulsar's own model, this thing doesn't seem to be handled very > > well, because the current limit is based on the connection itself, as > > Haiting and Lari said, if multiple topics reuse the same connection, then > > this situation will get tricky. > > > > So can we introduce other commands in Proto here, because the broker knows > > that the client has triggered various current-limiting thresholds? If the > > current-limiting threshold is triggered, the broker will notify the > > client-side of the information. > > > > -- > > Thanks > > Xiaolong Ran > > > > Lari Hotari <l...@hotari.net> 于2022年4月8日周五 00:21写道： > > > >> > IMO, the more serious problem about `disableAutoread` is > >> > Producer/Consumer isolation issue, > >> > since they shares the connection. For example, if topic-A is > >> rate-limited, > >> > topic-B in the same client is also affected. > >> > >> Exactly. The rate-limiting won't even work properly since other rate > >> limiters operating on the same connection will enable autoread and this > >> results in inconsistent behavior. > >> > >> -Lari > >> > >> On Thu, Apr 7, 2022 at 10:53 AM Haiting Jiang <jianghait...@apache.org> > >> wrote: > >> > >> > > send receipt to producer, the producer maybe timeout already. > >> > > >> > It's always possible that client got timeout, but the message is > >> actually > >> > successfully written, > >> > causing by network issue. > >> > > >> > IMO, the more serious problem about `disableAutoread` is > >> Producer/Consumer > >> > isolation issue, > >> > since they shares the connection. For example, if topic-A is > >> rate-limited, > >> > topic-B in the same > >> > client is also affected. > >> > > >> > Thanks, > >> > Haiting > >> > > >> > On 2022/04/07 07:13:50 Jiuming Tao wrote: > >> > > Hi all: > >> > > According to the issue ( > >> > https://github.com/apache/pulsar/issues/15038), when producer send a > >> > message to broker and broker rate-limited at that time, the request will > >> > blocked in buffer, after broker read the message and send receipt to > >> > producer, the producer maybe timeout already. > >> > > So, in the case, do we need to provider a new flow-control > >> > strategy(fail-fast)? It can be configurable, when broker in rate-limit, > >> > broker not `disableAutoread` and reply a `FlowControlException` instead. > >> > > > >> > > Thanks, > >> > > Tao Jiuming > >> > > >> > > >
