I think we are talking about 2 cases.
1. Protect the broker
2. Topic publish throughput/rate limitation
For the first case, It was originally to avoid brokers OOM,
we don't have the publish buffer limitation at that time,
frankly speaking, It's not a good solution for avoiding broker OOM.
So, aft
Hi Xiaolong,
> Similar to triggering the backlog threshold full processing method:
AFAIK, this is handled in creating producers. But with produce throttling,
it's more like a transient state. One second is throttling, the next is not.
IMO, throttling on connection brings more benefit for protect
Similar to triggering the backlog threshold full processing method:
When the backlog hits the threshold, it is assumed that the user has set
the producer_exception policy, and the Error of
ProducerBlockedQuotaExceededError is returned to the client-side, so that
the user can perceive this behavior
We also encountered the same problem. When the current limit is triggered,
fast failure is an ideal way to deal with it. In this way, the behavior of
the current limit itself can be notified to the client-side in time, and
the user can feel the behavior of the current limit, instead of letting the
> IMO, the more serious problem about `disableAutoread` is
> Producer/Consumer isolation issue,
> since they shares the connection. For example, if topic-A is
rate-limited,
> topic-B in the same client is also affected.
Exactly. The rate-limiting won't even work properly since other rate
limiters
Yes, I agree with `it’s always possible that client got timeout`. But mostly,
it’s an UNKNOWN state, producer didn’t know whether the message was persisted
or not.
Fail-fast strategy could let producers know this message was rejected by broker
clearly.
Thanks,
Tao Jiuming
> 2022年4月7日 下午3:53，H
> send receipt to producer, the producer maybe timeout already.
It's always possible that client got timeout, but the message is actually
successfully written,
causing by network issue.
IMO, the more serious problem about `disableAutoread` is Producer/Consumer
isolation issue,
since they shar
> So, in the case, do we need to provider a new flow-control
> strategy(fail-fast)? It can be configurable, when broker in rate-limit,
> broker not `disableAutoread` and reply a `FlowControlException` instead.
Broker not `disableAutoread` but reply a `FlowControlException` instead.
>
Hi all:
According to the issue (https://github.com/apache/pulsar/issues/15038),
when producer send a message to broker and broker rate-limited at that time,
the request will blocked in buffer, after broker read the message and send
receipt to producer, the producer maybe timeout already
