> I would invest more time in: > - monitoring tools (tools to detect quickly stuck consumers) > - circuit breakers (fast fail/shut the door to consumers/producers > that don't behave correctly) > - guard rails (limits to clients to prevent them to exhaust the resources on the brokers)
I agree with this point of view, we should not increase the complexity of the code, add non-essential APIs to extend unload, unload is not essentially to solve consumer stuck or producer block problem. Thanks, Bo Enrico Olivelli <eolive...@gmail.com> 于2023年1月12日周四 16:12写道： > > Yubiao, > thanks for sharing your problem and a proposal, this is very helpful > for the community to get in touch with the pain of Pulsar > users/administrators. > > In my experience if a "subscription is stuck", the problems are: > * the client has some problems (bug in the client/misconfiguration > somewhere) - 99.9% > * there is a bug in Pulsar - 0.1% > > Unloading a topic is an operation that triggers some reset of the > state on both the broker and the clients and this usually TEMPORARY > unblocks the subscription. > > I have never seen a problem that is temporarily solved by topic > unload/broker restart to be permanently solved with that operation. > If there is a problem we should spend time on investigating the > problem and not in adding this kind of tool. > > I believe that we should not continue to add these kinds of hacks into Pulsar: > - easy reset... > - ignore errors... (catch Throwable...) > > The overall result is a system that "seems to work" but it actually > doesn't work properly > > I would invest more time in: > - monitoring tools (tools to detect quickly stuck consumers) > - circuit breakers (fast fail/shut the door to consumers/producers > that don't behave correctly) > - guard rails (limits to clients to prevent them to exhaust the > resources on the brokers) > > > > Enrico > > Il giorno gio 12 gen 2023 alle ore 08:22 <mattisonc...@gmail.com> ha scritto: > > > > Hi, Yubiao > > > > I agree with this idea because some users care about the production rate. > > They don't want to unload the whole topic to fix the subscription problem. > > > > I've got some questions: > > > > 1. How do you handle the race condition when you are trying to unload the > > subscription, and the new consumer wants to subscribe to this subscription > > at the same time? I'm unsure if it has the race condition. I just want to > > remind you about that. :) > > 2. Would you like to add some restful API design to clarify the > > implementation? > > a. Request method > > b. Request path > > c. Response code > > d. etc. > > > > > > Thanks for your work. > > Mattison > > On Jan 11, 2023, 17:01 +0800, Yubiao Feng > > <yubiao.f...@streamnative.io.invalid>, wrote: > > > Hi community > > > > > > I am starting a DISCUSS for PIP-240: A new API to unload subscriptions. > > > > > > PIP issue: https://github.com/apache/pulsar/issues/19187 > > > > > > ### Motivation > > > > > > We sometimes try to unload the topic to resolve some consumption-stop > > > issues. But the unloading topic will also impact the producer side. > > > > > > ### Goal > > > > > > Providing a new API to unload the subscription dimension triggers > > > reconnection of all consumers on that subscription and reconnection is > > > guaranteed by the client. The API will be used in these ways: > > > - unload special subscription of one topic(or partitioned topic) > > > - unload all subscriptions of one topic(or partitioned topic) > > > - unload subscriptions of one topic(or partitioned topic) by regular > > > expression > > > - If a reader's subscription name is not set, a random subscription name > > > prefixed with 'multiTopicsReader-' or 'reader-' will be used, and users > > > can > > > uninstall these subscriptions using regular expressions. > > > > > > In addition to triggering consumer disconnection, Unloading Subscribers > > > will restart the Dispatcher, which resets the redeliver message queue and > > > delayed message queue in the Broker's memory, which can help resolve > > > issues > > > caused by an abnormal dispatcher state. However, the execution flow of > > > Unloading Subscribers does not include a restart of the Managed Cursor > > > related to this dispatcher; if there is a problem with the cursor, we can > > > only rely on the unload topic to solve it. > > > > > > Note: From the client's perspective, this connection may be shared by > > > consumers, producers, and transactions, so Unloading Subscribers maybe > > > impact the producer and transaction. > > > > > > #### These scenarios are not supported > > > - Functions `message-dedup`, `geo-replication,` and `shadow-topic` also > > > read messages from the topic, but Unloading subscribers will not support > > > triggering restarts of these three functions( because the cursor is used > > > directly to read the data in these scenarios, not the consumer or reader > > > ). > > > - The Compression task(subscription name is `__compaction`) also use a > > > reader to read data, but Unloading Subscribers does not support it because > > > this task creates a new reader each time it starts. > > > - Do not support all topics related to Transaction features. > > > - `__transaction_buffer_snapshot` works with the task TB recover, and > > > this task will create a new reader each time they start. > > > - `__transaction_pending_ack` works with the task Transaction Pending Ack > > > Store replay, and this task will use managed cursor directly to read data. > > > - `__transaction_log_xxx` works with the task Transaction Log, which will > > > use managed cursor directly to read data. > > > - `transaction_coordinator_assign` No data will be written on this topic. > > > > > > #### Special system topic supports > > > The system topic `__change_events` is used to support topic-level > > > policies, > > > there may also be some message delivery issues in this scenario, so > > > Unloading Subscribers will support this topic. > > > > > > ### API Changes > > > > > > #### For persistent topic > > > ``` > > > pulsar-admin persistent unload {topic_name} -s {sub_name} > > > ``` > > > > > > #### For non-persistent topic > > > ``` > > > pulsar-admin non-persistent unload {topic_name} -s {sub_name} > > > ``` > > > > > > #### Explain the param `-s` > > > - set param `-s` to special sub name to unload special subscription > > > - set param `-s` to `**` to unload all subscriptions under this topic > > > - set param `-s` to `regexp` to unload a batch subscriptions under this > > > topic > > > > > > > > > Thanks > > > Yubiao Feng
