Hi Minjian, Thanks for sharing the proposal. Sharing my thoughts here:
The maximum unacknowledged message limitation is crucial for consumer load balancing and managing broker resources in Pulsar's Shared and Key_Shared subscriptions: Consumer Load Balancing: The limit prevents any single consumer from being overwhelmed with more messages than it can process. When a consumer hits this cap, Pulsar stops sending messages and, in a shared subscription, will dispatch new messages to other available consumers. This effectively balances the workload across the consumer group, preventing bottlenecks and ensuring smooth processing. Manages Broker Resources: Every message sent but not yet acknowledged consumes broker memory and CPU to track its state. By capping the number of unacknowledged messages per consumer and subscription (ack state), the broker avoids being swamped by slow or unresponsive consumers. This conserves vital resources, preventing performance degradation and ensuring the stability of the entire Pulsar cluster. Failover and Exclusive subscriptions don't require a max unacknowledged message limit because they operate more simply: One Active Consumer: Only a single consumer receives messages at any time. This eliminates the need to load balance or protect individual consumers from being overwhelmed relative to others. Cumulative Acknowledgement: The consumer can acknowledge the last message it processed, and Pulsar automatically marks all previous messages as acknowledged. This is highly efficient and drastically reduces the number of unacknowledged messages the broker needs to track for that single consumer. I agree that there could be use cases for limiting the max unacked messages for Failover or Exclusive subscriptions. If you have any real-world scenarios where this limitation is required, it would be great to include them in the proposal. This would help users understand when and why they should use this feature. Since this proposal will change the behavior of the existing Failover or Exclusive subscriptions, please consider making it disabled by default to avoid breaking existing users and having a flag for users to enable it. Regards, Penghui On Tue, Jun 10, 2025 at 4:21 AM 蔡敏健 <xiaocair...@gmail.com> wrote: > Hi, Pulsar Community, > > I opened a new pip to enable consumer throttling and accurate > unacknowledged message tracking for exclusive and failover subscriptions. > I'm looking > forward to your suggestions! > > link: https://github.com/apache/pulsar/pull/24400 > > Thanks, > Minjian cai >
