Re: [DISCUSS] Enhance dynamic topic subscription management in PatternMultiTopicsConsumer

2025-01-05 Thread SiNan Liu
Adding a MultiTopicConsumerHandle is indeed a brilliant way. Not only can it implement the addition and removal of certain consumers within it, but perhaps it could also allow for other operations on individual consumers within the MultiTopicConsumer? Such as pause(), resume(), because I see there

Re: [DISCUSS] Enhance dynamic topic subscription management in PatternMultiTopicsConsumer

2025-01-04 Thread Lari Hotari
Thanks for sharing details about the broader context of the use case. I think we could start drafting ideas of what the API solution could be. One of the first things to address in designing APIs is to focus on the naming of concepts. Concepts should be clear and not conflicting with other names

Re: [DISCUSS] Enhance dynamic topic subscription management in PatternMultiTopicsConsumer

2025-01-04 Thread SiNan Liu
I would like to momentarily set aside the issue of topic processing speed disparities causing bottlenecks and instead focus on enhancing our dynamic subscription management capabilities. Currently, when a consumer subscribes to topics using a pattern, any modification to this pattern necessitates

Re: [DISCUSS] Enhance dynamic topic subscription management in PatternMultiTopicsConsumer

2025-01-03 Thread Lari Hotari
Before proceeding with implementation details, it would be helpful to better understand the underlying use cases and requirements. Could you provide more details in this Pulsar dev mailing list thread about: 1. The specific scenarios where topic processing speed differences create bottlenecks -

[DISCUSS] Enhance dynamic topic subscription management in PatternMultiTopicsConsumer

2025-01-03 Thread SiNan Liu
Hi, Pulsar Community. I added a new PR that aims to enhance our management of topic subscriptions within PatternMultiTopicsConsumer in Pulsar client. This PR introduces the ability to dynamically add or remove topics from a regex pattern subscription, which can significantly improve our system’s r