Hi Nikolai, Thanks for bringing this to the attention. I have seen multiple PIP and bug fixes for the Key-Shared subscription and I feel Key_Shared subscription dispatcher is developed with multiple fundamental issues which required to introduce scheduled-task to unblock stuck reads, introducing stuck delivery flag, and now we are seeing at least 2-3 PIP to add few more hacks to control such stuck dispatch. After reviewing code, it seems PersistentStickyKeyDispatcherMultipleConsumers::getRestrictedMaxEntriesForConsumer(..) function can still return 0 dispatch messages with multiple different edge cases and fundamental dispatch is not clean at all. This is the only dispatcher introduced after open sourcing Pulsar and unfortunately it was not developed with a cleaner approach. I understand the pain of such stuck dispatch and even the current release is having a bugs where dispatch is getting stuck and many production systems are struggling with it and you must be suggesting this PIP to get work around for this stuck issue. But I don't think this is the right approach to follow. I think the correct approach is to get rid of this "stuck" keyword from the dispatcher because having "stuck" concept in dispatcher means it has a known flow and bug which we are not solving but we are adding more debt to perform the work around. We never had any such concept in other dispatchers which was introduced when Pulsar was introduced originally and it just works with clear semantics and permit flow mechanism , and we should follow the same practice for this subscription.
Thanks, Rajan On Thu, Nov 30, 2023 at 9:18 AM Nikolai <tadeus...@gmail.com> wrote: > Hello! > > I submitted a new PIP to add a configuration option which allows to skip > blocking recently joined consumers for Key_Shared subscriptions > > It introduces additional memory consumption, so it will be disabled by > default. It would fix issues like > https://github.com/apache/pulsar/issues/21199 > > Link to the PIP: https://github.com/apache/pulsar/pull/21615 > > PR with the PIP implementation: > https://github.com/apache/pulsar/pull/21579 > > > Best Regards, > > Nikolai > >
