How about edge cases? In Andra's PIP he took into account cases where updates were lost, so he created a secondary poll. Not saying it's the best situation for your case of course. I'm saying that when a broker sends an update CommandPartitionUpdate, how do you know it arrived successfully? From my memory, there is no ACK in the protocol, saying "I'm the client, I got the update successfully" and only then it removed the "dirty" flag for that topic, for this watcher ID.
Are there any other edge cases we can have? Let's be exhaustive. On Wed, Feb 22, 2023 at 1:14 PM houxiaoyu <anonhx...@gmail.com> wrote: > Thanks for your great suggestion Enrico. > > I agreed with you. It's more reasonable to add a > `supports_partition_update_watchers` in `FeatureFlags` to detect that the > connected broker supporting this feature , and add a new broker > configuration property `enableNotificationForPartitionUpdate` with default > value true, which is much like PIP-145. > > I have updated the descriptions. > > Enrico Olivelli <eolive...@gmail.com> 于2023年2月22日周三 17:26写道： > > > I support this proposal. > > Coping here my comments from GH: > > > > can't we enable this by default in case we detect that the connected > > Broker supports it ? > > I can't find any reason for not using this mechanism if it is available. > > > > Maybe we can set the default to "true" and allow users to disable it > > in case it impacts their systems in an unwanted way. > > > > Maybe It would be useful to have a way to disable the mechanism on the > > broker side as well > > > > Enrico > > > > Il giorno mer 22 feb 2023 alle ore 10:22 houxiaoyu > > <anonhx...@gmail.com> ha scritto: > > > > > > Hi Pulsar community: > > > > > > I opened a PIP to discuss "Notifications for partitions update" > > > > > > ### Motivation > > > > > > Pulsar client will poll brokers at fix time for checking the partitions > > > update if we publish/subscribe the partitioned topics with > > > `autoUpdatePartitions` as true. This causes unnecessary load for both > > > clients and brokers since most of the time the number of partitions > will > > > not change. In addition polling introduces latency in partitions update > > > which is specified by `autoUpdatePartitionsInterval`. > > > This PIP would like to introduce a notification mechanism for partition > > > update, which is much like PIP-145 for regex subscriptions > > > https://github.com/apache/pulsar/issues/14505. > > > > > > For more details, please read the PIP at: > > > https://github.com/apache/pulsar/issues/19596 > > > Looking forward to hearing your thoughts. > > > > > > Thanks, > > > Xiaoyu Hou > > > ---- > > >
