Hi all, I would like propose change the subscription name "Key_Shared" to "Key_Based".
This subscription type is based on the key and key based dispatching strategies for message dispatching. I would like to use "Key_Based" as the subscription name, may express the characteristics of this subscription type more accurately. "Key_Based" also used by flink data exchanges strategies, this is a better understanding for the users. Sijie Guo <guosi...@gmail.com> 于2019年4月11日周四 下午2:22写道： > Acked. `ordering_key` looks good to me. > > - Sijie > > On Mon, Apr 8, 2019 at 11:40 AM Jia Zhai <zhaiji...@gmail.com> wrote: > >> Thanks Penghui & Sijie for the comments, updated the wiki link. >> >> 1. we should add the new ordering_key, it is as Penghui mentioned, There >> may features that all enabled, but want to use key for different purposes. >> 2. regarding consumer listener in "failover", it is originally for >> failover/exclusive mode, which only let one consumer alive. In >> Key_Failover, it is more like shared mode, all alive consumer should be >> able to consume messages. >> >> Best Regards. >> >> >> Jia Zhai >> >> Beijing, China >> >> Mobile: +86 15810491983 >> >> >> >> >> On Fri, Apr 5, 2019 at 1:04 AM Sijie Guo <guosi...@gmail.com> wrote: >> >> > Penguin, >> > >> > Thank you for your clarification! That makes sense to me then. >> > >> > Sijie >> > >> > On Thu, Apr 4, 2019 at 2:21 AM PengHui Li <codelipeng...@gmail.com> >> wrote: >> > >> > > @Sijie Guo <si...@apache.org> >> > > >> > > Sorry, I may not have described my point in the previous email. >> > > I try to describe it again. :) >> > > >> > > User can use key to achieve business logic(e.g. use user_id as message >> > key) >> > > At the same time, want to use another identifier as the basis for the >> > > order. >> > > >> > > As the message key has many uses, avoid duplicate messages, compaction >> > > topics. >> > > I prefer provide a mechanism to overwrite key based sorting behavior. >> > > Of course this is not mandatory. >> > > >> > > If the basis for sorting is exactly the same as the real purpose of >> the >> > > message key, >> > > just use the message key. Otherwise user provide the order key to >> > > overwrite. >> > > >> > > Best Regards. >> > > Penghui >> > > >> > > Sijie Guo <guosi...@gmail.com> 于2019年4月4日周四 下午3:36写道： >> > > >> > >> Jia thank you for raising this up. >> > >> >> > >> A couple of comments here >> > >> >> > >> 1. technically you don't need a consistent hashing mechanism for >> > dividing >> > >> hash ranges for consumers. Any hash mechanism that can map a key to a >> > >> consumer should work here. The only advantage of consistent hashing >> > >> mechanism is to reduce the disruptions when a consumer joins and >> leaves. >> > >> So >> > >> I would prefer making the hashing mechanism pluggable. >> > >> >> > >> 2. regarding using key or introducing a new ordering key. I would >> prefer >> > >> using `key` here. having two `keys` actually makes people confused. >> The >> > >> `key` was used for compaction but the compaction doesn't mutate the >> > >> original data. it just creates a new compacted ledger. in most of the >> > >> time, >> > >> when people use `key_failover`, they are probably accessing the >> original >> > >> data. Event they are trying to subscribe to compacted topic using >> > >> `key_failover`, the ordering of messages by key will still be true. >> so I >> > >> wouldn't worry about this part that much. >> > >> >> > >> 3. in `failover` there is a consumer listener that listens on the >> > >> partition >> > >> assignment changes. Is there any requirements or plans on adding >> similar >> > >> listeners for `key_failover`? >> > >> >> > >> Thanks, >> > >> Sijie >> > >> >> > >> On Wed, Apr 3, 2019 at 2:09 AM Jia Zhai <zhai...@apache.org> wrote: >> > >> >> > >> > Hi all, >> > >> > >> > >> > I would like to init a discuss of "PIP 34: Add new subscribe type >> -- >> > >> > Key_Failover" here. >> > >> > >> > >> > At present, there are 3 kinds of subscription modes: exclusive, >> > shared, >> > >> and >> > >> > failover. Shared mode subscription is used a lot, because consumers >> > >> could >> > >> > effectively parallel consume messages of same partition. But using >> > >> shared >> > >> > mode will not keep the message order of same key, It would be good >> to >> > >> > leverage share mode, while also keeping the ordering of messages. >> > >> > >> > >> > This Proposal trying to introduce a new subscribe type >> Key_Failover to >> > >> > extend shared type. By Key_Failover, one partition could have >> several >> > >> > consumers to parallel consume messages, while all messages with the >> > same >> > >> > key will be dispatched to only one consumer. >> > >> > >> > >> > Here is the link contains more information of this PIP: >> > >> > >> > >> > >> > >> >> > >> https://github.com/apache/pulsar/wiki/PIP-34%3A-Add-new-subscribe-type-Key_Failover >> > >> > >> > >> > Thanks. >> > >> > >> > >> >> > > >> > >> >
