Jia thank you for raising this up. A couple of comments here
1. technically you don't need a consistent hashing mechanism for dividing hash ranges for consumers. Any hash mechanism that can map a key to a consumer should work here. The only advantage of consistent hashing mechanism is to reduce the disruptions when a consumer joins and leaves. So I would prefer making the hashing mechanism pluggable. 2. regarding using key or introducing a new ordering key. I would prefer using `key` here. having two `keys` actually makes people confused. The `key` was used for compaction but the compaction doesn't mutate the original data. it just creates a new compacted ledger. in most of the time, when people use `key_failover`, they are probably accessing the original data. Event they are trying to subscribe to compacted topic using `key_failover`, the ordering of messages by key will still be true. so I wouldn't worry about this part that much. 3. in `failover` there is a consumer listener that listens on the partition assignment changes. Is there any requirements or plans on adding similar listeners for `key_failover`? Thanks, Sijie On Wed, Apr 3, 2019 at 2:09 AM Jia Zhai <zhai...@apache.org> wrote: > Hi all, > > I would like to init a discuss of "PIP 34: Add new subscribe type -- > Key_Failover" here. > > At present, there are 3 kinds of subscription modes: exclusive, shared, and > failover. Shared mode subscription is used a lot, because consumers could > effectively parallel consume messages of same partition. But using shared > mode will not keep the message order of same key, It would be good to > leverage share mode, while also keeping the ordering of messages. > > This Proposal trying to introduce a new subscribe type Key_Failover to > extend shared type. By Key_Failover, one partition could have several > consumers to parallel consume messages, while all messages with the same > key will be dispatched to only one consumer. > > Here is the link contains more information of this PIP: > > https://github.com/apache/pulsar/wiki/PIP-34%3A-Add-new-subscribe-type-Key_Failover > > Thanks. >
