Hi, Aloys: Yes, it will work with `consumer.seek()`. Sorry, I missed this, I will add this description to the PIP.
But the current seek method has some problems, the detail in https://lists.apache.org/thread/97o9t4ltkds5pfq41l9xbbd31t41qm8w, I am not sure, does it make sense to support seek method in this PIP. Thanks, Bo Aloys Zhang <aloyszh...@apache.org> 于2023年3月21日周二 19:08写道： > > Nice proposal. > > I'm interested in a point > > So when we need to reset the cursor, the client consumer should all be > closed, and then reset the cursor then restart the consumer. > > Does this requirement apply to `consumer.seek`? > Because in some scenarios, we need to create consumers first and then seek > a position or timestamp. > > > Yunze Xu <y...@streamnative.io.invalid> 于2023年3月21日周二 17:19写道： > > > First, I agree with Yubiao that we can avoid calling the `isDuplicate` > > method once this option is enabled. > > > > Then, I'm wondering in which case would users want to disable this > > option? What's the disadvantage to disable the option? I think we can > > just record the latest position (ledger id, entry id, batch index) of > > the message received if the subscription type is Exclusive or > > Failover. > > > > Is there any breaking change if we just apply this filter without > > adding a configuration option? > > > > Thanks, > > Yunze > > > > On Tue, Mar 21, 2023 at 2:26 PM 丛搏 <congbobo...@gmail.com> wrote: > > > > > > Hi, Michael > > > > > > Michael Marshall <mmarsh...@apache.org> 于2023年3月21日周二 13:03写道： > > > > > > > > This is a great problem to improve. > > > > > > > > What if we instead expand the CommandSubscribe [0] protocol message > > > > with a new field to represent the client's desired read position? This > > > > way, the client can tell the second broker where to start sending > > > > messages, and there is no need to send the messages twice. > > > > > > > > I like the protocol expansion because it saves on unnecessary network > > > > transfer in several places and because it will be more straightforward > > > > for clients in other languages to implement. > > > > > > > > What do you think? > > > if we add the new field in CommandSubscribe, we should ensure > > > the synchronization between consumer reconnection and user > > > calling receive and redeliverUnack method. it will affect the performance > > > of receive. expose synchronization to hot paths it not a good idea. > > > Although the message is re-delivered twice, I don't think it > > > will cause too much performance loss. > > > > > > This filtering is rigorous, and there cannot be some race condition > > problems > > > because it involves transactions. I want it to be simple and efficient, > > > and I don't want it to become complicated and difficult to maintain. > > > > > > Of course, if the failover and exclusive consumers are changed to pull > > mode, > > > I believe that the change protocol is a very good idea. But at present, > > > there is obviously no sufficient reason to do so. > > > > > > Thanks, > > > Bo > > > > > > > > > > > Thanks, > > > > Michael > > > > > > > > [0] > > https://github.com/apache/pulsar/blob/af1360fb167c1f9484fda5771df3ea9b21d1440b/pulsar-common/src/main/proto/PulsarApi.proto#L339-L400 > > > > > > > > > > > > On Mon, Mar 20, 2023 at 10:56 AM Xiangying Meng <xiangy...@apache.org> > > wrote: > > > > > > > > > > Hi Congbo, > > > > > I think this is a great idea. > > > > > This is more efficient in filtering duplicate messages for a single > > > > > consumer. > > > > > And maybe more details about implementation should be shown in the > > proposal. > > > > > > > > > > Best regards, > > > > > Xiangying > > > > > > > > > > On Mon, Mar 20, 2023 at 10:53 PM Yubiao Feng > > > > > <yubiao.f...@streamnative.io.invalid> wrote: > > > > > > > > > > > Hi Bo > > > > > > > > > > > > I think this is a good way to filter messages that the client has > > received. > > > > > > > > > > > > And I have two questions: > > > > > > > > > > > > 1. This is more powerful than the original way > > > > > > (`acknowledgmentsGroupingTracker.isDuplicate(msgId)) to filter out > > > > > > duplicated messages. > > > > > > Is it possible to turn off the original de-replay logic to improve > > > > > > performance after enabling this new feature? > > > > > > > > > > > > 2. There should be a typo in the article > > > > > > > > > > > > > ## Only support Consumer#redeliverUnacknowledgedMessages() > > > > > > > If we redeliver individual messages, they will be filtered. > > Because we > > > > > > can't clear the record latest message > > > > > > >in the consumer when redelivering individual messages. It will > > make this > > > > > > config unclear, and if every redeliver > > > > > > > method changes, it will bring a lot of redundant code, which is > > difficult > > > > > > to maintain. If there is a need in the > > > > > > > future, just support it. > > > > > > > > > > > > I suppose you want to say not support > > `redeliverUnacknowledgedMessages`, > > > > > > right? > > > > > > > > > > > > > > > > > > Thanks > > > > > > Yubiao Feng > > > > > > > > > > > > On Mon, Mar 20, 2023 at 10:21 PM 丛搏 <congbobo...@gmail.com> wrote: > > > > > > > > > > > > > Hi, pulsar community: > > > > > > > > > > > > > > I started a PIP about `Client consumer filter received messages`. > > > > > > > > > > > > > > PIP: https://github.com/apache/pulsar/issues/19864 > > > > > > > > > > > > > > Thanks, > > > > > > > Bo > > > > > > > > > > > > > > >
