This is a great problem to improve. What if we instead expand the CommandSubscribe [0] protocol message with a new field to represent the client's desired read position? This way, the client can tell the second broker where to start sending messages, and there is no need to send the messages twice.
I like the protocol expansion because it saves on unnecessary network transfer in several places and because it will be more straightforward for clients in other languages to implement. What do you think? Thanks, Michael [0] https://github.com/apache/pulsar/blob/af1360fb167c1f9484fda5771df3ea9b21d1440b/pulsar-common/src/main/proto/PulsarApi.proto#L339-L400 On Mon, Mar 20, 2023 at 10:56 AM Xiangying Meng <xiangy...@apache.org> wrote: > > Hi Congbo, > I think this is a great idea. > This is more efficient in filtering duplicate messages for a single > consumer. > And maybe more details about implementation should be shown in the proposal. > > Best regards, > Xiangying > > On Mon, Mar 20, 2023 at 10:53 PM Yubiao Feng > <yubiao.f...@streamnative.io.invalid> wrote: > > > Hi Bo > > > > I think this is a good way to filter messages that the client has received. > > > > And I have two questions: > > > > 1. This is more powerful than the original way > > (`acknowledgmentsGroupingTracker.isDuplicate(msgId)) to filter out > > duplicated messages. > > Is it possible to turn off the original de-replay logic to improve > > performance after enabling this new feature? > > > > 2. There should be a typo in the article > > > > > ## Only support Consumer#redeliverUnacknowledgedMessages() > > > If we redeliver individual messages, they will be filtered. Because we > > can't clear the record latest message > > >in the consumer when redelivering individual messages. It will make this > > config unclear, and if every redeliver > > > method changes, it will bring a lot of redundant code, which is difficult > > to maintain. If there is a need in the > > > future, just support it. > > > > I suppose you want to say not support `redeliverUnacknowledgedMessages`, > > right? > > > > > > Thanks > > Yubiao Feng > > > > On Mon, Mar 20, 2023 at 10:21 PM 丛搏 <congbobo...@gmail.com> wrote: > > > > > Hi, pulsar community: > > > > > > I started a PIP about `Client consumer filter received messages`. > > > > > > PIP: https://github.com/apache/pulsar/issues/19864 > > > > > > Thanks, > > > Bo > > > > >
