+1 This suggestion is really awesome. The new solution can reduce the learning cost of Pulsar users and avoid them taking many detours. Yours, Xiangying
On Fri, Sep 9, 2022 at 8:05 PM Shivji Kumar Jha <shiv4...@gmail.com> wrote: > Created https://github.com/apache/pulsar/issues/17574 for Yunze's > suggestion on exposing the following in java client > > void acknowledgeCumulative(Map<String, MessageId> topicToMessageId); > > > Regards, > Shivji Kumar Jha > http://www.shivjijha.com/ > +91 8884075512 > > > On Fri, 9 Sept 2022 at 11:44, Shivji Kumar Jha <shiv4...@gmail.com> wrote: > > > Tarun is a colleague at Nutanix who is very eager to do his first patch > in > > an apache Project :-) > > > > Regards, > > Shivji Kumar Jha > > http://www.shivjijha.com/ > > +91 8884075512 > > > > > > On Wed, 7 Sept 2022 at 17:05, Yunze Xu <y...@streamnative.io.invalid> > > wrote: > > > >> Sure. I’m glad to see that. Just a little confused about who is Tarun? > >> > >> Thanks, > >> Yunze > >> > >> > >> > >> > >> > On Sep 6, 2022, at 17:40, Shivji Kumar Jha <shiv4...@gmail.com> > wrote: > >> > > >> > ++ Tarun > >> > > >> > Hi Yunze, > >> > > >> > We would love to have this. > >> > > >> > ```java > >> > // the key is the partitioned topic name like my-topic-partition-0 > >> > void acknowledgeCumulative(Map<String, MessageId> topicToMessageId); > >> > ``` > >> > > >> > If you are busy with other things, do you mind Tarun taking this up ? > >> Happy > >> > to have you as a reviewer. > >> > > >> > Regards, > >> > Shivji Kumar Jha > >> > http://www.shivjijha.com/ > >> > +91 8884075512 > >> > > >> > > >> > On Sun, 4 Sept 2022 at 21:25, Yunze Xu <y...@streamnative.io.invalid> > >> wrote: > >> > > >> >> I am busy on other things recently so there is no further update. But > >> >> I found there is already two methods to acknowledge multiple messages > >> >> in Java client. > >> >> > >> >> ```java > >> >> void acknowledge(Messages<?> messages) throws > PulsarClientException; > >> >> > >> >> void acknowledge(List<MessageId> messageIdList) throws > >> >> PulsarClientException; > >> >> ``` > >> >> > >> >> And here is the issue to track the catch up: > >> >> > >> >> https://github.com/apache/pulsar/issues/17428 > >> >> > >> >> Yunze > >> >> > >> >> > >> >> > >> >> > >> >>> On Sep 4, 2022, at 22:37, Asaf Mesika <asaf.mes...@gmail.com> > wrote: > >> >>> > >> >>> What eventually happened with this idea? > >> >>> > >> >>> On Fri, Jul 29, 2022 at 8:02 AM PengHui Li <codelipeng...@gmail.com > > > >> >> wrote: > >> >>> > >> >>>> +1 > >> >>>> > >> >>>> Penghui > >> >>>> On Jul 28, 2022, 20:14 +0800, lordcheng10 > <1572139...@qq.com.invalid > >> >, > >> >>>> wrote: > >> >>>>> Nice feature! > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> ------------------ Original ------------------ > >> >>>>> From: "Yunze Xu"<y...@streamnative.io.INVALID>; > >> >>>>> Date: 2022Äê7ÔÂ15ÈÕ(ÐÇÆÚÎå) ÍíÉÏ6:04 > >> >>>>> To: "dev"<dev@pulsar.apache.org>; > >> >>>>> Subject: [DISCUSS] User-friendly acknowledgeCumulative API on a > >> >>>> partitioned topic or multi-topics > >> >>>>> > >> >>>>> > >> >>>>> > >> >>>>> Hi all, > >> >>>>> > >> >>>>> Long days ago I opened a PR to support cumulative acknowledgement > >> >>>>> for C++ client, but it's controversial about whether should a > >> >>>>> partitioned consumer acknowledge a message ID cumulatively. > >> >>>>> > >> >>>>> See https://github.com/apache/pulsar/pull/6796 for discussion. > >> >>>>> > >> >>>>> Currently, the Java client acknowledges the specific partition of > >> the > >> >>>>> message ID, while the C++ client just fails when calling > >> >>>>> `acknowledgeCumulative` on a partitioned topic. However, even if > the > >> >>>>> Java client doesn't fail, it's not user friendly. > >> >>>>> > >> >>>>> Assuming users called `acknowledgeCumulative` periodically, there > >> is a > >> >>>>> chance that some messages of the specific partition has never been > >> >>>>> passed to the method. > >> >>>>> > >> >>>>> For example, a consumer received: > >> >>>>> > >> >>>>> P0-M0, P1-M0, P0-M1, P1-M1, P0-M2, P1-M2... > >> >>>>> > >> >>>>> And the user acknowledged every two messages, i.e. > >> >>>>> > >> >>>>> P0-M0, P0-M1, P0-M2 > >> >>>>> > >> >>>>> Eventually, partition 1 has never been acknowledged. > >> >>>>> > >> >>>>> User must maintain its own `Map<String, MessageId>` cache for a > >> >>>>> partitioned topic or multi-topics consumer with the existing > >> >>>>> `acknowledgeCumulative` API. > >> >>>>> > >> >>>>> Should we make it more friendly for users? For example, we can > make > >> >>>>> `acknowledgeCumulative` accept the map to remind users to maintain > >> >>>>> the map from topic name to message ID: > >> >>>>> > >> >>>>> ```java > >> >>>>> // the key is the partitioned topic name like my-topic-partition-0 > >> >>>>> void acknowledgeCumulative(Map<String, MessageId> > >> topicToMessageId); > >> >>>>> ``` > >> >>>>> > >> >>>>> For those who don't want to maintain the map by themselves, maybe > we > >> >>>>> can provide a simpler API like: > >> >>>>> > >> >>>>> ```java > >> >>>>> // acknowlegde all latest received messages > >> >>>>> void acknowledgeCumulative(); > >> >>>>> ``` > >> >>>>> > >> >>>>> and provide an option to enable this behavior. > >> >>>>> > >> >>>>> Do you have any suggestion on this idea? I will prepare a proposal > >> if > >> >>>>> there is no disagreement. > >> >>>>> > >> >>>>> Thanks, > >> >>>>> Yunze > >> >>>> > >> >> > >> >> > >> > >> >
