Hi, There is a concern about the validation of the topic string key in the map we are using. In this case is it good to have Map<String, Message> topicToMessage? It will help us to validate that the message truly belongs to the topic/partition.
Thank you, Tarun. On Fri, 9 Sept 2022 at 19:55, Xiangying Meng <xiangy...@apache.org> wrote: > +1 > This suggestion is really awesome. > The new solution can reduce the learning cost of Pulsar users and avoid > them taking many detours. > Yours, > Xiangying > > On Fri, Sep 9, 2022 at 8:05 PM Shivji Kumar Jha <shiv4...@gmail.com> > wrote: > > > Created https://github.com/apache/pulsar/issues/17574 for Yunze's > > suggestion on exposing the following in java client > > > > void acknowledgeCumulative(Map<String, MessageId> topicToMessageId); > > > > > > Regards, > > Shivji Kumar Jha > > http://www.shivjijha.com/ > > +91 8884075512 > > > > > > On Fri, 9 Sept 2022 at 11:44, Shivji Kumar Jha <shiv4...@gmail.com> > wrote: > > > > > Tarun is a colleague at Nutanix who is very eager to do his first patch > > in > > > an apache Project :-) > > > > > > Regards, > > > Shivji Kumar Jha > > > http://www.shivjijha.com/ > > > +91 8884075512 > > > > > > > > > On Wed, 7 Sept 2022 at 17:05, Yunze Xu <y...@streamnative.io.invalid> > > > wrote: > > > > > >> Sure. I’m glad to see that. Just a little confused about who is Tarun? > > >> > > >> Thanks, > > >> Yunze > > >> > > >> > > >> > > >> > > >> > On Sep 6, 2022, at 17:40, Shivji Kumar Jha <shiv4...@gmail.com> > > wrote: > > >> > > > >> > ++ Tarun > > >> > > > >> > Hi Yunze, > > >> > > > >> > We would love to have this. > > >> > > > >> > ```java > > >> > // the key is the partitioned topic name like my-topic-partition-0 > > >> > void acknowledgeCumulative(Map<String, MessageId> topicToMessageId); > > >> > ``` > > >> > > > >> > If you are busy with other things, do you mind Tarun taking this up > ? > > >> Happy > > >> > to have you as a reviewer. > > >> > > > >> > Regards, > > >> > Shivji Kumar Jha > > >> > http://www.shivjijha.com/ > > >> > +91 8884075512 > > >> > > > >> > > > >> > On Sun, 4 Sept 2022 at 21:25, Yunze Xu <y...@streamnative.io.invalid > > > > >> wrote: > > >> > > > >> >> I am busy on other things recently so there is no further update. > But > > >> >> I found there is already two methods to acknowledge multiple > messages > > >> >> in Java client. > > >> >> > > >> >> ```java > > >> >> void acknowledge(Messages<?> messages) throws > > PulsarClientException; > > >> >> > > >> >> void acknowledge(List<MessageId> messageIdList) throws > > >> >> PulsarClientException; > > >> >> ``` > > >> >> > > >> >> And here is the issue to track the catch up: > > >> >> > > >> >> https://github.com/apache/pulsar/issues/17428 > > >> >> > > >> >> Yunze > > >> >> > > >> >> > > >> >> > > >> >> > > >> >>> On Sep 4, 2022, at 22:37, Asaf Mesika <asaf.mes...@gmail.com> > > wrote: > > >> >>> > > >> >>> What eventually happened with this idea? > > >> >>> > > >> >>> On Fri, Jul 29, 2022 at 8:02 AM PengHui Li < > codelipeng...@gmail.com > > > > > >> >> wrote: > > >> >>> > > >> >>>> +1 > > >> >>>> > > >> >>>> Penghui > > >> >>>> On Jul 28, 2022, 20:14 +0800, lordcheng10 > > <1572139...@qq.com.invalid > > >> >, > > >> >>>> wrote: > > >> >>>>> Nice feature! > > >> >>>>> > > >> >>>>> > > >> >>>>> > > >> >>>>> > > >> >>>>> ------------------ Original ------------------ > > >> >>>>> From: "Yunze Xu"<y...@streamnative.io.INVALID>; > > >> >>>>> Date: 2022Äê7ÔÂ15ÈÕ(ÐÇÆÚÎå) ÍíÉÏ6:04 > > >> >>>>> To: "dev"<dev@pulsar.apache.org>; > > >> >>>>> Subject: [DISCUSS] User-friendly acknowledgeCumulative API on a > > >> >>>> partitioned topic or multi-topics > > >> >>>>> > > >> >>>>> > > >> >>>>> > > >> >>>>> Hi all, > > >> >>>>> > > >> >>>>> Long days ago I opened a PR to support cumulative > acknowledgement > > >> >>>>> for C++ client, but it's controversial about whether should a > > >> >>>>> partitioned consumer acknowledge a message ID cumulatively. > > >> >>>>> > > >> >>>>> See https://github.com/apache/pulsar/pull/6796 for discussion. > > >> >>>>> > > >> >>>>> Currently, the Java client acknowledges the specific partition > of > > >> the > > >> >>>>> message ID, while the C++ client just fails when calling > > >> >>>>> `acknowledgeCumulative` on a partitioned topic. However, even if > > the > > >> >>>>> Java client doesn't fail, it's not user friendly. > > >> >>>>> > > >> >>>>> Assuming users called `acknowledgeCumulative` periodically, > there > > >> is a > > >> >>>>> chance that some messages of the specific partition has never > been > > >> >>>>> passed to the method. > > >> >>>>> > > >> >>>>> For example, a consumer received: > > >> >>>>> > > >> >>>>> P0-M0, P1-M0, P0-M1, P1-M1, P0-M2, P1-M2... > > >> >>>>> > > >> >>>>> And the user acknowledged every two messages, i.e. > > >> >>>>> > > >> >>>>> P0-M0, P0-M1, P0-M2 > > >> >>>>> > > >> >>>>> Eventually, partition 1 has never been acknowledged. > > >> >>>>> > > >> >>>>> User must maintain its own `Map<String, MessageId>` cache > for a > > >> >>>>> partitioned topic or multi-topics consumer with the existing > > >> >>>>> `acknowledgeCumulative` API. > > >> >>>>> > > >> >>>>> Should we make it more friendly for users? For example, we can > > make > > >> >>>>> `acknowledgeCumulative` accept the map to remind users to > maintain > > >> >>>>> the map from topic name to message ID: > > >> >>>>> > > >> >>>>> ```java > > >> >>>>> // the key is the partitioned topic name like > my-topic-partition-0 > > >> >>>>> void acknowledgeCumulative(Map<String, MessageId> > > >> topicToMessageId); > > >> >>>>> ``` > > >> >>>>> > > >> >>>>> For those who don't want to maintain the map by themselves, > maybe > > we > > >> >>>>> can provide a simpler API like: > > >> >>>>> > > >> >>>>> ```java > > >> >>>>> // acknowlegde all latest received messages > > >> >>>>> void acknowledgeCumulative(); > > >> >>>>> ``` > > >> >>>>> > > >> >>>>> and provide an option to enable this behavior. > > >> >>>>> > > >> >>>>> Do you have any suggestion on this idea? I will prepare a > proposal > > >> if > > >> >>>>> there is no disagreement. > > >> >>>>> > > >> >>>>> Thanks, > > >> >>>>> Yunze > > >> >>>> > > >> >> > > >> >> > > >> > > >> > > >
