Yes, I agree. BTW, we can continue the discussion in your proposal. https://lists.apache.org/thread/3vd136jfhh7g0mhgwxrnqk7mx69qy17m
Thanks, Yunze > On Sep 14, 2022, at 14:17, Tarun Annapareddy <tarunannapareddy1...@gmail.com> > wrote: > > Hi, > There is a concern about the validation of the topic string key in the > map we are using. In this case is it good to have Map<String, Message> > topicToMessage? It will help us to validate that the message truly belongs > to the topic/partition. > > Thank you, > Tarun. > > On Fri, 9 Sept 2022 at 19:55, Xiangying Meng <xiangy...@apache.org> wrote: > >> +1 >> This suggestion is really awesome. >> The new solution can reduce the learning cost of Pulsar users and avoid >> them taking many detours. >> Yours, >> Xiangying >> >> On Fri, Sep 9, 2022 at 8:05 PM Shivji Kumar Jha <shiv4...@gmail.com> >> wrote: >> >>> Created https://github.com/apache/pulsar/issues/17574 for Yunze's >>> suggestion on exposing the following in java client >>> >>> void acknowledgeCumulative(Map<String, MessageId> topicToMessageId); >>> >>> >>> Regards, >>> Shivji Kumar Jha >>> http://www.shivjijha.com/ >>> +91 8884075512 >>> >>> >>> On Fri, 9 Sept 2022 at 11:44, Shivji Kumar Jha <shiv4...@gmail.com> >> wrote: >>> >>>> Tarun is a colleague at Nutanix who is very eager to do his first patch >>> in >>>> an apache Project :-) >>>> >>>> Regards, >>>> Shivji Kumar Jha >>>> http://www.shivjijha.com/ >>>> +91 8884075512 >>>> >>>> >>>> On Wed, 7 Sept 2022 at 17:05, Yunze Xu <y...@streamnative.io.invalid> >>>> wrote: >>>> >>>>> Sure. I’m glad to see that. Just a little confused about who is Tarun? >>>>> >>>>> Thanks, >>>>> Yunze >>>>> >>>>> >>>>> >>>>> >>>>>> On Sep 6, 2022, at 17:40, Shivji Kumar Jha <shiv4...@gmail.com> >>> wrote: >>>>>> >>>>>> ++ Tarun >>>>>> >>>>>> Hi Yunze, >>>>>> >>>>>> We would love to have this. >>>>>> >>>>>> ```java >>>>>> // the key is the partitioned topic name like my-topic-partition-0 >>>>>> void acknowledgeCumulative(Map<String, MessageId> topicToMessageId); >>>>>> ``` >>>>>> >>>>>> If you are busy with other things, do you mind Tarun taking this up >> ? >>>>> Happy >>>>>> to have you as a reviewer. >>>>>> >>>>>> Regards, >>>>>> Shivji Kumar Jha >>>>>> http://www.shivjijha.com/ >>>>>> +91 8884075512 >>>>>> >>>>>> >>>>>> On Sun, 4 Sept 2022 at 21:25, Yunze Xu <y...@streamnative.io.invalid >>> >>>>> wrote: >>>>>> >>>>>>> I am busy on other things recently so there is no further update. >> But >>>>>>> I found there is already two methods to acknowledge multiple >> messages >>>>>>> in Java client. >>>>>>> >>>>>>> ```java >>>>>>> void acknowledge(Messages<?> messages) throws >>> PulsarClientException; >>>>>>> >>>>>>> void acknowledge(List<MessageId> messageIdList) throws >>>>>>> PulsarClientException; >>>>>>> ``` >>>>>>> >>>>>>> And here is the issue to track the catch up: >>>>>>> >>>>>>> https://github.com/apache/pulsar/issues/17428 >>>>>>> >>>>>>> Yunze >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On Sep 4, 2022, at 22:37, Asaf Mesika <asaf.mes...@gmail.com> >>> wrote: >>>>>>>> >>>>>>>> What eventually happened with this idea? >>>>>>>> >>>>>>>> On Fri, Jul 29, 2022 at 8:02 AM PengHui Li < >> codelipeng...@gmail.com >>>> >>>>>>> wrote: >>>>>>>> >>>>>>>>> +1 >>>>>>>>> >>>>>>>>> Penghui >>>>>>>>> On Jul 28, 2022, 20:14 +0800, lordcheng10 >>> <1572139...@qq.com.invalid >>>>>> , >>>>>>>>> wrote: >>>>>>>>>> Nice feature! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ------------------ Original ------------------ >>>>>>>>>> From: "Yunze Xu"<y...@streamnative.io.INVALID>; >>>>>>>>>> Date: 2022Äê7ÔÂ15ÈÕ(ÐÇÆÚÎå) ÍíÉÏ6:04 >>>>>>>>>> To: "dev"<dev@pulsar.apache.org>; >>>>>>>>>> Subject: [DISCUSS] User-friendly acknowledgeCumulative API on a >>>>>>>>> partitioned topic or multi-topics >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Long days ago I opened a PR to support cumulative >> acknowledgement >>>>>>>>>> for C++ client, but it's controversial about whether should a >>>>>>>>>> partitioned consumer acknowledge a message ID cumulatively. >>>>>>>>>> >>>>>>>>>> See https://github.com/apache/pulsar/pull/6796 for discussion. >>>>>>>>>> >>>>>>>>>> Currently, the Java client acknowledges the specific partition >> of >>>>> the >>>>>>>>>> message ID, while the C++ client just fails when calling >>>>>>>>>> `acknowledgeCumulative` on a partitioned topic. However, even if >>> the >>>>>>>>>> Java client doesn't fail, it's not user friendly. >>>>>>>>>> >>>>>>>>>> Assuming users called `acknowledgeCumulative` periodically, >> there >>>>> is a >>>>>>>>>> chance that some messages of the specific partition has never >> been >>>>>>>>>> passed to the method. >>>>>>>>>> >>>>>>>>>> For example, a consumer received: >>>>>>>>>> >>>>>>>>>> P0-M0, P1-M0, P0-M1, P1-M1, P0-M2, P1-M2... >>>>>>>>>> >>>>>>>>>> And the user acknowledged every two messages, i.e. >>>>>>>>>> >>>>>>>>>> P0-M0, P0-M1, P0-M2 >>>>>>>>>> >>>>>>>>>> Eventually, partition 1 has never been acknowledged. >>>>>>>>>> >>>>>>>>>> User must maintain its own `Map<String, MessageId>` cache >> for a >>>>>>>>>> partitioned topic or multi-topics consumer with the existing >>>>>>>>>> `acknowledgeCumulative` API. >>>>>>>>>> >>>>>>>>>> Should we make it more friendly for users? For example, we can >>> make >>>>>>>>>> `acknowledgeCumulative` accept the map to remind users to >> maintain >>>>>>>>>> the map from topic name to message ID: >>>>>>>>>> >>>>>>>>>> ```java >>>>>>>>>> // the key is the partitioned topic name like >> my-topic-partition-0 >>>>>>>>>> void acknowledgeCumulative(Map<String, MessageId> >>>>> topicToMessageId); >>>>>>>>>> ``` >>>>>>>>>> >>>>>>>>>> For those who don't want to maintain the map by themselves, >> maybe >>> we >>>>>>>>>> can provide a simpler API like: >>>>>>>>>> >>>>>>>>>> ```java >>>>>>>>>> // acknowlegde all latest received messages >>>>>>>>>> void acknowledgeCumulative(); >>>>>>>>>> ``` >>>>>>>>>> >>>>>>>>>> and provide an option to enable this behavior. >>>>>>>>>> >>>>>>>>>> Do you have any suggestion on this idea? I will prepare a >> proposal >>>>> if >>>>>>>>>> there is no disagreement. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Yunze >>>>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>> >>
