Hi, Enrico
I think we temporarily don't add the new API, it will make the users
very hard to use. we can try to improve the performance first.
the performance can improve:
1. create multi consumer clients to consume the multi-partition topic.
2. change the MultiTopicsConsumerImpl incoming queue, using
ConsumerImpl.batchReceive() directly when use
MultiTopicsConsumerImpl.batchReceive()
Thanks,
bo
Enrico Olivelli <eolive...@gmail.com> 于2022年10月20日周四 17:37写道：
>
> Il giorno gio 20 ott 2022 alle ore 11:09 丛搏 <bog...@apache.org> ha scritto:
> >
> > Hi,
> >
> > I have another solution, we only guarantee the invoke batch receive
> > returns the messages are the same topic message.
>
> This is going to reduce performances of applications that consume from
> a partitioned topic or using a pattern or multiple topics, the
> application will receive smaller batches.
> Also, in the current API you don't have this assumption.
>
> I think that if we want to go this way we have to add a new
> batchReceive API that guarantees that all the messages are from the
> same message
>
> batchReceiveSingleTopic()
>
> Enrico
>
>
> In this way, we don't
> > need to add the new API for ack. use example is:
> > ```
> > // consumer the multi-partition messages
> > while (true) {
> > Messages<String> messages = consumer.batchReceive();
> > Message<String> finalMessage = null;
> > for (Message<String> message : messages) {
> > process(message);
> > finalMessage = message;
> > }
> > if (finalMessage != null) {
> > consumer.acknowledgeCumulativeAsync(finalMessage);
> > }
> > }
> > ```
> > implementation pr: https://github.com/congbobo184/pulsar/pull/5
> >
> > Thanks,
> > bo
> >
> > Tarun Annapareddy <tarunannapareddy1...@gmail.com> 于2022年9月14日周三 17:46写道：
> > >
> > > Hi Enrico,
> > > I can understand the concern about the validation of the topic
> > > string key in the map we are using. In this case is it good to have
> > > Map<String, Message> topicToMessage? It will help us to validate that the
> > > message truly belongs to the topic/partition.
> > >
> > > Thank you,
> > > Tarun.
> > >
> > > On Wed, 14 Sept 2022 at 07:50, Yunze Xu <y...@streamnative.io.invalid>
> > > wrote:
> > >
> > > > Hi Enrico,
> > > >
> > > > > There is no way to confirm that the MessageId really belongs to the
> > > > > partition and you can pass whatever you want
> > > >
> > > > True. But still, the existing `acknowledgeCumulative` API is not
> > > > convenient to use. The original purpose for this new overload is
> > > > allowing users to maintain each partition's latest message ID to avoid
> > > > frequent ACK commands, then they can flush all these message IDs once.
> > > >
> > > > Though the keys are actually not used, a natural way is to maintain a
> > > > map, not a list or set. For a multi-topics consumer, we can also
> > > > add a constraint that the key must exist in an internal consumer. For
> > > > a single topic consumer, the key must be the topic name.
> > > >
> > > > Without the new API, users have to configure `acknowledgmentGroupTime`
> > > > and call `acknowledgeCumulative` for each message received. However,
> > > > the ACK grouping tracker is very simple that it can only configure the
> > > > grouping timeout. I think users need a way to control precisely for when
> > > > to send the ACK commands.
> > > >
> > > > In my design, `acknowledgeCumulative` that accepts a single message ID
> > > > will go through the ACK grouping tracker, while
> > > > `acknowledgeCumulative` that accepts a map should send ACK commands
> > > > immediately because it represents users cache the pending message IDs
> > > > by themselves.
> > > >
> > > > Thanks,
> > > > Yunze
> > > >
> > > >
> > > >
> > > >
> > > > > On Sep 13, 2022, at 18:36, Enrico Olivelli <eolive...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > -1
> > > > > I am not sure this is much useful in this form.
> > > > > I had also commented on the PR: We pass a Map<String, MessageId> but
> > > > > actually the keys of the map will not be used.
> > > > >
> > > > > I understand that the current API is not user-friendly, but I don't
> > > > > think that adding a dummy parameter to the API will really
> > > > > help.
> > > > > There is no way to confirm that the MessageId really belongs to the
> > > > > partition and you can pass whatever you want
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Il giorno mar 13 set 2022 alle ore 12:24 Xiangying Meng
> > > > > <xiangy...@apache.org> ha scritto:
> > > > >>
> > > > >> +1 (non-binding)
> > > > >> Good work!
> > > > >>
> > > > >> Sincerely,
> > > > >> Xiangying
> > > > >>
> > > > >> On Tue, Sep 13, 2022 at 6:15 PM Tarun Annapareddy <
> > > > >> tarunannapareddy1...@gmail.com> wrote:
> > > > >>
> > > > >>> Hi devs,
> > > > >>> This is the official thread to vote for Java Client support to
> > > > >>> Cumulative Acknowledge messages for multiple partitions or topics
> > > > >>>
> > > > >>> PIP Issue: https://github.com/apache/pulsar/issues/17574
> > > > >>> Discussion thread:
> > > > >>> https://lists.apache.org/thread/k090ftlqc149yr1cnprxb29vxg160131
> > > > >>> PR: https://github.com/apache/pulsar/pull/17577
> > > > >>>
> > > > >>> Thank you,
> > > > >>> Tarun.
> > > > >>>
> > > >
> > > >
