Thanks Chris:
> It seems like an application can already do something like this without
any
> changes to the clients?
Now we can't implement such logic. Although we now provide the interface of
reconsumerLater, the real situation is that if there are some problems in
the system, it may be difficult to receive the message correctly after a
retry. We need to manually trigger the logic of ReconsumerLater multiple
times, so whether we can implement ReconsumerLater In the form of automatic
fallback, allowing users to retry multiple times, instead of the current
solution that only retryes once.
> An application could read the message properties and then figure out the
> next time interval it should use the for the ReconsumeLater method. The
> leveling concept could be a small piece of code the application implements
> to figure out the next time to reconsume the message.
Yes, In the implementation of the Go SDK, we also put the time of the next
retry in the message properties, and call ReconsumerLater again by reading
the time in the message properties. Now our point of disagreement is
whether this part of the code logic is whether we encapsulate a new
interface and expose it to the user, or whether we let the user implement
this part of the code logic. I prefer the first option.
--
Thanks
Xiaolong Ran
Chris Kellogg <cckell...@gmail.com> 于2021年9月28日周二 下午11:45写道：
> It seems like an application can already do something like this without any
> changes to the clients?
>
> An application could read the message properties and then figure out the
> next time interval it should use the for the ReconsumeLater method. The
> leveling concept could be a small piece of code the application implements
> to figure out the next time to reconsume the message.
>
> I think this API is a little confusing as well. I'll for more flexibility
> if users need that. However, in my opinion if this feature is really needed
> an interface should be exposed as an API as opposed to a specific
> implementation.
>
>
>
> On Tue, Sep 28, 2021 at 4:01 AM PengHui Li <peng...@apache.org> wrote:
>
> > Hi Xiaolong,
> >
> > Currently, in the Pulsar client, we have ack timeout, negative ack, and
> > reconsume later method to achieve diverse message redelivery
> requirements.
> > I agree with the client side flexible message redelivery controlling but
> I
> > have a few concerns with the new API.
> >
> > 1. The new API looks very similar to the existing delay-queue based
> > implementation but It's very different in nature, which might confuse
> > users.
> > 2. Does the redelivery level can be specified by users? In my opinion, we
> > can provide a default exponentially backed off policy for users and we'd
> > better support customize it.
> > 3. I think if make some enhancements for the ack timeout is more
> > reasonable, the ack timeout handling will not use the delay queue such as
> > we have an AckTimePolicy there
> > And by default, we can support an ExponentiallyBackoffAckTimePolicy,
> > and XXXAckTimeoutPolicy and YYYAckTimeoutPolicy can be implemented by
> > users.
> >
> > Thanks,
> > Penghui.
> >
> > On Fri, Sep 10, 2021 at 4:33 PM r...@apache.org <ranxiaolong...@gmail.com
> >
> > wrote:
> >
> > > Hello everyone:
> > >
> > > I wrote a proposal to enhance the functionality of ReconsumeLater, the
> > > specific content is as follows:
> > >
> > > ---
> > >
> > > # PIP 94: Support level increment delay for ReconsumerLater interface
> > >
> > > - Status: Draft
> > > - Author: Xiaolong Ran
> > > - Pull request:
> > > - Mailing list discussion:
> > > - Release:
> > >
> > > The purpose of this proposal is mainly to add ReconsumerLater on the
> > > consumer side to retry in an incremental level
> > >
> > > ## Motivation
> > >
> > > At present, ReconsumrLater only supports specifying a specific delay
> time
> > > for distribution processing. The usage is as follows:
> > >
> > > ```
> > > while (true) {
> > > Message<String> msg = consumer.receive();
> > >
> > > try {
> > > // Process message...
> > >
> > > consumer.acknowledge(msg);
> > > } catch (Throwable t) {
> > > log.warn("Failed to process message");
> > > consumer.reconsumeLater(msg, 1000 , TimeUnit.MILLISECONDS);
> > > }
> > > }
> > > ```
> > >
> > > Its implementation principle is to use Pulsar's built-in delay message
> to
> > > pass in the specified time as the parameter
> > > of deliverAfter(), and then push the message to the consumer side again
> > > after the time arrives.
> > >
> > > This is a good idea, which allows users to flexibly define their own
> > delay
> > > time in a specific scenario. But assuming
> > > that the message is not processed correctly within the time specified
> by
> > > the user, the behavior of ReconsumerLater has
> > > ended at this time. Whether we can consider adding a retry scheme
> > according
> > > to the time level. Then when the first
> > > specified time range is not processed correctly, ReconsumerLater() can
> > > automatically retry according to the time level
> > > until the user correctly processes the specific message.
> > >
> > > ## Implementation
> > >
> > > As mentioned above, if we can here follow a certain delay level from
> low
> > to
> > > high and allow it to automatically retry,
> > > it is a more user-friendly way. For example, we can define the
> following
> > > delay level:
> > >
> > > ```
> > > MESSAGE_DELAYLEVEL = "1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m
> > 30m
> > > 1h 2h"
> > > ```
> > >
> > >
> > > In this PIP, we mainly introduce two new API interfaces to users:
> > >
> > > 1. Specify the delay level
> > >
> > > ```
> > > reconsumeLater(Message<?> message, int delayLevel)
> > > ```
> > >
> > > This implementation method is consistent with the current
> reconsumeLater
> > > interface, but instead of specifying the
> > > delay level, specify the specific delay time. For example, level `1`
> > > corresponds to `1s`, and level `3` corresponds to `10s`.
> > >
> > >
> > > 2. Retry with increasing level
> > >
> > > ```
> > > reconsumeLater(Message<?> message)
> > > ```
> > >
> > > Different from the above two, it is a back-off retry, that is, the
> retry
> > > interval after the first failure is 1 second,
> > > and the retry interval after the second failure is 5 seconds, and so
> on,
> > > the more the number of times, the longer the
> > > interval.
> > >
> > > This kind of retry mechanism often has more practical applications in
> > > business scenarios. If the consumption fails,
> > > the general service will not be restored immediately. It is more
> > reasonable
> > > to use this gradual retry method.
> > >
> > >
> > > ## Compatibility
> > >
> > > The current proposal will not cause damage to compatibility. It exposes
> > two
> > > new API interfaces based on the
> > > original API for users to use, so there will be no
> compatibility-related
> > > issues here.
> > >
> > > --
> > > Thanks
> > > Xiaolong Ran
> > >
> >
>
