> My main problem is that when you use some subscription types the feature does not work and there is no error or any kind of feedback on the consumer side.
If the problem is "there is no error or any kind of feedback on the consumer side", we should fix the error message. But I don't think we should go down the path of making every feature work for all subscription types. We should also avoid bringing complexity to the client as well. Once you add a feature in one language client, you have to consider adding it to all language clients. So we have to have a clear reason on why we are adding a feature. > If we agree on implementing this at least in the Java Client Are you suggesting implementing per-message-delay or delayed order processing on Failover/Exclusive subscriptions? It is unclear what you are proposing here. - Sijie On Tue, Apr 20, 2021 at 12:59 PM Enrico Olivelli <eolive...@gmail.com> wrote: > Il giorno mar 20 apr 2021 alle ore 19:01 Devin Bost > <devin.b...@gmail.com> ha scritto: > > > > We've handled this with a function by just putting a sleep on incoming > > messages. > > For any flow that needs the pattern, we just drop the function into the > > flow. So, Matteo has a good point. > > > > -- > > Devin G. Bost > > > > On Tue, Apr 20, 2021, 10:55 AM Matteo Merli <matteo.me...@gmail.com> > wrote: > > > > > The main point of the delayed delivery feature is to get messages out > > > of order. That's why you can impose a per-message delay (as opposed to > > > a per-producer delay). That also includes that different producers can > > > have different settings and produce messages with different delays. > > > > > > Implementing an ordered delay processing is trivial to do in client > > > side, and I don't see any advantages in doing so in broker. > > > > > > > this way we can prevent a message to reach the client in case it is > > > still not ready to be delivered. > > > Eventually that message will have to be sent to the client. If the > > > client stops consuming (until the time to process a message), then the > > > regular flow control works perfectly. > > > > > > More practically, the decision to not support delayed delivery on > > > exclusive/failover subscriptions was made in order to keep the > > > dispatcher simple. Since these dispatchers are designed for ordered > > > delivery, the focus is more on efficiency and low usage of resources. > > > For example, we don't keep track of pending messages and so on, since > > > after a failure we just need to rollback the cursor. > > Good points. > > I would like to implement this at least in the Java Client, and make > it work out-of-the box. > My main problem is that when you use some subscription types the > feature does not work > and there is no error or any kind of feedback on the consumer side. > > If we agree on implementing this at least in the Java Client I will > create an issue with the details and check how to concretely implement > it. > > Enrico > > > > > > > Matteo > > > > > > -- > > > Matteo Merli > > > <matteo.me...@gmail.com> > > > > > > On Tue, Apr 20, 2021 at 5:15 AM Enrico Olivelli <eolive...@gmail.com> > > > wrote: > > > > > > > > Il giorno mar 20 apr 2021 alle ore 06:37 Yuva raj <uvar...@gmail.com > > > > > > ha scritto: > > > > > > > > > > On Tue, 20 Apr 2021 at 07:40, Sijie Guo <guosi...@gmail.com> > wrote: > > > > > > > > > > > Delayed delivery indicates out-of-order delivery. I am not sure > how > > > you can > > > > > > preserve ordering with delayed delivery. > > > > > > > > Probably the best option is to simply stop dispatching messages when > > > > you reach a message that has to be delivered in the future. > > > > > > > > But I would like to see this implemented on the broker side and not > on > > > > the client, this way we can prevent a message to reach the client > > > > in case it is still not ready to be delivered. > > > > > > > > > > > > Enrico > > > > > > > > > > > > It would be good to clarify the > > > > > > semantic before stepping into implementations. > > > > > > > > > > > > - Sijie > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > On Mon, Apr 19, 2021 at 6:42 AM Enrico Olivelli < > eolive...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > Penghui > > > > > > > > > > > > > > Il giorno lun 19 apr 2021 alle ore 15:11 PengHui Li > > > > > > > <codelipeng...@gmail.com> ha scritto: > > > > > > > > > > > > > > > > Hi enrico, > > > > > > > > > > > > > > > > The delayed message feature is to support messages that can > have > > > any > > > > > > > delay in a single partition. > > > > > > > > So, if the broker dispatches the messages based on the > available > > > time > > > > > > of > > > > > > > the message, this will break the FIFO order. > > > > > > > > > > > > > > > > So, if I understand correctly, "the behaviour of delayed > > > messages even > > > > > > > > for Exclusive subscriptions, still preserving a strict FIFO > > > order” > > > > > > means > > > > > > > messages of a partition has same delay time? > > > > > > > > If so, I think we can achieve this at the client-side(Just > check > > > the > > > > > > > head message is available or not) > > > > > > > > > > > > > > At a high level I would like that the feature works well for > the > > > user, > > > > > > > independently from how it is implemented, > > > > > > > infact now it is difficult to explain to users that they may > > > receive > > > > > > > unexpected messages if they are using some "wrong" subscription > > > type. > > > > > > > > > > > > > > Regarding a possible implementation I wonder if we can > implement > > > it at > > > > > > > the broker side in order to save resources: > > > > > > > - do not transfer the message if not needed > > > > > > > - do not need to bounce back the message (negativeAck ?) in > case > > > that > > > > > > > the message is not ready to be consumed > > > > > > > > > > > > > > Holding the message on the client until the message is > deliverable > > > is > > > > > > > a bit complicated to implement and if we go this way we will > have > > > to > > > > > > > implement > > > > > > > it on every client (Java....) > > > > > > > > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Penghui > > > > > > > > On Apr 19, 2021, 8:00 PM +0800, Enrico Olivelli < > > > eolive...@gmail.com>, > > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > I came across this great feature, Delayed Messages > > > > > > > > > > > > > > > > > > https://github.com/apache/pulsar/wiki/PIP-26:-Delayed-Message-Delivery > > > > > > > > > > > > > > > > > > And I see that this feature does not work with Exclusive > and > > > Failover > > > > > > > > > subscriptions. > > > > > > > > > The reason explained in the PIP is that this is because we > > > want to > > > > > > > > > guarantee FIFO order. > > > > > > > > > > > > > > > > > > I wonder if we could implement the behaviour of delayed > > > messages even > > > > > > > > > for Exclusive subscriptions, still preserving a strict FIFO > > > order. > > > > > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > *Thanks* > > > > > > > > > > *Yuvaraj L* > > > >
