Il giorno mar 20 apr 2021 alle ore 19:01 Devin Bost <devin.b...@gmail.com> ha scritto: > > We've handled this with a function by just putting a sleep on incoming > messages. > For any flow that needs the pattern, we just drop the function into the > flow. So, Matteo has a good point. > > -- > Devin G. Bost > > On Tue, Apr 20, 2021, 10:55 AM Matteo Merli <matteo.me...@gmail.com> wrote: > > > The main point of the delayed delivery feature is to get messages out > > of order. That's why you can impose a per-message delay (as opposed to > > a per-producer delay). That also includes that different producers can > > have different settings and produce messages with different delays. > > > > Implementing an ordered delay processing is trivial to do in client > > side, and I don't see any advantages in doing so in broker. > > > > > this way we can prevent a message to reach the client in case it is > > still not ready to be delivered. > > Eventually that message will have to be sent to the client. If the > > client stops consuming (until the time to process a message), then the > > regular flow control works perfectly. > > > > More practically, the decision to not support delayed delivery on > > exclusive/failover subscriptions was made in order to keep the > > dispatcher simple. Since these dispatchers are designed for ordered > > delivery, the focus is more on efficiency and low usage of resources. > > For example, we don't keep track of pending messages and so on, since > > after a failure we just need to rollback the cursor.
Good points. I would like to implement this at least in the Java Client, and make it work out-of-the box. My main problem is that when you use some subscription types the feature does not work and there is no error or any kind of feedback on the consumer side. If we agree on implementing this at least in the Java Client I will create an issue with the details and check how to concretely implement it. Enrico > > > > Matteo > > > > -- > > Matteo Merli > > <matteo.me...@gmail.com> > > > > On Tue, Apr 20, 2021 at 5:15 AM Enrico Olivelli <eolive...@gmail.com> > > wrote: > > > > > > Il giorno mar 20 apr 2021 alle ore 06:37 Yuva raj <uvar...@gmail.com> > > > ha scritto: > > > > > > > > On Tue, 20 Apr 2021 at 07:40, Sijie Guo <guosi...@gmail.com> wrote: > > > > > > > > > Delayed delivery indicates out-of-order delivery. I am not sure how > > you can > > > > > preserve ordering with delayed delivery. > > > > > > Probably the best option is to simply stop dispatching messages when > > > you reach a message that has to be delivered in the future. > > > > > > But I would like to see this implemented on the broker side and not on > > > the client, this way we can prevent a message to reach the client > > > in case it is still not ready to be delivered. > > > > > > > > > Enrico > > > > > > > > > It would be good to clarify the > > > > > semantic before stepping into implementations. > > > > > > > > > > - Sijie > > > > > > > > > > > > > > +1 > > > > > > > > > On Mon, Apr 19, 2021 at 6:42 AM Enrico Olivelli <eolive...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > Penghui > > > > > > > > > > > > Il giorno lun 19 apr 2021 alle ore 15:11 PengHui Li > > > > > > <codelipeng...@gmail.com> ha scritto: > > > > > > > > > > > > > > Hi enrico, > > > > > > > > > > > > > > The delayed message feature is to support messages that can have > > any > > > > > > delay in a single partition. > > > > > > > So, if the broker dispatches the messages based on the available > > time > > > > > of > > > > > > the message, this will break the FIFO order. > > > > > > > > > > > > > > So, if I understand correctly, "the behaviour of delayed > > messages even > > > > > > > for Exclusive subscriptions, still preserving a strict FIFO > > order” > > > > > means > > > > > > messages of a partition has same delay time? > > > > > > > If so, I think we can achieve this at the client-side(Just check > > the > > > > > > head message is available or not) > > > > > > > > > > > > At a high level I would like that the feature works well for the > > user, > > > > > > independently from how it is implemented, > > > > > > infact now it is difficult to explain to users that they may > > receive > > > > > > unexpected messages if they are using some "wrong" subscription > > type. > > > > > > > > > > > > Regarding a possible implementation I wonder if we can implement > > it at > > > > > > the broker side in order to save resources: > > > > > > - do not transfer the message if not needed > > > > > > - do not need to bounce back the message (negativeAck ?) in case > > that > > > > > > the message is not ready to be consumed > > > > > > > > > > > > Holding the message on the client until the message is deliverable > > is > > > > > > a bit complicated to implement and if we go this way we will have > > to > > > > > > implement > > > > > > it on every client (Java....) > > > > > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > Penghui > > > > > > > On Apr 19, 2021, 8:00 PM +0800, Enrico Olivelli < > > eolive...@gmail.com>, > > > > > > wrote: > > > > > > > > Hi, > > > > > > > > I came across this great feature, Delayed Messages > > > > > > > > > > > > > > > https://github.com/apache/pulsar/wiki/PIP-26:-Delayed-Message-Delivery > > > > > > > > > > > > > > > > And I see that this feature does not work with Exclusive and > > Failover > > > > > > > > subscriptions. > > > > > > > > The reason explained in the PIP is that this is because we > > want to > > > > > > > > guarantee FIFO order. > > > > > > > > > > > > > > > > I wonder if we could implement the behaviour of delayed > > messages even > > > > > > > > for Exclusive subscriptions, still preserving a strict FIFO > > order. > > > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > -- > > > > *Thanks* > > > > > > > > *Yuvaraj L* > >
