One thing missing from this discussion is details on the motivating use-case. How many delayed messages per second are we expecting? And what is the payload size?
> If consumer control the delayed message specific execution time we must > trust clock of consumer, this can cause delayed message process ahead of > time, some applications cannot tolerate this condition. This can be handled in a number of ways. Consumer clocks can be skewed with regard to other clocks, but it is generally safe to assume that clocks advance at the same rate, especially at the granularity of a couple of hours. So rather than specifying the absolute timestamp that the message should appear to the user, the dispatcher can specify the relative delay after dispatch that it should appear to the user. > > My concern of this category of approaches is "bandwidth" usage. It is > > basically trading bandwidth for complexity. > > @Sijie Guo <si...@apache.org> Agree with you, such an trading can cause the > broker's out going network to be more serious. I don't think PIP-26's approach may not use less bandwidth in this regard. With PIP-26, the msg ids are stored in a ledger, and when the timeout triggers it dispatches? Are all the delayed message being cached at the broker? If so, that is using a lot of memory, and it's exactly the kind of memory usage pattern that is very bad for JVM garbage collection. If not, then you have to read the message back in from bookkeeper, so the bandwidth usage is the same, though on a different path. In the client side approach, the message could be cached to avoid a redispatch. When I was discussing with Matteo, we discussed this. The redelivery logic has to be there in any case, as any cache (broker or client side) must have a limited size. Another option would be to skip sending the payload for delayed messages, and only send it when the client request redelivery, but this has the same issue with regard to the entry likely falling out the cache at the broker-side. -Ivan
