Are you saying that the manual solution (a configurable threshold on a broker's network connection to another broker) doesn't sound reasonable, or just that the automatic solution (where the broker would measure end-to-end latency all the way to the consumer) doesn't? I agree that the automatic solution is a) hard, and b) not in line with ActiveMQ's current approach of brokers being agnostic of the full route to a consumer (though an exception is made for decreaseNetworkConsumerPriority) so I don't expect it to be implemented anytime soon.
But I think that the manual configuration option would provide a reasonable but crude tool for dealing with high-latency networks, and it wouldn't violate the JMS ordering guarantees since messages would be discarded or not but never delivered out of order. The problem set is pretty straight-forward: we're given a pretty limited bandwidth allocation across a WAN, and although we've tried to estimate our bandwidth usage to make sure we're going to stay within it, fluctuations in usage patterns might push us beyond our allocation. If that happens, we'd like to continue processing in a degraded manner - some messages would be lost and all messages would be processed just before their message expiration time - rather than stop processing entirely, which is what happens now. So yes, ActiveMQ would be used here to throttle traffic instead of allowing the network link to do so, to provide better end-to-end functionality across the network of brokers as a whole. And although there's an obvious need for all ActiveMQ users to allocate adequate bandwidth for their needs, having this option in ActiveMQ would provide a safeguard for what happens if either the actual ActiveMQ usage or the available bandwidth changes unexpectedly. Your suggestion of adding jitter to the message expiration time is an interesting one that I hadn't considered. Since we're using Camel to send our messages and to set that expiration time, it would be non-trivial to implement, but it might give us a way to handle this if ActiveMQ doesn't. But I think having a single setting in the config of each ActiveMQ broker would be preferable to having to implement a way to use a jittery expiration date and then configure that jitter on every producer we configure, since it would reduce the odds of someone misconfiguring a producer and not getting protected. Tim On Sun, Sep 28, 2014 at 12:25 AM, artnaseef <a...@artnaseef.com> wrote: > Honestly, that doesn't sound reasonable to me. > > ActiveMQ has no solid way to know which messages *should* or *should not* > reach their destination before the TTL expires. That would require some > feedback from across the broker-to-broker connection so the sending broker > could take that into account. In addition, JMS has ordering guarantees > (although it's never wise to fully depend on them). > > The problem set sounds interesting - almost like ActiveMQ is being used as > a > throttle for network traffic. > > Anyway, how about altering the message TTLs plus or minus the maximum > expected network latency times two? Either using randomness or simply > alternating between + and -. In that way, some messages will still survive > the trip while others don't when the backlog of messages reaches that > critical point. And, some (most?) should expire before hitting the wire, > saving that wasted trip. > > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/ActiveMQ-stops-delivering-messages-to-consumers-when-saturating-a-high-latency-network-link-tp4685557p4685951.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. >
