Test case is great, thanks, I have opened https://issues.apache.org/activemq/browse/AMQ-2754 to track this, just validating the fix at the moment.
On 8 May 2010 07:43, Jamie McCrindle <jamiemccrin...@gmail.com> wrote: > Thanks Bruce, > > It's just the 2 brokers at the moment and prefetchSize is set to 1. It > looks like a combination of spring's dynamic connection scaling, > having maxMessagesPerTask set to 1 and the the network bridge > connection getting lost. > > cheers, > j. > > On Sat, May 8, 2010 at 1:16 AM, Bruce Snyder <bruce.sny...@gmail.com> > wrote: > > On Fri, May 7, 2010 at 3:19 PM, Jamie McCrindle > > <jamiemccrin...@gmail.com> wrote: > >> Greetings all, > >> > >> After some weeks of scratching my head, I _believe_ I have found the > >> magic combination that appears to be causing messages to become stuck > >> in our network of brokers. It's so convoluted that it's entirely > >> likely that the error isn't what I think it is but I have managed to > >> create a test case that mirrors the behaviour we're seeing live. And > >> it goes something like this: > >> > >> We have two brokers in a network of brokers. Producers were publishing > >> to a queue on one of the brokers and consumers reading off the queue > >> in the other broker. After a short while, messages would suddenly pile > >> up on the 'producer' broker and not get read off on the 'consumer' > >> broker. > >> > >> Pause for lots of random testing... > >> > >> It appeared that the network bridge subscription from the 'consumer' > >> broker was disappearing from the 'producer' broker, causing the pile > >> up. > >> > >> More testing later... > >> > >> And it looks like if we have a DefaultMessageListenerContainer with > >> the following configuration: > >> > >> maxMessagesPerTask: 1 > >> cacheLevel: CONSUMER > >> maxConcurrentConsumer: 3 (more than 1, basically) > >> concurrentConsumers: 1 > >> sessionAcknowledgeMode: Session.AUTO_ACKNOWLEDGE > >> > >> When Spring scales back the dynamic amount of consumers, the network > >> subscription appears to get lost and messages pile up on the producer > >> side. > >> > >> Workaround: > >> > >> Use concurrentConsumers instead of maxConcurrentConsumers so that > >> there are a static number of consumers (not setting maxMessagesPerTask > >> also seems to work). > > > > If this is a topology of more than two brokers, you may want to try > > setting the suppressDuplicateQueueSubscriptions=true on the network > > connector to improve the dispatching sufficiency. It essentially > > eliminates some of the duplicate routes to a given consumer. More info > > is available here: > > > > > http://activemq.apache.org/networks-of-brokers.html#NetworksofBrokers-NetworkConnectorProperties > > > > Also, there is a prefetchSize property on the network connector that > > is not documented on the ActiveMQ website. You may try setting that to > > 1 on the network connector. > > > > Bruce > > -- > > perl -e 'print > unpack("u30","D0G)u8...@4vyy9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*" > > );' > > > > ActiveMQ in Action: http://bit.ly/2je6cQ > > Blog: http://bruceblog.org/ > > Twitter: http://twitter.com/brucesnyder > > > -- http://blog.garytully.com Open Source Integration http://fusesource.com
