Re: Problem With Network of Brokers 5.14.0

2016-09-15 Thread Geoffrey Mina
The consumers are setup with a failover() URI that has all 3 of our AMQ servers in the list. We have conduitSubscriptions=false so that we can achieve even load balancing across all of the consumers. I am not sure we hit a TTL limit as I was under the impression the messages were dequeuing, just

Re: Problem With Network of Brokers 5.14.0

2016-09-15 Thread Tim Bain
If that's true, the decreaseNetworkConsumerPriority property (see http://activemq.apache.org/networks-of-brokers.html for details) would prevent the problem. Though it could result in less even load balancing across consumers connected to different brokers; it depends on how the producers and cons

Re: Problem With Network of Brokers 5.14.0

2016-09-15 Thread Matt Pavlovich
How are your consumers setup? This looks like a mesh config.. I'm wondering if the max message hop ttl was reached and the messages are parked in that one broker. That scenarios is helped by splitting the consumerTTL and the messageTTL and not using networkTTL. Do you have a specific reason fo

Re: Problem With Network of Brokers 5.14.0

2016-09-15 Thread Geoffrey Mina
Thanks for responding, Tim. We only have about 50 destinations with almost no churn. The CPU at the time was minimal. Load was less than .5 on a 4 core system. RAM usage was less than 1G on 4G allocated to JVM heap. Maybe a good place to start is understanding what the correct troubleshooting

Re: Problem With Network of Brokers 5.14.0

2016-09-14 Thread Tim Bain
What's your volume of destinations, and how much churn do you have? Another user (Kevin Burton) experienced inefficiency in the destination GC algorithm with large numbers of short-lived destinations; if that sounds like your situation, I think he had some changes that never made it to trunk, whi

Problem With Network of Brokers 5.14.0

2016-09-14 Thread Geoffrey Mina
Greetings, I am pretty new to ActiveMQ as we just deployed into our stack. We had a network of brokers have a significant problem tonight due to (what appeared to be) a single broker. 2 of the 3 were processing messages OK and the third was queuing up and not processing messages quickly. This