Hello all, Using ActiveMQ 5.8.0 we are seeing a message hoarding issue that only crops up when we use a Master/Slave configuration.
A single producer sends persistent, never expiring messages to two discrete Queues in the Broker in spurts. Each queue has a single polling consumer (so we when we look in the Admin Web Console we see two consumers attached to each queue: One is our consumer, the other is automatically created for the Slave), polling for max X messages every Y seconds, no selectors. When we have just a plain (not a master) instance of the broker + jdbc/mySQL, everything works as expected: messages are sent to the queues and delivered in order, 100% throughput. When we have the Master/Slave setup using JDBC (http://activemq.apache.org/jdbc-master-slave.html) and mySQL, every spurt of messages (well under the max # messages/poll) results in only some being delivered while others are hoarded in the database. Example: First spurt sends 10 messages, only 7 get delivered to the consumer. The other 3 are stored in the database (ACTIVEMQ_MSGS table). When the next spurt comes, let's say 13 messages, only 8 of those get delivered and 5 of them are stored in the database; so now the database has 8 total hoarded messages. Every successive spurt results in only partial sets delivered and more messages being hoarded. This is happening with both queues. Some possible points of interest: 1) The messages selected for hoarding appear random. They are not necessarily consecutive, they are not necessarily the first messages, nor necessarily last messages of each spurt. 2) If we restart the consumers, nothing happens; The hoarded messages remain in the DB, undelivered. 3) If we restart the broker, the hoarded messages ARE delivered (as expected since they are persistent and never expire, which ostensibly is the point of those options). 4) We get the same hoarding results whether our producer/consumer connects to the master broker directly using tcp, or using the failover protocol with both master and slave listed, or using the failover protocol with both master and slave listed and the randomize=false param. (This is not particularly surprising, just included for completeness). Our activemq.xml configuration: http://pastebin.com/Y87hZuh5 We've been trying every permutation of configuration we can think of to see where the culprit may lay (even swapping in KahaDB for mySQL => whole other can of worms, but then we saw it wasn't supported anyway: http://activemq.apache.org/kahadb-master-slave.html, a story for another day). Does anyone have any suggestions/pointers of where we should look next? How to get us back on track? It would be greatly appreciated, -h -- View this message in context: http://activemq.2283324.n4.nabble.com/Master-Slave-Message-Hoarding-tp4669591.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
