After reading this thread I think I might have a workaround for this problem.
The main problem I have is stragglers. If I close a queue, and a moment later one messages is published, I’ll never be able to consume it. BUT… I could use this Advisory message: > ActiveMQ.Advisory.NoConsumer.Queue > No consumer is available to process messages being sent on a Queue … so basically I would do something along the lines of: - register listeners for ActiveMQ.Advisory.NoConsumer.Queue - wait 30 seconds until a queue is inactive. - mark that queue in my app as inactive - close the queue() - if any messages come in for ActiveMQ.Advisory.NoConsumer.Queue on that queue I immediately re-open a consumer. This should solve the straggler problem as I’ll immediately see if after it publishes. Further, normal GC would just work just the way it does now. We definitely use AMQ in a somewhat non-standard way. We have lots of small ephemeral queues which get flurries of messages, then cool down and go away. ActiveMQ has been GREAT so far and works well and even in this non-standard use case works great. This has been the only main issue we’re dealing with so hopefully I can resolve it easily. On Mon, Feb 2, 2015 at 6:10 AM, Tim Bain <tb...@alumni.duke.edu> wrote: > Kevin probably doesn't want a separate control channel given the volumes > he's been talking about; that would be twice as many destinations to GC. > Just send the "work is done" message via the real queue. > > But Tim's absolutely right about setting a message expiration time; either > make 100% sure every message will be consumed (which you give up on if > you're allowing the broker to infer completeness based on inactivity) or > make sure that when they get missed they go quickly into the bit bucket. > To that end, you probably want to use the DiscardingDLQPlugin to keep them > from sitting in the DLQ. > > Tim > On Feb 2, 2015 6:56 AM, "Timothy Bish" <tabish...@gmail.com> wrote: > > > On 02/02/2015 08:39 AM, Tim Bain wrote: > > > >> Also, the most common (I think) broker configuration will auto-create a > >> destination if a client tries to use it and it doesn't exist. So if you > >> delete the destination while a consumer is still connected, won't the > >> broker just immediately recreate it? If implemented, would this > >> enhancement actually do what you want? > >> > >> It seems like you need some means for the consumer to figure out that > the > >> producer is done and disconnect itself, which would then let the > existing > >> destination GC logic work without this enhancement. > >> On Feb 1, 2015 9:29 PM, "artnaseef" <a...@artnaseef.com> wrote: > >> > > In this case I'd guess it would be implemented such that the consumer > > would get closed via a ConsumerControl command or more harshly just > killing > > the consumer connection but if there were other consumers on that > > connection that could cause all sorts of problems in the users app. If > the > > connection was dropped and the consumer was using failover then it would > > attempt to reconnect and would end up recreating the destination. > > > > Doing this wouldn't really solve the scenario that the user is attempting > > to solve, that is that a message could arrive after the consumer shuts > down > > because as you've guessed the auto create feature would recreate the > > destination for a producer that suddenly appeared and published this > > presumed last second message. So adding the option to remove the Queue > > even if there are consumers doesn't really solve anything and more likely > > just creates a new set of issues to contend with. > > > > It'd be better to use some expiry on the messages and probably some > > control channel to tell the consumer that it's work is done and that it > is > > safe for it to close. > > > > Something has me confused here. If there is a consumer, why would > >>> messages > >>> sit around unconsumed? > >>> > >>> Also, I have trouble understanding why one would want the broker to > >>> remove > >>> a > >>> destination while it has a consumer -- what is the consumer doing with > >>> the > >>> destination then, and what is expected to happen to the consumer? > >>> > >>> Also, if messages are sitting in a queue, the queue cannot be GC'ed - > >>> otherwise the broker is losing messages. A good solution there may be > >>> the > >>> use of message TTLs. > >>> > >>> Perhaps a review of the architecture would help to determine the best > way > >>> to > >>> fit ActiveMQ and the application to the problem. > >>> > >>> BTW, it sounds like the application may have a consumer leak. > Consumers > >>> can > >>> be thought of as the center of purpose for a destination. A > destination > >>> that never has consumers would be a bit-bucket, or worse - a > >>> continually-growing message store. Consumers that don't expect to > >>> receive > >>> messages cannot be distinguished by ActiveMQ from those that do expect > to > >>> receive messages. > >>> > >>> Hope this helps. > >>> > >>> > >>> > >>> -- > >>> View this message in context: > >>> http://activemq.2283324.n4.nabble.com/ActiveMQ-should-GC- > >>> inactive-when-producers-is-0-even-if-consumers-is-0- > >>> tp4690776p4690797.html > >>> Sent from the ActiveMQ - User mailing list archive at Nabble.com. > >>> > >>> > > > > -- > > Tim Bish > > Sr Software Engineer | RedHat Inc. > > tim.b...@redhat.com | www.redhat.com > > skype: tabish121 | twitter: @tabish121 > > blog: http://timbish.blogspot.com/ > > > > > -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts> <http://spinn3r.com>
