I have a cluster of ActiveMQ boxes and a network of consumers. The problem I'm having now is that the prefetch on the consumers fetches too many messages and then other machines in the cluster get starved out of work.
What I really want is a pattern where I can have say 50 ActiveMQ brokers and 200 worker servers each with their own set of consumers. I want each worker server to only be able read up to nrThreads (usually like 100-200) to perform the work. Reading more than nrThreads means its hogging work which is preventing other workers from consuming these messages. The only way I can really figure out a way to do this is to have the prefetch temporarily suspended on some of the consumers. I really with the ActiveMQ client were fully async via NIO/netty and not simulated async with threads. The threading model really really complicates things and makes things harder than they need to be. -- We’re hiring if you know of any awesome Java Devops or Linux Operations Engineers! Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts>
