I'd start by trying to figure out whether what's slow is the broker's delivery of messages to the ActiveMQ client code or the ActiveMQ client code's delivery of messages to your code, which is what those JMX stats would help you figure out.
You say you're not seeing any log lines about slow consumers; which slow consumer strategy are you using? And what's your prefetch buffer size for these consumers? On Sun, Mar 1, 2015 at 1:23 PM, Kevin Burton <bur...@spinn3r.com> wrote: > Thanks.. I’m investigating whether this is slow consumer activity but I > don’t have any log error messages marking them as slow. > > > On Sun, Mar 1, 2015 at 6:59 AM, Tim Bain <tb...@alumni.duke.edu> wrote: > > > Look in JMX to view all the consumers for a queue; for each one, look at > > the dispatched count and see whether it's around zero, around your > prefetch > > buffer size, or somewhere in the middle. > > > > If they're all near zero, that means the broker isn't delivering messages > > to the clients' prefetch buffer fast enough, though I don't have any > > guesses about what mighht cause that. If they're all near the prefetch > > size, then the client code isn't turning messages over to your code fast > > enough (possibly due to transactional behavior as Tim alluded to). And > if > > they're all in the middle, then i have no idea what that means. > > > > Tim > > On Feb 28, 2015 10:40 PM, "Tim Robbins" <tim.robb...@outlook.com> wrote: > > > > > Hi Kevin, > > > > > > Knowing how the clients are consuming messages could help, i.e. whether > > > it’s transactional or not, whether MessageConsumers are being > > > reused/pooled, etc. As a long shot, I’d try lowering the prefetch limit > > > client side. I’ve had to do this occasionally to work around > inefficient > > > Camel clients. > > > http://activemq.apache.org/what-is-the-prefetch-limit-for.html < > > > http://activemq.apache.org/what-is-the-prefetch-limit-for.html> > > > > > > Regards, > > > > > > Tim > > > > > > > On 1 Mar 2015, at 1:32 pm, Kevin Burton <bur...@spinn3r.com> wrote: > > > > > > > > I’m having a weird situation where ActiveMQ is refusing to send > > messages > > > to > > > > my consumers. > > > > > > > > I have 4 main queues that my consumers receive messages on. I have > > plenty > > > > of them. They have plenty of CPU, network, etc. > > > > > > > > I looked at their stack traces and they’re ALL waiting to receive > > > messages. > > > > > > > > ActiveMQ has about 7000 messages in these 4 queues that it’s just > > > trickling > > > > out to the consumers. VERY slow rate. > > > > > > > > The box it’s running on is fine. ActiveMQ has plenty of memory, > isn’t > > > > GCing. I also looked at its stack traces and nothing weird here. > > > > > > > > It’s just not sending at a very fast rate. > > > > > > > > Now, if I restart all my daemons, including ActiveMQ , everything > works > > > > fine for about 1-2 hours, then I get to this stall point again. > > > > > > > > I’ve read about the slow consumers. I don’t have any log warnings > for > > > this > > > > so I don’t think that’s a problem. (plus these are queues). > > > > > > > > Anything else I should look into? > > > > > > > > -- > > > > > > > > Founder/CEO Spinn3r.com > > > > Location: *San Francisco, CA* > > > > blog: http://burtonator.wordpress.com > > > > … or check out my Google+ profile > > > > <https://plus.google.com/102718274791889610666/posts> > > > > <http://spinn3r.com> > > > > > > > > > > > > -- > > Founder/CEO Spinn3r.com > Location: *San Francisco, CA* > blog: http://burtonator.wordpress.com > … or check out my Google+ profile > <https://plus.google.com/102718274791889610666/posts> > <http://spinn3r.com> >
