there is a direct correlation between dispatch attempts and message
enqueues, what I think you are seeing is the broker trying to dispatch each
message in turn and each time determining that all consumes are full. The
spike should quickly dissipate once one attempt to dispatch each message is
made. So the spike is expected but should be short lived.
If your consumers have the resource to deal with a larger prefetch and the
likely hood is that they will consume all of the prefetched messages
increasing the prefetch limit is advantageous.
On 4 April 2010 15:38, dcheckoway <dchecko...@gmail.com> wrote:
>
> I'm experiencing an issue with ActiveMQ 5.3.1 broker. Here's my setup:
>
> - 1 queue that experiences heavy usage (the only relevant destination in
> this case)
> - 80 concurrentConsumers on the queue
> - consumer.prefetchSize not set (default = 1000, I believe)
>
> Occasionally my service sees a spike in traffic, and the queue may
> experience a backlog of tens of thousands of messages (which is obviously
> undesirable, but it's ok). As traffic subsides, the consumers are able to
> catch up with the backlog and all is well.
>
> BUT...here's the problem...
>
> When the # of messages (N) in the queue rises, there comes a point where
> the
> broker CPU spikes to 100%.
>
> if N >= (concurrentConsumers * prefetchSize) {
> cpu spikes;
> }
>
> In my case, 80,000 messages is the magic number where the CPU sikes. And
> the math adds up: 80 concurrent consumers * 1000 prefetch size. You can
> literally watch the CPU spike and stay spiked if N >= 80,000. And when N
> drops back down < 80,000, you can see the CPU come back down to nominal
> values. Predictable & reliable. But I'm never happy seeing 100% CPU or
> high load on my servers! :-)
>
> 1. What is ActiveMQ doing to cause the CPU spike?
>
> 2. Is this expected or is it a bug?
>
> 3. I'm obviously tempted to crank consumer.prefetchSize way up to avoid
> this, but what other problem(s) might I cause as a result?
>
> 4. I realize this is an amorphous question, but here goes anyway...which is
> worse: cranking up prefetchSize, or adding more consumers?
>
> Attached are thread dumps both in normal conditions (CPU nominal) and in
> the
> CPU spiked condition.
> http://old.nabble.com/file/p28133368/thread-dump-normal.log
> thread-dump-normal.log
> http://old.nabble.com/file/p28133368/thread-dump-cpu-spiked.log
> thread-dump-cpu-spiked.log
>
> Please let me know if you need any more detail or config.
>
> Thanks!
> Dan
> --
> View this message in context:
> http://old.nabble.com/100--CPU-spike-with-ActiveMQ-5.3.1-tp28133368p28133368.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com
