I have producerFlowControl to false. I seperated my producer and consumers.
But still I was able to reproduce issue. For some time bunch of messages
started pile up in queue. Then they started to endup in DLQ. In DLQ I saw
most of them have these property:
dlqDeliveryFailureCause: java.lang.Throwable: Exceeded redelivery policy
limit:RedeliveryPolicy {destination = null, collisionAvoidanceFactor =
0.15, maximumRedeliveries = 6, maximumRedeliveryDelay = -1,
initialRedeliveryDelay = 1000, useCollisionAvoidance = false,
useExponentialBackOff = false, backOffMultiplier = 5.0, redeliveryDelay =
1000}, cause:null
Does this thing has anything to do with the issue?
I was not monitoring heap size when it got stuck. that will be next thing
to do. But I can hit webconsole from that activemq instance. So if it had
memory issues how come console is still working? isn't it same process?
thanks,
chirag
On Sat, Feb 8, 2014 at 12:36 AM, artnaseef <a...@artnaseef.com> wrote:
> How about a stack trace on the client? Can you look for the consumer
> threads
> and see what they are doing?
>
> Note that with producers and consumers on the same connection, it's
> possible
> to reach a deadlock if producer-flow-control kicks in because the entire
> connection is blocked, not just the one producer. There are two ways this
> can happen. First, if the client produces and consumes the same
> destination. Second, if the client is producing one destination and
> consuming a second destination and another client is consuming the first
> and
> producing to the second. Actually, more complicated possibilities exist as
> well.
>
> Your best bet is to never consume and produce on the same connection so
> that
> consumption never blocks due to producer-flow-control. In that scenario,
> deadlocks won't happen because consumers can always consume.
>
> Back to stack traces - if the consuming threads are blocked waiting to
> obtain a lock while producing threads hold that lock, that could indicate
> the problem.
>
> Anyway, slow consumption is quite often the cause of broker hangs, so
> looking for slow consumption and understanding speeding consumption (or
> slowing production) is important. If the broker JVM becomes completely
> unresponsive, try connecting jconsole or visualvm before the broker hangs
> and then watch memory. If the JVM is running out of Heap or Permgen space,
> that would explain the hung broker.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Broker-Hangs-after-some-time-or-does-nothing-tp4677506p4677656.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
