Re: Messages stuck in queue 5.9.0

2014-04-05 Thread artnaseef
In JMX, can you see the subscriptions? Are they growing at all? Are the numbers of subscriptions consistent with the numbers expected? One thing that may help - check the stack traces on the broker and clients. Things to look for: are there consumer threads active on the clients? Is the broker

Re: Messages stuck in queue 5.9.0

2014-04-04 Thread Gary Tully
two config options to disable some (possibly related) thread complexity broker side: On 4 April 2014 03:48, Yin Wang wrote: > Can you try to disable cache consumer option? > Or can you use a plain jms consumer to try to reproduce the issue? > Just guess it's maybe

Re: Messages stuck in queue 5.9.0

2014-04-03 Thread Yin Wang
Can you try to disable cache consumer option? Or can you use a plain jms consumer to try to reproduce the issue? Just guess it's maybe related with cache consumer and spring jms template. 2014-03-29 4:19 GMT+08:00 lookers : > Just checked the queuesize and it is increasing when it is blocked. >

Re: Messages stuck in queue 5.9.0

2014-03-28 Thread lookers
Just checked the queuesize and it is increasing when it is blocked. -- View this message in context: http://activemq.2283324.n4.nabble.com/Messages-stuck-in-queue-5-9-0-tp4679266p4679651.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Messages stuck in queue 5.9.0

2014-03-28 Thread lookers
Thanks again Artnaseef, Just to answer your questions: no it is not a network of brokers. Standalone broker,producer,consumer. The producers is using failover in the connection URL. Yes the messages are persistent and use transactions. No, I'm not using JMSXGroups. I think that I posted a stack

Re: Messages stuck in queue 5.9.0

2014-03-27 Thread artnaseef
That helps. Is this a network of brokers? Are clients using transactions? In addition to the diagnostics Gary mentioned, grabbing a stack trace when this happens may be helpful. Are JMSXGroups in-use? Also, are the producers using the failover transport, or using any other technology that migh

Re: Messages stuck in queue 5.9.0

2014-03-27 Thread lookers
Hi Artnaseef, Unconsumed messages are detected with the application which is time sensitive is not updating. I am sure that there is data in the queue. When blocked for a period of time the enqueue will increase until the broker is restarted. The data payload is time stamped to I can tell that

Re: Messages stuck in queue 5.9.0

2014-03-27 Thread artnaseef
I recommend taking a step back and looking at a few fundamentals. First, how are unconsumed messages on the queue detected? The following seems to indicate there are none (pagedInMessage.size 0): {noformat} ReceiveQueue toPageIn: 200, Inflight: 0, pagedInMessages.size 0, enqueueCount: 120149, de

Re: Messages stuck in queue 5.9.0

2014-03-27 Thread lookers
Hi gtully, here is the output of the consumer when it fails 27/03/14 12:35:39 - [TRACE,MessageListenerAdapter:360] No result object given - no result to handle 27/03/14 12:35:39 - [TRACE,TransactionSynchronizationManager:140] Retrieved value [org.springframework.jms.listener.LocallyExposedJmsReso

Re: Messages stuck in queue 5.9.0

2014-03-26 Thread Gary Tully
On a tcp transport url, client or broker. tcp://localhost:2345?trace=true If broker side all commands are logged. If client side it is per connection in the clients log. On 26 Mar 2014 09:09, "lookers" wrote: > Hi gtully > where do you put the trace = true parameter. Is it on the consumer? > > R

Re: Messages stuck in queue 5.9.0

2014-03-26 Thread lookers
Hi gtully where do you put the trace = true parameter. Is it on the consumer? Regards lookers -- View this message in context: http://activemq.2283324.n4.nabble.com/Messages-stuck-in-queue-5-9-0-tp4679266p4679528.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Messages stuck in queue 5.9.0

2014-03-25 Thread lookers
Hi, Please find attached a log file and thread dump of the incident. The freeze happen around 08:13:51. I am using openwire to connect to the broker failover:(tcp://localhost:2345,tcp://localhost:2345)?initialReconnectDelay=100&randomize=false and consumer url tcp://localhost:2345 activemq.l

Re: Messages stuck in queue 5.9.0

2014-03-20 Thread lookers
Hi, I am unable to view messages in the queue through the JMX console when it is stuck I can browse other queues in the same broker though. lookers -- View this message in context: http://activemq.2283324.n4.nabble.com/Messages-stuck-in-queue-5-9-0-tp4679266p4679390.html Sent from the ActiveM

Re: Messages stuck in queue 5.9.0

2014-03-20 Thread lookers
There are no message selectors on this queue. There are on others, but they are processed in a different thread. There is just one consumer and it consumes messages in FIFO manner. they are consumer using spring jms template. -- View this message in context: http://activemq.2283324.n4.nabbl

Re: Messages stuck in queue 5.9.0

2014-03-20 Thread Gary Tully
any message selectors? Is it just one consumer that does not get messages? Are they stomp/operwire? I think the best bet to understand what is going is to capture full trace logging. Also trace=true on the openwire (or relevant) tcp transport url. Possibly a broker side thread dump. Any chance t

Re: Messages stuck in queue 5.9.0

2014-03-20 Thread lookers
There is no ttl on the messages. I have not tried browsing with JMS when they are stuck. I can try that when it fails again No there is nothing in the DLQ for this queue. It is happening quiet regularly. At lease twice a day. I have similar environment set up of different instance and it does not

Re: Messages stuck in queue 5.9.0

2014-03-20 Thread Gary Tully
that looks fine. Is there a ttl on the messages, anything in the DLQ? Can you browse the messages via jmx? On 20 March 2014 14:36, lookers wrote: > Hi gtully > > Here is a line in the logs with > ReceiveQueue toPageIn: 200, Inflight: 0, pagedInMessages.size 0, > enqueueCount: 120149, dequeueCount

Re: Messages stuck in queue 5.9.0

2014-03-20 Thread lookers
Hi gtully Here is a line in the logs with ReceiveQueue toPageIn: 200, Inflight: 0, pagedInMessages.size 0, enqueueCount: 120149, dequeueCount: 113644 | org.apache.activemq.broker.region.Queue | ActiveMQ BrokerService[localhost] Task-619 This was when the messages are stuck Does this give you an

Re: Messages stuck in queue 5.9.0

2014-03-20 Thread Gary Tully
is the inflight count 0 in the jconsole jmx view for your consumer? enable trace level logging for org.apache.activemq.broker in log4j and use jmx to have it refresh the log4j config - a broker operation. send another message to that dest and see what is in the logs. in other words, jmx and logg