Waiting in Reader from binary stream: that's expected. Another thread
in dispatcher is normal too.
I think you need to turn on debugging on the client side. See my
article on how to do it:
http://vchekan.blogspot.com/2008/08/tracing-in-nms-it-easy-to-turn-protocol.html
Vadim.
On Fri, Aug 29, 2008 at 8:01 AM, Bryan Murphy <[EMAIL PROTECTED]> wrote:
> More information...
> There were ~ 4,000 messages still pending in the queue. I took a closer
> look at the two active consumers under jconsole, the consumer that was
> running had approximately 800 messages in it's
> MessageCountAwaitingAcknowledge, the consumer that was idle had 0.
>
> I put a few new messages into the message queue, and this caused the idle
> process to unblock and start processing messages. Whatever this did, it
> unstuck the idle process and I now have significantly less than 4,000
> messages and it's continuing to tick downwards.
>
> It's pretty clear that ActiveMQ decided to stop sending messages to my
> consumer.
>
> Bryan
>
> On Fri, Aug 29, 2008 at 9:40 AM, Bryan Murphy <[EMAIL PROTECTED]> wrote:
>
>> I changed my consumers to use client-acknowledge and immediately
>> acknowledge the message upon receipt. My handler does the following:
>>
>> void Handle(IMessage message)
>> {
>> log message
>> acknowledge message
>> Thread.Sleep(random amount of time);
>> }
>>
>> I bound the handler to consumer.Listen. I started two processes last
>> night, one that sleeps randomly from 0 to 1,000ms, and another that sleeps
>> randomly from 0 to 30,000ms.
>>
>> This morning, when I got in, the process that sleeps for up to 1000ms is
>> idle, and the process that sleeps for up to 30,000ms is still handling
>> messages.
>>
>> I attached the debugger to the idle process and it has three active
>> threads. One is in my code where I sit in a while(true) { Thread.Sleep(); }
>> loop to prevent the application from exiting, and the other two are ActiveMQ
>> threads.
>>
>> Thead appears to be blocked trying to read binary data from the tcp stream
>> :
>> * OpenWireBinaryReader.cs line 132
>> * OpenWireFormat.cs line 165
>> * TcpTransport.cs line 266
>>
>> Thread two is blocked waiting for an AutoResetEvent:
>> * DispatchingThread.cs line 125
>>
>> I opened up jconsole and took a look at the ActiveMQ process but I'm no
>> expert with that and couldn't find anything obviously wrong. I'm at a loss
>> what to do next.
>>
>> Thanks,
>> Bryan
>>
>> On Thu, Aug 28, 2008 at 9:01 PM, Bryan Murphy <[EMAIL PROTECTED]>wrote:
>>
>>> I read more about calling acknowledge, and as far as I can tell, because I
>>> was using transactional acknowledgement as well as session commit and
>>> rollback, the call to acknowledge would have been a no-op.
>>> Now, some of my transactions are in fact very long transactions. We have
>>> two sets of services, one that sits in our data center near our database,
>>> and another one that sits out on Amazon's EC2 cloud processing audio and
>>> video files and syncing them up to S3. The service by our database handles
>>> messages very quickly and never takes longer than a second (in theory),
>>> however, it often sends out new messages as part of the transaction (which
>>> I'll get to below).
>>>
>>> The services out on EC2 can take minutes to execute and hold the
>>> transaction open the whole time. It may very well be the case that ActiveMQ
>>> thinks my consumers are slow and is throttling them.
>>>
>>> The database handlers fail more often than the EC2 handlers. I found this
>>> odd at first, until I added some performance metrics to our tracing. It
>>> turns out (on my local machine at least) that establishing a connection can
>>> take upwards of 4.5 seconds. We establish a new connection and a new
>>> session every single time we send a message out. Most of the handlers on
>>> our database side send out at least one message, so the transactions are
>>> held open for at least that long.
>>>
>>> So, it's obvious to me that there's no connection pooling going on and
>>> reading the FAQ confirms this. Assuming there was connection pooling was
>>> clearly a bad assumption on my part. We don't use spring internally, so we
>>> have no pre-built components available to handle connection pooling. For
>>> now, I'm going to deal with it by simply moving away from transactional
>>> message processing, and immediately acknowledging every message when it is
>>> received. I'll deal with the consequences of failing message handlers
>>> later.
>>>
>>> I was also consuming messages in a very different fashion last time I used
>>> a non-transactional method, so perhaps this change in combination with that
>>> previous change will get my system into working order. Once again, I'll
>>> post back here with the results of these changes.
>>>
>>> If you notice any flaw in my reasoning, please do tell me. :)
>>>
>>> Thanks,
>>> Bryan
>>>
>>>
>
--
>From RFC 2631: In ASN.1, EXPLICIT tagging is implicit unless IMPLICIT
is explicitly specified
