Re: Configuring Distributed Queues in Store/Forward Network

2010-10-14 Thread Gary Tully
great that that helped. I don't think there is a need to adjust prefetch at runtime. The rate a which a prefetch is consumed is directly proportional to the speed of the consumer, so faster consumers will always get more messages and the messages will go elsewhere if it slows down. On 13 October 2

Re: Configuring Distributed Queues in Store/Forward Network

2010-10-13 Thread jpeng
Thanks Gary, I applied the configuration changes you suggested and the fast consumer in my test case now processes more messages. This gets me half way to where I would like to be, because consumer processing times in my system will vary and I need to adjust dynamically. Can prefetch be adjusted b

Re: Configuring Distributed Queues in Store/Forward Network

2010-10-12 Thread Gary Tully
had a quick look at the code, you use consumer.dispatchAsync=false which ensures that a single thread does the dispatch, which will not allow for any of the network latency to accumulate. Leave this at the default true so that dispatch happens using two threads. Also, set decreaseNetworkConsumerPri

Re: Configuring Distributed Queues in Store/Forward Network

2010-10-12 Thread jpeng
I have changed my test case and set the prefetch for the fast consumer to 100 and the prefetch for the slow consumer is still 1. The results didn't really change. 50 messages were processed by the fast consumer and 50 messages were processed by the slow consumer. Attached is the test case and th

Re: Configuring Distributed Queues in Store/Forward Network

2010-10-12 Thread schow
Dejan Bosanac wrote: > > You can set prefetchSize property on the network connector, which > should help with your use case. > See http://activemq.apache.org/networks-of-brokers.html for more details. > Unfortunately, setting the prefetchSize on the NetworkConnector URL does not change the beh

Re: Configuring Distributed Queues in Store/Forward Network

2010-10-11 Thread Dejan Bosanac
You can set prefetchSize property on the network connector, which should help with your use case. See http://activemq.apache.org/networks-of-brokers.html for more details. Cheers -- Dejan Bosanac - http://twitter.com/dejanb Open Source Integration - http://fusesource.com/ ActiveMQ in Action - htt

Re: Configuring Distributed Queues in Store/Forward Network

2010-10-08 Thread schow
> The prefetch limit determines how many messages can be sent to a > consumer to be processed. The default value for the queue prefetch > limit is 1000. Oftentimes this limit is fine for fast consumers that > never slow down, but it can easily flood a slow consumer. This is why > we recommend adju

Re: Configuring Distributed Queues in Store/Forward Network

2010-10-07 Thread Bruce Snyder
On Thu, Oct 7, 2010 at 2:36 PM, jpeng wrote: > > I am trying to setup distributed queue with a network of 3 brokers. 1 > producer, 1 fast consumer and 1 slow consumer. I cannot figure out how to > configure ActiveMQ distribute the messages properly. It seems that messages > are dispatched to each