Thank you! On Mon Feb 02 2015 at 9:06:40 PM Timothy Bish <tabish...@gmail.com> wrote:
> > On 02/02/2015 05:59 PM, Thiago Kronig wrote: > > Timothy, last question, I promise. > > > > Prefetch = 1 means that at most 2 messages will be dispatched to a > > consumer, the in-flight message and the "prefetched" one? I ask you > because > > setting to one resulted in four consumers with dispatched counter = 1, > > three consumers with dispatched counter = 2, and the other three > consumers > > waiting for messages, as seen via MBean. > > On Mon Feb 02 2015 at 8:34:48 PM Timothy Bish <tabish...@gmail.com> > wrote: > You can reduce the chance by turning of prefetch extension, see: > http://activemq.apache.org/per-destination-policies.html > > >> On 02/02/2015 05:27 PM, Thiago Kronig wrote: > >>> First of all, thank you Timothy and Tim for your time. > >>> > >>> In fact, setting jms.prefetchPolicy.all=0 solves the problem, because > >>> "*Specifying > >>> a prefetch limit of zero means the consumer will poll for more > messages, > >>> one at a time"*. > >>> > >>> Please, validate my understanding: the default prefetch size is 1000 > >>> messages for some queue costumer, which means that all my 10 consumers > >> were > >>> actually competing to dequeue 1000 messages from the broker. The > broker, > >> in > >>> some runnings dispatched 2 messages to one and 8 messages to another. > >> With competing consumers that have prefetch things will eventuall smooth > >> out to where the broker will round robin dispatch but only after they > >> are all online, prior to that the broker will dispatch based on the > >> arrival of the consumers so the first one likely got seven then the > >> second got one and then first got one...or other variations. > >> > >> You could set prefetch to one, but you can still have a slight imbalance > >> depending on how fast the messages get processed on each consumer. You > >> need to determine the optimal configuration based on your use case and > >> requirements. > >> > >>> On Mon Feb 02 2015 at 7:59:26 PM Timothy Bish <tabish...@gmail.com> > >> wrote: > >>>> On 02/02/2015 04:47 PM, Thiago Kronig wrote: > >>>>> I just tried the same approach using JMS consumers. No success. > >>>>> Source at: > >>>>> https://github.com/thiagokronig/activemq-camel- > >>>> test/blob/master/src/main/java/com/thiagokronig/activemqcameltest/ > >>>> ActiveMQJMSParallelTest.java > >>>>> On Mon Feb 02 2015 at 7:30:00 PM Thiago Kronig < > thiagokro...@gmail.com > >>>>> wrote: > >>>> You need to take consumer prefetch values into account when dealing > with > >>>> concurrent consumers. > >>>> http://activemq.apache.org/what-is-the-prefetch-limit-for.html > >>>> > >>>>>> Tim, you're right: 1 second is to small timeout to begin with. > >>>>>> But even if I set to 10 minutes, I get the same discouraging > results: > >> 3 > >>>>>> consumers, sometimes 8 consumers processing different messages in > >>>> parallel. > >>>>>> Sometimes 10 consumers, as my test requires, consumes all the > messages > >>>> in > >>>>>> parallel and exit before the `10 minute` timeout; but that is > >>>> *sometimes*. > >>>>>> On Mon Feb 02 2015 at 7:12:58 PM Tim Bain <tb...@alumni.duke.edu> > >>>> wrote: > >>>>>>> Are you sure that your assumption that a Camel context can be > >>>> initialized > >>>>>>> enough to consume messages in all 10 threads within 1 second is a > >> valid > >>>>>>> one? In my experience, Camel contexts aren't quick to start, and > it > >>>>>>> wouldn't surprise me if there was some asynchronous startup logic > >> that > >>>>>>> took > >>>>>>> place in parallel after control returns from you call to > >>>>>>> camelContext.addRoutes(). Can you point us to any documentation > that > >>>>>>> validates that startup is supposed to work as quickly as your code > >>>>>>> assumes? If not, you could increase the timeouts to 2 seconds, > and I > >>>>>>> would > >>>>>>> expect your test would start succeeding most/all of the time... > >>>>>>> > >>>>>>> On Mon, Feb 2, 2015 at 1:49 PM, Thiago Kronig < > >> thiagokro...@gmail.com> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Tim, other thing: I tried to Thread.sleep(5000); between the > >> producer > >>>>>>>> commit and the consumer initialization, but had no success. > >>>>>>>> > >>>>>>>> On Mon Feb 02 2015 at 6:45:19 PM Thiago Kronig < > >>>> thiagokro...@gmail.com> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Tim, in may scenario I want to consume N messages in N consumers, > >> one > >>>>>>> for > >>>>>>>>> each. My consumers wait on a barrier only to show the > parallelism, > >>>>>>> and to > >>>>>>>>> show that none of them processed two messages or more. Maybe my > >> Camel > >>>>>>>>> configuration is wrong, or there is something wrong with the > broker > >>>>>>>>> configuration. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Mon Feb 02 2015 at 6:17:38 PM Tim Bain <tb...@alumni.duke.edu > > > >>>>>>> wrote: > >>>>>>>>>> Why do your consumers all wait for the Nth message to be > received > >>>>>>> before > >>>>>>>>>> they return to process another? Why don't you use an > >> AtomicInteger > >>>>>>> to > >>>>>>>>>> subtract one each time, and succeed if you hit 0 and fail if you > >>>>>>> haven't > >>>>>>>>>> hit 0 by the end of your timeout interval? Based on what you've > >>>>>>> shows > >>>>>>>> us > >>>>>>>>>> so far, this appears to be an artifact of your test code, not a > >>>>>>> problem > >>>>>>>>>> with the broker. > >>>>>>>>>> > >>>>>>>>>> On Mon, Feb 2, 2015 at 1:02 PM, Thiago Kronig < > >>>>>>> thiagokro...@gmail.com> > >>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hi list. > >>>>>>>>>>> > >>>>>>>>>>> I'm trying to concurrently consume 10 messages sent to an > >> embedded > >>>>>>>>>> ActiveMQ > >>>>>>>>>>> broker over the VM transport. Sometimes my code works, > sometimes > >> it > >>>>>>>>>> hangs > >>>>>>>>>>> at the n-th message, randomly. > >>>>>>>>>>> > >>>>>>>>>>> If I change to a JBossMQ, my Camel client works. > >>>>>>>>>>> > >>>>>>>>>>> Can someone help me in setting my ActiveMQ broker? > >>>>>>>>>>> > >>>>>>>>>>> Source at: https://github.com/thiagokronig/activemq-camel-test > >>>>>>>>>>> > >>>>>>>>>>> Thanks in advance. > >>>>>>>>>>> > >>>> -- > >>>> Tim Bish > >>>> Sr Software Engineer | RedHat Inc. > >>>> tim.b...@redhat.com | www.redhat.com > >>>> skype: tabish121 | twitter: @tabish121 > >>>> blog: http://timbish.blogspot.com/ > >>>> > >>>> > >> > >> -- > >> Tim Bish > >> Sr Software Engineer | RedHat Inc. > >> tim.b...@redhat.com | www.redhat.com > >> skype: tabish121 | twitter: @tabish121 > >> blog: http://timbish.blogspot.com/ > >> > >> > > > -- > Tim Bish > Sr Software Engineer | RedHat Inc. > tim.b...@redhat.com | www.redhat.com > skype: tabish121 | twitter: @tabish121 > blog: http://timbish.blogspot.com/ > >
