With prefetch=0, the client polls the server then, right? Is polling frequency a settable value? (Though as I write this, I'm assuming if so, it would be set on the client-side.)
On Fri, Nov 15, 2013 at 4:55 AM, Gary Tully <gary.tu...@gmail.com> wrote: > prefetch=0 will do it, so long as you don't use a messagelistener > directly. via spring the listener does a receive(...) under the hood > so it will be ok with prefetch=0 > > On 14 November 2013 16:14, Ned Wolpert <ned.wolp...@imemories.com> wrote: > > After I say you wrote 'prefetchExtension=false' I looked it up and found > > this bug sounds exactly like what I'm hitting: > > https://issues.apache.org/jira/browse/AMQ-2651 which led me to you > talking > > on > > > http://grokbase.com/t/activemq/users/103bdh5cgx/prefetchextension-off-by-1-for-transacted-consumers-with-prefetchsize-0 > > > > So... right now I have prefetch=1.... and I'm using 5.3.0. WIth the > grails > > jms (spring) plugin, its auto-ack for messages, and they are in a > > transaction. So it sounds like I'm hitting this. Does > > prefetchExtension=false exist in 5.3? (Looks like it was fixed in 5.4) > > Should I really be using prefetch=0? In this one queue, I have 16 > > listeners now, and messages are usually in groups < 10 but take a long > time > > to process. (hours) > > > > > > On Wed, Nov 13, 2013 at 4:27 PM, Gary Tully <gary.tu...@gmail.com> > wrote: > > > >> can you try a different ack mode, like clientack or using transactions > >> - the prefetch will be deferred till the ack which will be later than > >> in the auto ack case. Also, in the transacted case, use the > >> destination policy prefetchExtension=false > >> > >> On 13 November 2013 14:54, Ned Wolpert <ned.wolp...@imemories.com> > wrote: > >> > Did anyone have an idea into what I could do different to route > messages > >> to > >> > idle consumers? Just came into the same situation this morning where > a > >> > queue has 1 message processing on one consumer, one message waiting, > and > >> 15 > >> > idle consumers. (See notes below for my current configs) > >> > > >> > > >> > On Wed, Nov 6, 2013 at 9:40 AM, Ned Wolpert < > ned.wolp...@imemories.com > >> >wrote: > >> > > >> >> Forgot to add, broker url only has one query param.... > >> >> > >> >> jms.prefetchPolicy.queuePrefetch=1 > >> >> > >> >> which, as I mentioned above, does seem to work. > >> >> > >> >> > >> >> On Tue, Nov 5, 2013 at 10:56 AM, Ned Wolpert < > ned.wolp...@imemories.com > >> >wrote: > >> >> > >> >>> I can see the preFetch values being set in the console, and they are > >> all > >> >>> one. I've not set priorities. > >> >>> > >> >>> These are 'java' processes, using groovy/grails. The same executable > >> on 4 > >> >>> boxes, each executable with 4 listeners, treaded. Using the grails > jms > >> >>> plugin, which wraps the Spring jms template configuration. > >> >>> (concurrentConsumers is set to 4 per instance) > >> >>> > >> >>> When I have 1000's of messages pending, all instances are working. > This > >> >>> issue is only really viewable when there is 10 messages working. > >> >>> > >> >>> The following is the (redacted) activemq.xml. I'm assuming this > config > >> >>> could be better. I should mention typical usage of our JMS server > has > >> a > >> >>> few consumers and tons of producers. Thirty queues. Most queues > process > >> >>> quickly and do not fill up. Two queues are for slow producers. The > >> goal is > >> >>> for the producers to send a message and break away, so we don't want > >> slow > >> >>> producers at all. Producers are very spiky.... from 10m/min to > bursts > >> of > >> >>> 100's/min. We have growth concern as that number is increasing > >> steadily. > >> >>> > >> >>> <beans > >> >>> xmlns="http://www.springframework.org/schema/beans"; > >> >>> xmlns:amq="http://activemq.apache.org/schema/core"; > >> >>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > >> >>> xsi:schemaLocation="http://www.springframework.org/schema/beans > >> >>> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd > >> >>> http://activemq.apache.org/schema/core > >> >>> http://activemq.apache.org/schema/core/activemq-core.xsd";> > >> >>> > >> >>> <bean > >> >>> > >> > class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> > >> >>> <property name="locations"> > >> >>> > <value>file:${activemq.base}/conf/credentials.properties</value> > >> >>> </property> > >> >>> </bean> > >> >>> > >> >>> <broker xmlns="http://activemq.apache.org/schema/core"; > >> >>> brokerName="stagingMQ" > >> >>> useJmx="true" > >> >>> enableStatistics="true" > >> >>> useLocalHostBrokerName="false" > >> >>> useLoggingForShutdownErrors="true" > >> >>> dataDirectory="XXXXX"> > >> >>> > >> >>> <managementContext> > >> >>> <managementContext createConnector="true" > >> >>> connectorPort="XXXXX"/> > >> >>> </managementContext> > >> >>> > >> >>> <persistenceAdapter> > >> >>> <journaledJDBC journalLogFiles="5" > >> >>> journalLogFileSize="20 Mb" > >> >>> dataDirectory="XXXXXX" > >> >>> createTablesOnStartup="false" > >> >>> useDatabaseLock="false" > >> >>> dataSource="#XXXXX"> > >> >>> </journaledJDBC> > >> >>> </persistenceAdapter> > >> >>> > >> >>> <destinationPolicy> > >> >>> <policyMap> > >> >>> <policyEntries> > >> >>> <policyEntry topic=">" producerFlowControl="true" > >> >>> memoryLimit="1mb"> > >> >>> <pendingSubscriberPolicy> > >> >>> <vmCursor /> > >> >>> </pendingSubscriberPolicy> > >> >>> </policyEntry> > >> >>> <policyEntry queue=">" producerFlowControl="true" > memoryLimit="30mb"> > >> >>> <pendingQueuePolicy> > >> >>> <vmQueueCursor/> > >> >>> </pendingQueuePolicy> > >> >>> </policyEntry> > >> >>> </policyEntries> > >> >>> </policyMap> > >> >>> </destinationPolicy> > >> >>> > >> >>> <transportConnectors> > >> >>> <transportConnector name="openwire" uri="XXXX"/> > >> >>> <transportConnector name="stomp" uri="XXXXX"/> > >> >>> </transportConnectors> > >> >>> > >> >>> </broker> > >> >>> > >> >>> <import resource="jetty.xml"/> > >> >>> <import resource="databaseconfig.xml"/> > >> >>> </beans> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> On Tue, Nov 5, 2013 at 9:27 AM, Paul Gale <paul.n.g...@gmail.com> > >> wrote: > >> >>> > >> >>>> Have you verified via broker logging that the prefetch values > you've > >> >>>> configured are being honored by the broker? Are consumer > priorities in > >> >>>> use? Are your consumers instances of the same executable or are > they > >> >>>> implemented individually? > >> >>>> > >> >>>> Can you post your broker configuration: activemq.xml? > >> >>>> > >> >>>> How are your clients implemented, e.g., technology: Ruby or Java > etc, > >> >>>> choice of client libraries? Just wondering. > >> >>>> > >> >>>> > >> >>>> Thanks, > >> >>>> Paul > >> >>>> > >> >>>> On Tue, Nov 5, 2013 at 10:28 AM, Ned Wolpert < > >> ned.wolp...@imemories.com> > >> >>>> wrote: > >> >>>> > Thanks for the response... > >> >>>> > > >> >>>> > Any idea on the round-robin not working? I have a queue with 16 > >> >>>> consumers, > >> >>>> > all have pre-fetch set to 1. Five consumers are actively > processing > >> >>>> > requests and 3 requests are pending.... the 11 other consumers > are > >> >>>> idle. > >> >>>> > History has shown that a new request may go to one of the 11 idle > >> >>>> works, > >> >>>> > but its like those 3 requests are reserved for some of the > working > >> >>>> ones. I > >> >>>> > can't figure out what setting would help this, or if this just > was a > >> >>>> bug > >> >>>> > with 5.3.... > >> >>>> > > >> >>>> > > >> >>>> > On Mon, Nov 4, 2013 at 4:37 PM, Christian Posta > >> >>>> > <christian.po...@gmail.com>wrote: > >> >>>> > > >> >>>> >> The clients should negotiate the correct open-wire (protocol > >> version) > >> >>>> >> so in theory the broker will be backward compatible with older > >> >>>> >> clients. Just make sure the activemq-openwire-legacy jar is on > the > >> >>>> >> classpath (should be by default). > >> >>>> >> > >> >>>> >> Of course I would test this out to make sure :) > >> >>>> >> > >> >>>> >> On Mon, Nov 4, 2013 at 10:20 AM, Ned Wolpert < > >> >>>> ned.wolp...@imemories.com> > >> >>>> >> wrote: > >> >>>> >> > Folks- > >> >>>> >> > > >> >>>> >> > I have a 5.3 installation that we're using, and I have 2 > >> >>>> questions for > >> >>>> >> it: > >> >>>> >> > > >> >>>> >> > 1) We have prefetch set to 1 for all of the message consumers > on > >> one > >> >>>> >> queue, > >> >>>> >> > where message handling is slow. But it still seems like > messages > >> >>>> aren't > >> >>>> >> > really 'round robin' to the next available message consumer. > I'll > >> >>>> see a > >> >>>> >> few > >> >>>> >> > consumers are free but messages are waiting around. Is there a > >> >>>> >> > configuration that can help? (I should note that the server > has > >> >>>> been > >> >>>> >> > running consistently for 9 months and it seems to be getting > >> >>>> worse.... > >> >>>> >> > would a restart help?) > >> >>>> >> > > >> >>>> >> > 2) We are looking to upgrade to 5.9. I haven't started the > >> process > >> >>>> of > >> >>>> >> > testing, but I wanted to see if this is a case where the 5.3 > >> >>>> clients need > >> >>>> >> > to be upgraded at the same time as the server, or if the > clients > >> >>>> can be > >> >>>> >> > rolled over a few weeks to 5.9 after the server gets updated? > >> >>>> >> > > >> >>>> >> > Thanks! > >> >>>> >> > > >> >>>> >> > -- > >> >>>> >> > Virtually, Ned Wolpert > >> >>>> >> > > >> >>>> >> > "Settle thy studies, Faustus, and begin..." --Marlowe > >> >>>> >> > >> >>>> >> > >> >>>> >> > >> >>>> >> -- > >> >>>> >> Christian Posta > >> >>>> >> http://www.christianposta.com/blog > >> >>>> >> twitter: @christianposta > >> >>>> >> > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > -- > >> >>>> > Virtually, Ned Wolpert > >> >>>> > > >> >>>> > "Settle thy studies, Faustus, and begin..." --Marlowe > >> >>>> > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Virtually, Ned Wolpert > >> >>> > >> >>> "Settle thy studies, Faustus, and begin..." --Marlowe > >> >>> > >> >> > >> >> > >> >> > >> >> -- > >> >> Virtually, Ned Wolpert > >> >> > >> >> "Settle thy studies, Faustus, and begin..." --Marlowe > >> >> > >> > > >> > > >> > > >> > -- > >> > Virtually, Ned Wolpert > >> > > >> > "Settle thy studies, Faustus, and begin..." --Marlowe > >> > >> > >> > >> -- > >> http://redhat.com > >> http://blog.garytully.com > >> > > > > > > > > -- > > Virtually, Ned Wolpert > > > > "Settle thy studies, Faustus, and begin..." --Marlowe > > > > -- > http://redhat.com > http://blog.garytully.com > -- Virtually, Ned Wolpert "Settle thy studies, Faustus, and begin..." --Marlowe
