Forgot to add, broker url only has one query param.... jms.prefetchPolicy.queuePrefetch=1
which, as I mentioned above, does seem to work. On Tue, Nov 5, 2013 at 10:56 AM, Ned Wolpert <ned.wolp...@imemories.com>wrote: > I can see the preFetch values being set in the console, and they are all > one. I've not set priorities. > > These are 'java' processes, using groovy/grails. The same executable on 4 > boxes, each executable with 4 listeners, treaded. Using the grails jms > plugin, which wraps the Spring jms template configuration. > (concurrentConsumers is set to 4 per instance) > > When I have 1000's of messages pending, all instances are working. This > issue is only really viewable when there is 10 messages working. > > The following is the (redacted) activemq.xml. I'm assuming this config > could be better. I should mention typical usage of our JMS server has a > few consumers and tons of producers. Thirty queues. Most queues process > quickly and do not fill up. Two queues are for slow producers. The goal is > for the producers to send a message and break away, so we don't want slow > producers at all. Producers are very spiky.... from 10m/min to bursts of > 100's/min. We have growth concern as that number is increasing steadily. > > <beans > xmlns="http://www.springframework.org/schema/beans"; > xmlns:amq="http://activemq.apache.org/schema/core"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans-2.0.xsd > http://activemq.apache.org/schema/core > http://activemq.apache.org/schema/core/activemq-core.xsd";> > > <bean > class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> > <property name="locations"> > <value>file:${activemq.base}/conf/credentials.properties</value> > </property> > </bean> > > <broker xmlns="http://activemq.apache.org/schema/core"; > brokerName="stagingMQ" > useJmx="true" > enableStatistics="true" > useLocalHostBrokerName="false" > useLoggingForShutdownErrors="true" > dataDirectory="XXXXX"> > > <managementContext> > <managementContext createConnector="true" > connectorPort="XXXXX"/> > </managementContext> > > <persistenceAdapter> > <journaledJDBC journalLogFiles="5" > journalLogFileSize="20 Mb" > dataDirectory="XXXXXX" > createTablesOnStartup="false" > useDatabaseLock="false" > dataSource="#XXXXX"> > </journaledJDBC> > </persistenceAdapter> > > <destinationPolicy> > <policyMap> > <policyEntries> > <policyEntry topic=">" producerFlowControl="true" > memoryLimit="1mb"> > <pendingSubscriberPolicy> > <vmCursor /> > </pendingSubscriberPolicy> > </policyEntry> > <policyEntry queue=">" producerFlowControl="true" memoryLimit="30mb"> > <pendingQueuePolicy> > <vmQueueCursor/> > </pendingQueuePolicy> > </policyEntry> > </policyEntries> > </policyMap> > </destinationPolicy> > > <transportConnectors> > <transportConnector name="openwire" uri="XXXX"/> > <transportConnector name="stomp" uri="XXXXX"/> > </transportConnectors> > > </broker> > > <import resource="jetty.xml"/> > <import resource="databaseconfig.xml"/> > </beans> > > > > > On Tue, Nov 5, 2013 at 9:27 AM, Paul Gale <paul.n.g...@gmail.com> wrote: > >> Have you verified via broker logging that the prefetch values you've >> configured are being honored by the broker? Are consumer priorities in >> use? Are your consumers instances of the same executable or are they >> implemented individually? >> >> Can you post your broker configuration: activemq.xml? >> >> How are your clients implemented, e.g., technology: Ruby or Java etc, >> choice of client libraries? Just wondering. >> >> >> Thanks, >> Paul >> >> On Tue, Nov 5, 2013 at 10:28 AM, Ned Wolpert <ned.wolp...@imemories.com> >> wrote: >> > Thanks for the response... >> > >> > Any idea on the round-robin not working? I have a queue with 16 >> consumers, >> > all have pre-fetch set to 1. Five consumers are actively processing >> > requests and 3 requests are pending.... the 11 other consumers are idle. >> > History has shown that a new request may go to one of the 11 idle works, >> > but its like those 3 requests are reserved for some of the working >> ones. I >> > can't figure out what setting would help this, or if this just was a bug >> > with 5.3.... >> > >> > >> > On Mon, Nov 4, 2013 at 4:37 PM, Christian Posta >> > <christian.po...@gmail.com>wrote: >> > >> >> The clients should negotiate the correct open-wire (protocol version) >> >> so in theory the broker will be backward compatible with older >> >> clients. Just make sure the activemq-openwire-legacy jar is on the >> >> classpath (should be by default). >> >> >> >> Of course I would test this out to make sure :) >> >> >> >> On Mon, Nov 4, 2013 at 10:20 AM, Ned Wolpert < >> ned.wolp...@imemories.com> >> >> wrote: >> >> > Folks- >> >> > >> >> > I have a 5.3 installation that we're using, and I have 2 questions >> for >> >> it: >> >> > >> >> > 1) We have prefetch set to 1 for all of the message consumers on one >> >> queue, >> >> > where message handling is slow. But it still seems like messages >> aren't >> >> > really 'round robin' to the next available message consumer. I'll >> see a >> >> few >> >> > consumers are free but messages are waiting around. Is there a >> >> > configuration that can help? (I should note that the server has been >> >> > running consistently for 9 months and it seems to be getting >> worse.... >> >> > would a restart help?) >> >> > >> >> > 2) We are looking to upgrade to 5.9. I haven't started the process of >> >> > testing, but I wanted to see if this is a case where the 5.3 clients >> need >> >> > to be upgraded at the same time as the server, or if the clients can >> be >> >> > rolled over a few weeks to 5.9 after the server gets updated? >> >> > >> >> > Thanks! >> >> > >> >> > -- >> >> > Virtually, Ned Wolpert >> >> > >> >> > "Settle thy studies, Faustus, and begin..." --Marlowe >> >> >> >> >> >> >> >> -- >> >> Christian Posta >> >> http://www.christianposta.com/blog >> >> twitter: @christianposta >> >> >> > >> > >> > >> > -- >> > Virtually, Ned Wolpert >> > >> > "Settle thy studies, Faustus, and begin..." --Marlowe >> > > > > -- > Virtually, Ned Wolpert > > "Settle thy studies, Faustus, and begin..." --Marlowe > -- Virtually, Ned Wolpert "Settle thy studies, Faustus, and begin..." --Marlowe
