Le 15/01/10 11:25, Gary Tully a écrit : > A thread dump of the broker jvm, when the clients are stalled, would help > diagnose what is going on. btw, what activemq version are you using? > BrokerService - ActiveMQ 5.2.0 JMS Message Broker (localhost) is starting
I'll come back with broker dump when there is a problem again. This can be in several hours or within a few weeks. However i'll try to force a stalling next week... > 2010/1/15 David Delbecq <de...@oma.be> > > >> Le 15/01/10 10:24, Dejan Bosanac a écrit : >> >>> Hi David, >>> >>> you're probably blocked by "producer flow control". Take a look here >>> for more info: http://activemq.apache.org/producer-flow-control.html >>> >>> Cheers >>> >> I already took a look at it, read it three time to be sure i understand. >> It's quite technical but beyond that, i don't think this apply for me. >> My queues are all empty!!! (they stalled when there were 50 messages >> pending but where finally emptied), connections are not shared by >> several queues and am not sure how to implement the cursor part of >> documentation. There is no "slow consumer" that could influence it at >> that right moment (customers are running an waiting for new messages as >> you can see in capture). Moreover my messages are of about 1k payload >> (about 10 small text entries in a MapMessage), and my activeMQ config is >> like this >> <policyEntry queue=">" memoryLimit="500mb"/> >> <policyEntry topic=">" memoryLimit="500mb"> >> .... >> <systemUsage> >> <systemUsage> >> <memoryUsage> >> <memoryUsage limit="20 mb"/> >> </memoryUsage> >> <storeUsage> >> <storeUsage limit="1 gb" name="foo"/> >> </storeUsage> >> <tempUsage> >> <tempUsage limit="100 mb"/> >> </tempUsage> >> </systemUsage> >> </systemUsage> >> >> >> There is no way a bunch of 10 pending messages could fill a 1gb store, >> isn't it? What config part of activemq do i need to change to ensure >> that it can store at least about 10000 persistent messages (that makes >> only 10Mb of datas!!) before failing? Currently it fails and is never >> recovering without a restart of broker when there are only 50 small >> pending messages in the queue. As a workaround i have set >> "sendFailIfNoSpace" to true, so my client will fail instead of stalling, >> i'll see when this happen more quickly like that.... >> >> >> Here is broker part of my config file, can someone help me fix this? >> Thank you: >> >> >> <broker xmlns="http://activemq.apache.org/schema/core"; >> brokerName="localhost" dataDirectory="${activemq.base}/data" >> persistent="true"> >> <plugins> >> <simpleAuthenticationPlugin> >> <users> >> <authenticationUser username="admin" password="******" >> groups="users,admins"/> >> <authenticationUser username="sharkTool" password="********" >> groups="serverTools"/> >> <authenticationUser username="adminConsole" >> password="*********" >> groups="serverTools"/> >> </users> >> </simpleAuthenticationPlugin> >> <authorizationPlugin> >> <map> >> <authorizationMap> >> <authorizationEntries> >> <authorizationEntry queue=">" read="admins" write="admins" >> admin="admins" /> >> <authorizationEntry queue="WF.>" read="users,serverTools" >> write="serverTools" admin="admins,serverTools" /> >> >> <authorizationEntry topic=">" read="admins" write="admins" >> admin="admins" /> >> <authorizationEntry topic="WF.>" read="users,serverTools" >> write="serverTools" admin="admins,serverTools" /> >> >> <authorizationEntry topic="ActiveMQ.Advisory.>" >> read="guests,users,serverTools" write="guests,users,serverTools" >> admin="guests,users,serverTools"/> >> </authorizationEntries> >> >> <!-- let's assign roles to temporary destinations. comment >> this entry if we don't want any roles assigned to temp destinations >> <tempDestinationAuthorizationEntry> >> <tempDestinationAuthorizationEntry >> read="tempDestinationAdmins" write="tempDestinationAdmins" >> admin="tempDestinationAdmins"/> >> </tempDestinationAuthorizationEntry> --> >> </authorizationMap> >> </map> >> </authorizationPlugin> >> >> >> </plugins> >> <!-- Destination specific policies using destination names or >> wildcards --> >> <destinationPolicy> >> <policyMap> >> <policyEntries> >> <policyEntry queue=">" memoryLimit="500mb"/> >> <policyEntry topic=">" memoryLimit="500mb"> >> <!-- you can add other policies too such as these >> <dispatchPolicy> >> <strictOrderDispatchPolicy/> >> </dispatchPolicy> >> <subscriptionRecoveryPolicy> >> <lastImageSubscriptionRecoveryPolicy/> >> </subscriptionRecoveryPolicy> >> --> >> </policyEntry> >> </policyEntries> >> </policyMap> >> </destinationPolicy> >> >> <!-- Use the following to configure how ActiveMQ is exposed in >> JMX --> >> <managementContext> >> <managementContext createConnector="false"/> >> </managementContext> >> >> <!-- The store and forward broker networks ActiveMQ will listen >> to --> >> <networkConnectors> >> <!-- by default just auto discover the other brokers --> >> <!-- <networkConnector name="default-nc" >> uri="multicast://default"/>--> >> <!-- Example of a static configuration: >> <networkConnector name="host1 and host2" >> uri="static://(tcp://host1:61616,tcp://host2:61616)"/> >> --> >> </networkConnectors> >> >> <persistenceAdapter> >> <amqPersistenceAdapter syncOnWrite="false" >> directory="${activemq.base}/data" maxFileLength="20 mb"/> >> </persistenceAdapter> >> >> >> >> <sslContext> >> <sslContext keyStore="file:${activemq.base}/conf/broker.ks" >> keyStorePassword="password" >> trustStore="file:${activemq.base}/conf/broker.ts" >> trustStorePassword="password"/> >> </sslContext> >> >> <!-- The maximum about of space the broker will use before >> slowing down producers --> >> <systemUsage> >> <systemUsage> >> <memoryUsage> >> <memoryUsage limit="20 mb"/> >> </memoryUsage> >> <storeUsage> >> <storeUsage limit="1 gb" name="foo"/> >> </storeUsage> >> <tempUsage> >> <tempUsage limit="100 mb"/> >> </tempUsage> >> </systemUsage> >> </systemUsage> >> >> >> <!-- The transport connectors ActiveMQ will listen to --> >> <transportConnectors> >> <transportConnector name="openwire" >> uri="tcp://localhost:61616"/> >> <!-- <transportConnector name="ssl" >> uri="ssl://localhost:61617"/> >> <transportConnector name="stomp" uri="stomp://localhost:61613"/> >> <transportConnector name="xmpp" >> uri="xmpp://localhost:61222"/> --> >> </transportConnectors> >> <destinations> >> <queue physicalName="WF.Regraph" /> >> <queue physicalName="WF.Notify"/> >> <topic physicalName="WF.Events"/> >> </destinations> >> <destinationInterceptors> >> <virtualDestinationInterceptor> >> <virtualDestinations> >> <compositeQueue name="WF.IncomingEvents"> >> <forwardTo> >> <queue physicalName="WF.Regraph"/> >> <queue physicalName="WF.Notify"/> >> <topic physicalName="WF.Events" /> >> </forwardTo> >> </compositeQueue> >> </virtualDestinations> >> </virtualDestinationInterceptor> >> </destinationInterceptors> >> >> >> </broker> >> >> >> >>> -- >>> >>> >>> On Fri, Jan 15, 2010 at 10:24 AM, David Delbecq <de...@oma.be >>> <mailto:de...@oma.be>> wrote: >>> >>> Hello, >>> >>> can someone explain me why my activeMQ client is stalled in such >>> situation (i have 10 threads blocked like this having as >>> consequence a complete blocking of my client application): >>> >>> "http-8081-Processor6" daemon prio=10 tid=00841c38 nid=512 >>> lwp_id=2426611 waiting on condition [236f3000..236f153c] >>> java.lang.Thread.State: WAITING (parking) >>> at sun.misc.Unsafe.park(Native Method) >>> - parking to wait for <40ae1828> (a >>> >>> >> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) >> >>> at >>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) >>> at >>> >>> >> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1889) >> >>> at >>> >>> >> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317) >> >>> at >>> >>> >> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40) >> >>> at >>> >>> >> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80) >> >>> at >>> >>> >> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233) >> >>> at >>> org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1659) >>> - locked <40ae01e8> (a java.lang.Object) >>> at >>> >>> >> org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:227) >> >>> at >>> >>> >> org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241) >> >>> at >>> be.rmi.shark.JMSEventManager.sendEvent(JMSEventManager.java:146) >>> >>> >>> considering that the activeMQ lists size, as referenced by >>> activeMQ admin interface are as follow: >>> Name Number Of Pending Messages Number Of Consumers >>> Messages Sent Messages Received Views Operations >>> WF.Regraph 0 1 1714 1720 Browse Send To >>> >> Purge Delete >> >>> WF.Notify 0 1 1696 1742 Browse Send To >>> >> Purge Delete >> >>> WF.IncomingEvents 0 0 0 0 Browse >>> >> Send To Purge Delete >> >>> Is this to be considered normal that i must restart activeMQ >>> producers on a regular basis because of those this stalling? Where >>> can i find clear documentation on proper configuration of activeMQ >>> to prevent this stalling? Thank you. >>> >>> -- >>> David Delbecq >>> ICT >>> Institut Royal Météorologique >>> Ext:557 >>> >>> >>> >>> >> -- >> David Delbecq >> ICT >> Institut Royal Météorologique >> Ext:557 >> >> >> > > > -- David Delbecq ICT Institut Royal Météorologique Ext:557
