Gary Tully wrote: > > A thread dump of the broker would help to diagnose what is going on. > > On 23 April 2010 14:44, alisenberg <lisenb...@gmail.com> wrote: > >> >> >> bsnyder wrote: >> > >> > On Thu, Apr 22, 2010 at 9:25 PM, alisenberg <lisenb...@gmail.com> >> wrote: >> >> >> >> This is the scenario: >> >> >> >> I open three web admin consoles and send 10.000 messages from each >> >> console >> >> at the same time. Each message is about 50k. I mark persistence. And >> all >> >> messages go to the same queue. When the Queue receives 20.000 >> messages, >> >> the >> >> broker appears to hang. >> >> I can see that tmp_storage is consuming all the allocated space so the >> >> “Temp >> >> percent used” reaches 100%. >> >> >> >> My first question is, why tmp_storage is growing if all messages were >> >> marked >> >> as persistent? >> >> >> >> After the broker hangs, I try to shut it down, but the only way is to >> >> force >> >> kill it. >> >> >> >> Second question: even if the tmp_storage is full, why the broker is >> not >> >> shutting down in a normal way? >> >> >> >> After killing it, I restart it but it hangs and does not complete the >> >> initialization. >> >> >> >> But, if I configure the “tempUsage” a Little bit higher, then the >> broker >> >> starts again and the problem gets solved. >> >> >> >> So, my last question is, why I need to increase tempUsage in order to >> >> restart the broker? >> > >> > What version of ActiveMQ are you using? How have you configured the >> > broker? Are you using the default configuration? What do you mean that >> > the broker is hanging? Are you unable to send anymore messages to it? >> > >> > I think your pursuit of the tempUsage is a red herring. Instead, I >> > think what you are experiencing is producer flow control: >> > >> > http://activemq.apache.org/producer-flow-control.html >> > >> > Trying disabling producer flow control via the conf/activemq.xml >> > config and see what happens. >> > >> > Bruce >> > -- >> > perl -e 'print >> > unpack("u30","D0G)u8...@4vyy9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*" >> > );' >> > >> > ActiveMQ in Action: http://bit.ly/2je6cQ >> > Blog: http://bruceblog.org/ >> > Twitter: http://twitter.com/brucesnyder >> > >> > >> >> I think I am not using flowcontrol. I´m using 5.3. The broker is not >> receiving messages any more and producers are stopped but no error is >> thrown. >> >> Also when I try to inspect that queue with jconsole, jconsole also hangs >> displaying only a gray screen. >> >> CPU and RAM seems to be ok. >> >> This is my configuration: >> >> <broker xmlns="http://activemq.apache.org/schema/core"; >> brokerName="localhost" dataDirectory="/data/activemq53/" useJmx="true" >> advisorySupport="false" useShutdownHook="true"> >> >> <plugins> >> <statisticsBrokerPlugin/> >> </plugins> >> >> <!-- Destination specific policies using destination names or >> wildcards --> >> <destinationPolicy> >> <policyMap> >> <policyEntries> >> <policyEntry queue=">" >> producerFlowControl="false"> >> <pendingQueuePolicy> >> <fileQueueCursor /> >> </pendingQueuePolicy> >> </policyEntry> >> <policyEntry topic=">" producerFlowControl="false"> >> <dispatchPolicy> >> <strictOrderDispatchPolicy/> >> </dispatchPolicy> >> <subscriptionRecoveryPolicy> >> <lastImageSubscriptionRecoveryPolicy/> >> </subscriptionRecoveryPolicy> >> </policyEntry> >> </policyEntries> >> </policyMap> >> </destinationPolicy> >> >> <destinationInterceptors> >> <virtualDestinationInterceptor> >> <virtualDestinations> >> <compositeQueue name="EVENTS.A"> >> <forwardTo> >> <queue physicalName="EVENTS.A.STATISTICS" /> >> <queue physicalName="EVENTS.A.WAREHOUSE" /> >> </forwardTo> >> </compositeQueue> >> </virtualDestinations> >> </virtualDestinationInterceptor> >> </destinationInterceptors> >> >> <persistenceAdapter> >> <kahaDB directory="/data/activemq53/kahadb" >> enableIndexWriteAsync="true" journalMaxFileLength="20mb"/> >> </persistenceAdapter> >> >> <systemUsage> >> <systemUsage sendFailIfNoSpace="true"> >> <memoryUsage> >> <memoryUsage limit="100 mb"/> >> </memoryUsage> >> <storeUsage> >> <storeUsage limit="80 gb" name="foo"/> >> </storeUsage> >> <tempUsage> >> <tempUsage limit="1 gb"/> >> </tempUsage> >> </systemUsage> >> </systemUsage> >> >> <transportConnectors> >> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616 >> "/> >> </transportConnectors> >> >> </broker> >> >> Thanks >> -- >> View this message in context: >> http://old.nabble.com/Broker-hangs-and-cant-restart-it-tp28337031p28341513.html >> Sent from the ActiveMQ - User mailing list archive at Nabble.com. >> >> > > > -- > http://blog.garytully.com > > Open Source Integration > http://fusesource.com > >
Sorry but I am not able to get the thread dump :( I found that if I change from filecursor to default cursor the problem is solved. I guess that filecursor is writing many files (data-TopicSubscription-#) to tmp_storage. With filecursor, when that directory reaches temp size, the problem appears. With default cursor no file is written, so that seems to fix the problem... Any idea of problems using filecursor? Thanks -- View this message in context: http://old.nabble.com/Broker-hangs-and-cant-restart-it-tp28337031p28345933.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
