not at the moment, a temp queue always keeps messages in memory. I know this has come up before and i think it is a sensible enhancement to allow the pending message cursor to be configured for a temp destination such that a temp destination need not be limited by memory. Can you raise a jira issue to track this enhancement so that it can get scheduled.
On the temp store cleanup, that depends a bit in the journal size used in the temp store, until a journal is fully unreferenced it can't be removed so that may be why it is not immediate in your case. But a test case should be able to demonstrate if there is a bug there. On 27 August 2012 13:36, Martin C. <mart...@gmx.at> wrote: > Hi, > > after some experiments, it seems that maybe one part of the problem is that > temp-queue messages are not spooled to disk. Is there a possibility to tell > ActiveMQ 5.5 to spool them to the temp store as well? > > Another issue I noticed, after purging a queue with non-persistent > messages, the space in the temp-store seems to not be freed. > > Best regards, > Martin > > On Mon, Aug 27, 2012 at 1:05 PM, Martin C. <mart...@gmx.at> wrote: > >> Hi, >> >> in one of the instances I didn't find anything in the broker log, just in >> the client log, in the second log I found the same log entry in both client >> and server: >> >> javax.jms.ResourceAllocationException: Usage Manager Memory Limit reached. >> Stopping producer (ID:hostname:26:33:1) to prevent flooding >> temp-queue://e1413fd6-2e46-76d5-dc62-df3260914cac:0 >> >> What would you suggest to use as a persistent <pendingQueuePolicy/> in my >> case, if I'd just want messages to go to disk if the memory-usage is >> approached? >> >> Thanks in advance! >> >> Best regards, >> Martin >> >> On Mon, Aug 27, 2012 at 12:25 PM, Gary Tully <gary.tu...@gmail.com> wrote: >> >>> what is out put to the broker log when it stops processing? It should >>> report the limit that is reached. >>> >>> The pendingQueueCursor will stop caching messages when the systemusage >>> limit reaches 70%, but it won't spool to disk. The persistent messages >>> will just remain in the store till they are needed. >>> >>> On 27 August 2012 10:59, Martin C. <mart...@gmx.at> wrote: >>> > Hi Gary, >>> > >>> > sorry for getting back to this rather old issue, but I just got an >>> issue in >>> > the field (twice) with the configuration below. The broker stopped >>> > accepting messages after it reached the 64MB memory limit, instead of >>> > spooling them out to disk. It basically is the previously discussed >>> > configuration with the <pendingQueuePolicy /> section removed. >>> > >>> > After increasing the MemoryLimit of the broker via JMX, the messages got >>> > processed again. The temp-store usage was at 0, the persistent store >>> usage >>> > at 11%. So, from my understanding, the broker should not have stopped >>> > accepting messages. Any hints? Or should I fall back to using >>> > <fileQueueCursor /> again? >>> > >>> > I am still on 5.5.0 and cannot upgrade that easily. >>> > >>> > Thanks in advance! >>> > >>> > Best regards, >>> > Martin >>> > >>> > The relevant parts of the configuration again. >>> > >>> > <broker xmlns="http://activemq.apache.org/schema/core"; >>> > brokerName="localhost" dataDirectory="${activemq.base}/data" >>> > populateJMSXUserID="true" useJmx="true"> >>> > >>> > <destinationPolicy> >>> > <policyMap> >>> > <policyEntries> >>> > <policyEntry topic=">" producerFlowControl="false" >>> > gcInactiveDestinations="false" inactiveTimoutBeforeGC="60000"> >>> > <vmCursor /> >>> > </policyEntry> >>> > <policyEntry queue=">" producerFlowControl="false" >>> > gcInactiveDestinations="false" inactiveTimoutBeforeGC="86400000"> >>> > <deadLetterStrategy> >>> > <individualDeadLetterStrategy queuePrefix="DLQ." >>> > useQueueForQueueMessages="true" >>> > processExpired="true" processNonPersistent="false" /> >>> > </deadLetterStrategy> >>> > </policyEntry> >>> > </policyEntries> >>> > </policyMap> >>> > </destinationPolicy> >>> > >>> > <persistenceAdapter> >>> > <kahaDB directory="${activemq.base}/data/kahadb" >>> > checkForCorruptJournalFiles="true" checksumJournalFiles="true" >>> > concurrentStoreAndDispatchTopics="false" >>> > concurrentStoreAndDispatchQueues="false" >>> > /> >>> > </persistenceAdapter> >>> > >>> > <systemUsage> >>> > <systemUsage sendFailIfNoSpace="true"> >>> > <memoryUsage> >>> > <memoryUsage limit="64 mb" /> >>> > </memoryUsage> >>> > <storeUsage> >>> > <storeUsage limit="10 gb" /> >>> > </storeUsage> >>> > <tempUsage> >>> > <tempUsage limit="1 gb" /> >>> > </tempUsage> >>> > </systemUsage> >>> > </systemUsage> >>> > </broker> >>> > >>> > On Wed, Nov 16, 2011 at 1:50 PM, Gary Tully <gary.tu...@gmail.com> >>> wrote: >>> > >>> >> yes. >>> >> >>> >> On 16 November 2011 12:01, Martin C. <mart...@gmx.at> wrote: >>> >> > Hi Gary, >>> >> > >>> >> > thanks for the very fast response. I will change the configuration >>> >> > accordingly. So basically you say I should remove the >>> >> > <pendingQueuePolicy> as a whole? >>> >> > >>> >> > Best regards, >>> >> > Martin >>> >> > >>> >> > On Wed, Nov 16, 2011 at 11:34 AM, Gary Tully <gary.tu...@gmail.com> >>> >> wrote: >>> >> >> You should not be using the file pending message cursor in this >>> case. >>> >> >> The default store cursor is best, it will use a file cursor for non >>> >> >> persistent messages and leave excess (exceed the cache) persistent >>> >> >> messages in the store. >>> >> >> When it is specified as the default cursor in configuration, it is >>> >> >> used for both persistent and non persistent messages. >>> >> >> >>> >> >> The file pending message cursor replays all messages on startup such >>> >> >> that it can do an append to the cursor on the next add but it is >>> only >>> >> >> intended for really slow stores, where reading from the local file >>> >> >> system is faster than reading from the store. This is not the case >>> >> >> with kahaDB. >>> >> >> >>> >> >> On 16 November 2011 10:26, Martin C. <mart...@gmx.at> wrote: >>> >> >>> Hi, >>> >> >>> >>> >> >>> I a have a question regarding fileQueueCursor and systemUsage on >>> >> >>> ActiveMQ 5.5.0: Below you see my configuration, which limits the >>> >> >>> memory to 64MB, store usage to 10GB and tempUsage to 1GB. >>> >> >>> >>> >> >>> I have a KahaDB with about 300000 messages, using 330MB in the >>> kahadb >>> >> >>> directory, all messages are persistent. If I start ActiveMQ, I see >>> the >>> >> >>> tmp_storage directory grow to 1GB, where it finally blocks and >>> >> >>> recovery of the messages starts hanging: >>> >> >>> >>> >> >>> [...] >>> >> >>> DEBUG | default:memory: usage change from: 2% of available memory, >>> to: >>> >> >>> 1% of available memory >>> >> >>> DEBUG | default:memory:queue://local.queue.myqueue:memory: usage >>> >> >>> change from: 1% of available memory, to: 0% of available memory >>> >> >>> DEBUG | default:memory: usage change from: 1% of available memory, >>> to: >>> >> >>> 0% of available memory >>> >> >>> INFO | cursor for queue://local.queue.myqueue has recovered 50000 >>> >> >>> messages. 15% complete >>> >> >>> INFO | cursor for queue://local.queue.myqueue has recovered 100000 >>> >> >>> messages. 31% complete >>> >> >>> INFO | cursor for queue://local.queue.myqueue has recovered 150000 >>> >> >>> messages. 47% complete >>> >> >>> INFO | cursor for queue://local.queue.myqueue has recovered 200000 >>> >> >>> messages. 62% complete >>> >> >>> >>> >> >>> The memory usage goes up to around 75%, where it starts dropping >>> down >>> >> >>> to the 0% from the logfile above. This is where it keeps hanging, >>> with >>> >> >>> the tmp_storage directory at 1GB diskspace. >>> >> >>> >>> >> >>> If I increase the tempUsage to 10GB, the startup works fine >>> >> >>> (tmp_storage becomes about 1.4GB in this case). Can you explain to >>> me, >>> >> >>> why I need to increase tempUsage in order to restart my broker >>> with a >>> >> >>> lot of persistent messages left? As far as I understood the >>> tempUsage >>> >> >>> is for storing non-persistent messages, persistent messages should >>> be >>> >> >>> affected by storeUsage? >>> >> >>> >>> >> >>> Thanks in advance! >>> >> >>> >>> >> >>> Best regards, >>> >> >>> Martin >>> >> >>> >>> >> >>> PS: the relevant parts from my configuration >>> >> >>> >>> >> >>> <broker xmlns="http://activemq.apache.org/schema/core"; >>> >> >>> brokerName="localhost" dataDirectory="${activemq.base}/data" >>> >> >>> populateJMSXUserID="true" useJmx="true"> >>> >> >>> >>> >> >>> <destinationPolicy> >>> >> >>> <policyMap> >>> >> >>> <policyEntries> >>> >> >>> <policyEntry topic=">" producerFlowControl="false" >>> >> >>> gcInactiveDestinations="false" inactiveTimoutBeforeGC="60000"> >>> >> >>> <vmCursor /> >>> >> >>> </policyEntry> >>> >> >>> <policyEntry queue=">" producerFlowControl="false" >>> >> >>> gcInactiveDestinations="false" inactiveTimoutBeforeGC="86400000"> >>> >> >>> <deadLetterStrategy> >>> >> >>> <individualDeadLetterStrategy queuePrefix="DLQ." >>> >> >>> useQueueForQueueMessages="true" >>> >> >>> processExpired="true" processNonPersistent="false" >>> /> >>> >> >>> </deadLetterStrategy> >>> >> >>> <pendingQueuePolicy> >>> >> >>> <fileQueueCursor /> >>> >> >>> </pendingQueuePolicy> >>> >> >>> </policyEntry> >>> >> >>> </policyEntries> >>> >> >>> </policyMap> >>> >> >>> </destinationPolicy> >>> >> >>> >>> >> >>> <persistenceAdapter> >>> >> >>> <kahaDB directory="${activemq.base}/data/kahadb" >>> >> >>> checkForCorruptJournalFiles="true" >>> checksumJournalFiles="true" >>> >> >>> concurrentStoreAndDispatchTopics="false" >>> >> >>> concurrentStoreAndDispatchQueues="false" >>> >> >>> /> >>> >> >>> </persistenceAdapter> >>> >> >>> >>> >> >>> <systemUsage> >>> >> >>> <systemUsage sendFailIfNoSpace="true"> >>> >> >>> <memoryUsage> >>> >> >>> <memoryUsage limit="64 mb" /> >>> >> >>> </memoryUsage> >>> >> >>> <storeUsage> >>> >> >>> <storeUsage limit="10 gb" /> >>> >> >>> </storeUsage> >>> >> >>> <tempUsage> >>> >> >>> <tempUsage limit="1 gb" /> >>> >> >>> </tempUsage> >>> >> >>> </systemUsage> >>> >> >>> </systemUsage> >>> >> >>> </broker> >>> >> >>> >>> >> >> >>> >> >> >>> >> >> >>> >> >> -- >>> >> >> http://fusesource.com >>> >> >> http://blog.garytully.com >>> >> >> >>> >> > >>> >> >>> >> >>> >> >>> >> -- >>> >> http://fusesource.com >>> >> http://blog.garytully.com >>> >> >>> >>> >>> >>> -- >>> http://fusesource.com >>> http://blog.garytully.com >>> >> >> -- http://fusesource.com http://blog.garytully.com
