Hi
we had a similar issue in our ActiveMQ broker. The reason was that we
had a topic that we use for guaranteed message delivery with message
persistence and durable subscribers. Some durable subscribers might be
offline and we spotted that even with a few MB worth of messages our
persistent store was growing to an unexplainable size.
The reason underneath was that we were using one kahadb instance for ALL
our messaging traffic. Since kahadb has a write only journal, the worst
case would be that our messages waiting for delivery would be spread
across journal files interleaved with all the other journal entries that
refer to already processed messages.
Gary has written a very good blog entry that describes our problem
nicely :
http://blog.garytully.com/2011/11/activemq-multiple-kahadb-instances.html
At the end we are now running with a kahadb split in 3 parts. 2 parts
serving a destination with occasinally offline consumers each and a
third part handling "the normal" JMS use case.
That solved our store problem pretty well.
For reference this is the interesting part of our config:
<amq:persistenceAdapter>
<amq:mKahaDB
directory="amq/mkahadb"
journalMaxFileLength="16m">
<amq:filteredPersistenceAdapters>
<amq:filteredKahaDB
topic="our.occasianally.disconnected.topic">
<amq:persistenceAdapter>
<amq:kahaDB
directory="amq/kahadb/topic"
journalMaxFileLength="16m"
/>
</amq:persistenceAdapter>
</amq:filteredKahaDB>
<amq:filteredKahaDB
queue="our.occasionally.disconnected.queue">
<amq:persistenceAdapter>
<amq:kahaDB
directory="amq/kahadb/queue"
journalMaxFileLength="16m"
/>
</amq:persistenceAdapter>
</amq:filteredKahaDB>
<amq:filteredKahaDB>
<amq:persistenceAdapter>
<amq:kahaDB
directory="amq/kahadb/default"
journalMaxFileLength="16m"
/>
</amq:persistenceAdapter>
</amq:filteredKahaDB>
</amq:filteredPersistenceAdapters>
</amq:mKahaDB>
</amq:persistenceAdapter>
Best regards
Andreas
On 26/09/14 17:57, petrk wrote:
Hi
I have a single instance of ActiveMQ v5.9 using persistent store and we are
constantly running out of space.
We use persistent messages with KahaDB and have 50gb store defined.
The queues directory only actually contains 26GB, but ActiveMQ admin says
"Store percent used 94", is that right?
Browsing the number of pending messages in queues, there is no way there is
26GB or 50GB worth of data!
I have noticed that we have lots of ActiveMQ.Advisory.Consumer.Queue.*
topics, which we don't use, but we use the advisoryWhenFull="true" in
conjuction with sendFailIfNoSpace="true" so that we get notified when
messages can't be queued. We also expire messages after 30 days, in hope
they will get deleted. If I disable advisory message am I going to lose
these two functions?
Any ideas?
Here are the relevant bits from our activemq.xml:
<bean id="store"
class="org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter" >
<property name="directory" value="/opt/xyz/queues" />
<property name="journalMaxFileLength" value="3145728" />
<property name="checkpointInterval" value="5000" />
<property name="cleanupInterval" value="5000" />
</bean>
<broker xmlns="http://activemq.apache.org/schema/core";
useJmx="true" brokerName="localhost" dataDirectory="/opt/xyz/queues"
persistenceAdapter="#store">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">">
<deadLetterStrategy>
<sharedDeadLetterStrategy processExpired="false"/>
</deadLetterStrategy>
</policyEntry>
<policyEntry queue=">" cursorMemoryHighWaterMark="90"
maxPageSize="10000"
queuePrefetch="10000" producerFlowControl="false" advisoryWhenFull="true">
<pendingQueuePolicy>
<storeCursor/>
</pendingQueuePolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<systemUsage>
<systemUsage sendFailIfNoSpace="true">
<memoryUsage>
<memoryUsage limit = "8 gb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="50 gb" store="#store"/>
</storeUsage>
<tempUsage>
<tempUsage limit="10 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<plugins>
<timeStampingBrokerPlugin
zeroExpirationOverride="2419200000"/>
<discardingDLQBrokerPlugin dropAll="true"
dropTemporaryTopics="true" dropTemporaryQueues="true" />
</plugins>
</broker>
--
View this message in context:
http://activemq.2283324.n4.nabble.com/Managing-persistent-store-space-tp4685913.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
--
Andreas Gies
WoQ -- Way of Quality GmbH
Geschäftsführer & CTO
/eMail:/andr...@wayofquality.de <mailto:andr...@wayofquality.de>
/Tel:/ +49 151 23470823
/Fax:/ +49 1805 006534 2114
/Twitter:/ andreasgies /Skype:/ giessonic
/LinkedIn:/ <http://de.linkedin.com/pub/andreas-gies/0/594/aa5/>
(http://de.linkedin.com/pub/andreas-gies/0/594/aa5/)
/Xing:/ <http://www.xing.com/profile/Andreas_Gies>
(http://www.xing.com/profile/Andreas_Gies)
/Blog:/ <http://www.wayofquality.de/index.php/en/blog>
(http://www.wayofquality.de/index.php/en/blog)
/Github:/ <https://github.com/atooni> (https://github.com/atooni)
/Amtsgericht Landshut:/HRB 8352//
//
/Ust.-Id.:/ DE274771254
Haftungsausschluss
Diese Email kann vertrauliche und/oder rechtlich geschützte
Informationen enthalten und ist ausschließlich für den/die benannten
Adressaten bestimmt. Sollten Sie nicht der beabsichtigte Empfänger sein
oder diese Email irrtümlich erhalten haben, ist es Ihnen nicht gestattet
diese Mail oder einen Teil davon ohne unsere Erlaubnis zu verbreiten, zu
kopieren, unbefugt weiterzuleiten oder zu behalten. Informieren Sie
bitte sofort den Absender telefonisch oder per Email und löschen Sie
diese Email und alle Kopien aus Ihrem System. Wir haften nicht für die
Unversehrtheit von Emails, nachdem sie unseren Einflussbereich verlassen
haben.
Disclaimer
This email may contain confidential and/or privileged information and is
intended solely for the attention and use of the named addressee(s). If
you are not the intended recipient, or a person responsible for
delivering it to the intended recipient, you are not authorized to and
must not disclose, copy, distribute, or retain this message or any part
of it without our authority. Please contact the sender by call or reply
email immediately and destroy all copies and the original message. We
are not responsible for the integrity of emails after they have left our
sphere of control.
//
