Do it the other way around. The system and store settings are global, the others per destination. So what you are doing doesn't make a ton of sense.
On Jun 5, 2013, at 9:24 AM, Jesus Roncero <je...@we7.com> wrote: > Hi all, > > I've got a testing activeMQ installation that I'm trying to test regarding > different limits on destinations and memory usage in general, with and without > flowControl enable. I'm running into some issues and I don't quite understand > how this works, so I would appreciate any tips or advise here. > > My setup is activeMQ 5.6 in a testing environment. I'm using a couple of > python scripts that use stomp.py to send some messages and read them from a > testqueue. Depending on a couple of settings, the behaviour changes, and it > behaves differently. > > I'm sendind 1 Mb messages to the queue as per: > > data = "x" * 1024 * 1024 > number = 1000 > print "producing %s %s kb messages" % (number, len(data) / 1024) > for i in range(number): > print i > conn.send(data, destination='/queue/testqueue') > > in python. > > Now, I have two settings: > > Scenario number 1 > ---------------- > > <policyEntries> > <policyEntry queue=">" memoryLimit="50mb" producerFlowControl="true"> > <pendingQueuePolicy> > <fileQueueCursor /> > </pendingQueuePolicy> > </policyEntry> > <policyEntry topic=">" memoryLimit="50mb" producerFlowControl="true"> > <!-- We want to store both persistent and non-persistent messages --> > <pendingSubscriberPolicy> > <fileCursor /> > </pendingSubscriberPolicy> > <pendingDurableSubscriberPolicy> > <fileDurableSubscriberCursor/> > </pendingDurableSubscriberPolicy> > > ... > > <systemUsage> > <systemUsage> > <memoryUsage> > <memoryUsage limit="40 mb"/> > </memoryUsage> > <storeUsage> > <storeUsage limit="1 gb" /> > </storeUsage> > <tempUsage> > <tempUsage limit="800 mb"/> > </tempUsage> > </systemUsage> > </systemUsage> > > Where the memoryLimit of each of the queues is bigger than the limit for > memoryUsage. When I try to insert into the queue, it insert 32 messages (32Mb) > and it hangs in there. I see this log messages: > > [2013-06-05 15:54:19,816] INFO ActiveMQ Transport: tcp:///10.0.9.124:36431 > org.apache.activemq.broker.region.Queue - Usage Manager Memory Limit > (52428800) reached on queue://testqueue. Producers will be throttled to the > rate at which messages are removed from this destination to prevent flooding > it. See http://activemq.apache.org/producer-flow-control.html for more info > 2013-06-05 15:54:19,816 [0.0.9.124:36431] INFO Queue - Usage Manager Memory > Limit (52428800) reached on queue://testqueue. Producers will be throttled to > the rate at which messages are removed from this destination to prevent > flooding it. See http://activemq.apache.org/producer-flow-control.html for > more info > [2013-06-05 15:54:20,818] INFO ActiveMQ Transport: tcp:///10.0.9.124:36431 > org.apache.activemq.broker.region.Queue - > Usage(default:memory:queue://testqueue:memory) percentUsage=62%, > usage=32537724, limit=52428800, > percentUsageMinDelta=1%;Parent:Usage(default:memory) percentUsage=101%, > usage=42443902, limit=41943040, percentUsageMinDelta=1%: Usage Manager Memory > Limit reached. Producer (ID:meme-49473-1370444050426-2:1:-1:1) stopped to > prevent flooding queue://testqueue. See > http://activemq.apache.org/producer-flow-control.html for more info (blocking > for: 1s) > 2013-06-05 15:54:20,818 [0.0.9.124:36431] INFO Queue - > Usage(default:memory:queue://testqueue:memory) percentUsage=62%, > usage=32537724, limit=52428800, > percentUsageMinDelta=1%;Parent:Usage(default:memory) percentUsage=101%, > usage=42443902, limit=41943040, percentUsageMinDelta=1%: Usage Manager Memory > Limit reached. Producer (ID:meme-49473-1370444050426-2:1:-1:1) stopped to > prevent flooding queue://testqueue. See > http://activemq.apache.org/producer-flow-control.html for more info (blocking > for: 1s) > > Which would indicate that flow control kicked in as per the first message, > that they reach a memory limit (50Mb) as per the per destination limit (second > message). It also says that the memory (40Mb) got used as a 100%. > > Also, noted that it says: default:memory:queue. > > > Scenario number 2 > ----------------- > > Now, if I change the memoryUsage to be slightly bigger than the memoryLimit on > each queue, from 40 to 60Mb, as per: > > <systemUsage> > <systemUsage> > <memoryUsage> > <memoryUsage limit="60 mb"/> > </memoryUsage> > <storeUsage> > <storeUsage limit="1 gb" /> > </storeUsage> > <tempUsage> > <tempUsage limit="800 mb"/> > </tempUsage> > </systemUsage> > </systemUsage> > > taking into account that the per-destination limits are just the same, 50Mb as > per the config above. When I run the same exact procedure, it inserts 801 1Mb > messages and it stops with this message: > > [2013-06-05 16:05:01,972] INFO BrokerService.worker.1 > org.apache.activemq.store.kahadb.plist.PListStore - > PListStore:[/var/lib/activemq/we7/data/localhost/tmp_storage ] initialized > 2013-06-05 16:05:01,972 [ervice.worker.1] INFO PListStore - > PListStore:[/var/lib/activemq/we7/data/localhost/tmp_storage ] initialized > [2013-06-05 16:05:12,959] INFO ActiveMQ Transport: tcp:///10.0.9.124:36615 > org.apache.activemq.broker.region.Queue - > Usage(default:temp:queue://testqueue:temp) percentUsage=99%, usage=839122944, > limit=838860800, percentUsageMinDelta=1%;Parent:Usage(default:temp) > percentUsage=100%, usage=839122944, limit=838860800, percentUsageMinDelta=1%: > Temp Store is Full (99% of 838860800). Stopping producer > (ID:meme-55481-1370444692514-2:1:-1:1) to prevent flooding queue://testqueue. > See http://activemq.apache.org/producer-flow-control.html for more info > (blocking for: 1s) > 2013-06-05 16:05:12,959 [0.0.9.124:36615] INFO Queue - > Usage(default:temp:queue://testqueue:temp) percentUsage=99%, usage=839122944, > limit=838860800, percentUsageMinDelta=1%;Parent:Usage(default:temp) > percentUsage=100%, usage=839122944, limit=838860800, percentUsageMinDelta=1%: > Temp Store is Full (99% of 838860800). Stopping producer > (ID:meme-55481-1370444692514-2:1:-1:1) to prevent flooding queue://testqueue. > See http://activemq.apache.org/producer-flow-control.html for more info > (blocking for: 1s) > > Where I see that it starts using the tempUsage space, and it pauses when it's > used the 800 Mb available, whining that the temp store is full. > > Now, the type is default:temp:queue rather than memory > > So, > > Could anyone explain to me why this is? I've read quite a few blog posts and > entries on the mailing list and I'm still a bit confused about it, so, to have > more concrete questions, > > 1) Why this change on the memoryUsage limit affects how activeMQ deals with > way too many messages? More so, why on the second example, the 50 Mb limit > doesn't kick in? > > 2) I uderstood that activeMQ passes messages to the temp storage area when a > queue is full 70% (unless configured otherwise), but this is not happening > here. Why? > > 3) I've seen a case where each queue is configured with a 500Mb limit, > memoryUsage limit is 20Mb and tempUsage is 100Mb, activeMQ accepts 350 Mb > worth of messages (70% of 500 Mb) and then it writes it the temporary storage, > and then it hangs, complaining temp storage is 350% of the maximum value. > Then, there's no way to recover activeMQ unless with a restart. > > > Any ideas/suggestions/pointers would be appreciated, as I'm getting a bit > frustrated trying to understand this :-) > > Apologies for the length of the email, hope is easily understandable. > > Thanks. > Regards. > > -- > JRF
