Hi, thanks for the answer. So when using persistence, i can't rely on producerFlowControl to limit the Flow.
If i have a consumer which is able to process only 10 messages by seconds and my queue is filled with 36000 messages (due to faster producers), i know that it will take 1 hour to process all my messages. I would reject new messages as my processing time shall be less than one hour. I would also keep the order of incoming messages (small and big) so i can't easily manage two queues without adding a complex synchronization mechanism. I see http://activemq.apache.org/slow-consumer-handling.html for non-persistent messages but it should not apply for persistent ones. (Prefetch is configurable for persistence : http://activemq.apache.org/slow-consumer-handling.html#SlowConsumerHandling-UsingthePrefetchPolicytoconfigurethelimit but nothing for Pending Message Limit Strategy). What is the best way to add a mechanism to constrain the queue size (handling slow consumer when using persistence) ? Modifying the cursor, producer flow control, another thing ? Regards Hervé On 2/14/12, Torsten Mielke <tors...@fusesource.com> wrote: > Hello Herve, > > You cannot configure producer flow control based on the various message > sizes. > However you could consider using different queues for these types of msgs > and configure > the limits for each queue differently. That way your small msgs won't be > blocked just because of a small amount of large msgs on the same queue > (assuming different producers for both kinds of msgs). > > Also, when sending persistent queue messages, you rarely see producer flow > control happening when using the default store cursor. That is because the > store cursor is configured to hold messages in memory up to only 70% of the > configured destination limit. If this 70% limit is reached, it won't keep > new msgs in memory but reload them from the store (kahadb) later (when its > time to dispatch the msg). Remember a persistent msg is first written to the > store when received by the broker. > Producer flow control however kicks in when 100% of the available memory for > each destination are used. This limit however you generally don't reach. > Things are different for non-persistent msgs. The default file cursor keeps > msgs in memory until 100% of the configured destination limit are reached. > Thereafter it swaps msgs to temp storage (which is different from kahadb) > and producer flow control would kick in. > > Hope this helps. > > > Torsten Mielke > tors...@fusesource.com > tmie...@blogspot.com > > > On Feb 13, 2012, at 2:37 PM, Hervé BARRAULT wrote: > >> Hi, >> >> i have looked to the documentation of the producer flow control >> functionality (http://activemq.apache.org/producer-flow-control.html). >> >> I see that it is based on the storage size but in my case, i think not >> being able to use the size. >> I have a queue where i put messages two kind of messages : Big 10MB and >> Small 1kB. >> >> As Big messages are rare (compared to small ones), i would limit the >> Memory >> to 100MB (10 messages). >> But for small messages i would limit the number of small messages to 5000 >> (so about 5MB) . We have noted that with JDBC persistence, the number of >> pending messages in the broker (not by queue) is important for >> "performances". >> >> As i would keep the order of Big and Small messages, i put them in the >> same >> queue. >> >> Is there a way to define a strategy for the flow control which is 100MB >> and >> 5k messages ? >> >> >> Regards >> Hervé > > > >
