Hi, You could use the vmQueueCursor as well. In both cases (assuming producerFlowControl=true) the cursors will only dispatch msg from memory and not from the store.
Regards, Torsten Mielke On Feb 15, 2012, at 3:48 PM, Hervé BARRAULT wrote: > Hi, > If i set cursorMemoryHighWaterMark to 100, is it a good idea to use > vmCursor instead of File Based Cursor (the default one) ? > > Regards > Hervé > > On Wed, Feb 15, 2012 at 10:50 AM, Torsten Mielke > <tors...@fusesource.com>wrote: > >> Hello, >> >>> So when using persistence, i can't rely on producerFlowControl to >>> limit the Flow. >> >> >> Yes you can. However when using persistent queue messages, >> producerFlowControl does not kick-in that early. Which is actually a good >> thing in general. >> >> All this is configurable as well. You can specify >> cursorMemoryHighWaterMark="100" on your destination policy configuration. >> With that the cursor will take up to 100% of the configured queue memory >> limit. When that limit is reached, producer flow control will also kick in >> (as now 100% of the memory are in use). >> >> >>> What is the best way to add a mechanism to constrain the queue size >>> (handling slow consumer when using persistence) ? >> >> ProducerFlowControl and perhaps in your case also setting >> cursorMemoryHighWaterMark="100" on the destination policy. >> >> >> Best Regards, >> >> Torsten Mielke >> tors...@fusesource.com >> tmie...@blogspot.com >> >> >> On Feb 14, 2012, at 2:20 PM, Hervé BARRAULT wrote: >> >>> Hi, >>> thanks for the answer. >>> >>> So when using persistence, i can't rely on producerFlowControl to >>> limit the Flow. >>> >>> If i have a consumer which is able to process only 10 messages by >>> seconds and my queue is filled with 36000 messages (due to faster >>> producers), i know that it will take 1 hour to process all my >>> messages. I would reject new messages as my processing time shall be >>> less than one hour. >>> >>> I would also keep the order of incoming messages (small and big) so i >>> can't easily manage two queues without adding a complex >>> synchronization mechanism. >>> >>> I see http://activemq.apache.org/slow-consumer-handling.html for >>> non-persistent messages but it should not apply for persistent ones. >>> (Prefetch is configurable for persistence : >>> >> http://activemq.apache.org/slow-consumer-handling.html#SlowConsumerHandling-UsingthePrefetchPolicytoconfigurethelimit >>> but nothing for Pending Message Limit Strategy). >>> >>> What is the best way to add a mechanism to constrain the queue size >>> (handling slow consumer when using persistence) ? >>> >>> Modifying the cursor, producer flow control, another thing ? >>> >>> Regards >>> >>> Hervé >>> >>> >>> On 2/14/12, Torsten Mielke <tors...@fusesource.com> wrote: >>>> Hello Herve, >>>> >>>> You cannot configure producer flow control based on the various message >>>> sizes. >>>> However you could consider using different queues for these types of >> msgs >>>> and configure >>>> the limits for each queue differently. That way your small msgs won't be >>>> blocked just because of a small amount of large msgs on the same queue >>>> (assuming different producers for both kinds of msgs). >>>> >>>> Also, when sending persistent queue messages, you rarely see producer >> flow >>>> control happening when using the default store cursor. That is because >> the >>>> store cursor is configured to hold messages in memory up to only 70% of >> the >>>> configured destination limit. If this 70% limit is reached, it won't >> keep >>>> new msgs in memory but reload them from the store (kahadb) later (when >> its >>>> time to dispatch the msg). Remember a persistent msg is first written >> to the >>>> store when received by the broker. >>>> Producer flow control however kicks in when 100% of the available >> memory for >>>> each destination are used. This limit however you generally don't reach. >>>> Things are different for non-persistent msgs. The default file cursor >> keeps >>>> msgs in memory until 100% of the configured destination limit are >> reached. >>>> Thereafter it swaps msgs to temp storage (which is different from >> kahadb) >>>> and producer flow control would kick in. >>>> >>>> Hope this helps. >>>> >>>> >>>> Torsten Mielke >>>> tors...@fusesource.com >>>> tmie...@blogspot.com >>>> >>>> >>>> On Feb 13, 2012, at 2:37 PM, Hervé BARRAULT wrote: >>>> >>>>> Hi, >>>>> >>>>> i have looked to the documentation of the producer flow control >>>>> functionality (http://activemq.apache.org/producer-flow-control.html). >>>>> >>>>> I see that it is based on the storage size but in my case, i think not >>>>> being able to use the size. >>>>> I have a queue where i put messages two kind of messages : Big 10MB and >>>>> Small 1kB. >>>>> >>>>> As Big messages are rare (compared to small ones), i would limit the >>>>> Memory >>>>> to 100MB (10 messages). >>>>> But for small messages i would limit the number of small messages to >> 5000 >>>>> (so about 5MB) . We have noted that with JDBC persistence, the number >> of >>>>> pending messages in the broker (not by queue) is important for >>>>> "performances". >>>>> >>>>> As i would keep the order of Big and Small messages, i put them in the >>>>> same >>>>> queue. >>>>> >>>>> Is there a way to define a strategy for the flow control which is 100MB >>>>> and >>>>> 5k messages ? >>>>> >>>>> >>>>> Regards >>>>> Hervé
