Hey James. The PFC documentation is very clear that async producers will not be notified when PFC kicks in, and that per-producer PFC will not be used. It does not say that whole-connection PFC won't kick in, and I believe that it will, as I outlined in my first response. From the Advantage section of the same page: "Though a client can ignore the producer ACKs altogether [as asynchronous producers do] and the broker should just stall the transport if it has to for slow consumer handling; though this does mean it'll stall the entire connection."
To be very clear about this, PFC is a broker-side activity that refuses to store messages in the store when a limit that applies to that message has been exceeded. As I understand it, it does not inherently prevent producers from sending messages (though per-producer PFC does, and per-connection will eventually, once you fill the TCP buffers and your sends hang, but that can take some time), only prevents the broker from processing those messages. And if PFC is enabled, it is supposed to kick in one way or the other: per-producer if possible, or whole-connection if that's the only option. Your response here says that PFC didn't kick in when you used async sends for your non-batched producer. In light of the above, can you please describe the results you saw? Did you see the number of messages on the destination continue to rise until you hit the storeLimit (same as you described for the batch size > 1), or did you simply see the producer continue to send (which wouldn't indicate whether PFC was occurring)? Did you see any PFC-related messages in the broker's logs? I'm digging deeper on the results of your test because in 5.8.0 I've observed PFC to kick in for async producers without a producer message window when a per-destination limit was exceeded. I've observed the correct operation of the scenario you're saying isn't working for you, so either there's something else wrong in your test, I'm misremembering whether we were using async sends (but I'm pretty sure about that, because it was a conscious decision to do so), or there's some functional difference between 5.8.0 and whatever version you're using. I also want to be clear that I do think the behavior you're seeing with batch size > 1 is a bug (though I'm assuming you've properly set up your broker config file to enforce the limits since you haven't posted that; if there was something wrong there then this might not be a bug, but I'm assuming that you've got that right), and my questions here are to try to tease out more specifics of what the cause is (and if we're lucky, help you find a workaround prior to a fix being made). Also, you never said what version of ActiveMQ you're using; that's always relevant information in any question to the list. Tim On Aug 19, 2015 6:49 PM, "jamesTheCruncher" <ja...@mcruncher.com> wrote: > Hi Tim, > > The PFC documentation is very clear that PFC won't kick in for async sends > unless /Producer Window Size/ is configured. > > / > Producers that use Async Sends - generally speaking, producers of > non-persistent messages - don't bother waiting for any acknowledgement from > the broker; so, if a memory limit has been exceeded, you will not get > notfied. If you do want to be aware of broker limits being exceeded, you > will need to configure the ProducerWindowSize connection option so that > even > async messages are flow controlled per producer. > / > > I have tested this with a producer that sends messages asynchronously (but > doesn't use batching) and I can confirm that PFC kicks in only when > /Producer Window Size/ is configured. > > But I haven't seen any documentation related to PFC and Batch Send. > > > > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/What-is-the-correct-way-to-throttle-ActiveMQ-producers-who-send-persistent-messages-in-batches-to-a--tp4701204p4701226.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. >
