Priya, if you want you can look at RequestPurgatory.scala for some more details.
The config is the size of the atomic requestCounter. Basically the purge in the purgatory is a way to check if the request has been satisfied and delayed and can get removed. It is a background scan when the size reaches a certain point to get it back low again if it can force it. Kind of like a minor compaction how I see it so it does not grow without bound. /******************************************* Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> ********************************************/ On Fri, Nov 1, 2013 at 2:45 AM, Priya Matpadi <priya.matp...@ecofactor.com>wrote: > Guozhang, > The documentation is not very clear. > Marc's response for producer purgatory makes sense. > I am not entirely clear on fetch purgatory. > How does broker use purgatory? Is it a temporary holding area? What happens > to the messages if purge interval is exceeded in case of either/both > producer and consumer? Are messages dropped in this case? > Thanks, > Priya > > > On Thu, Oct 31, 2013 at 2:47 PM, Guozhang Wang <wangg...@gmail.com> wrote: > > > Hello Priya, > > > > You can find the definitions of these two configs here: > > > > http://kafka.apache.org/documentation.html#brokerconfigs > > > > Guozhang > > > > > > On Thu, Oct 31, 2013 at 11:20 AM, Marc Labbe <mrla...@gmail.com> wrote: > > > > > Hi Priya > > > > > > my understanding is producer requests will be delayed (and put in > request > > > purgatory) only if your producer uses ack=-1. It will be in the > purgatory > > > (delayed) until all brokers have acknowledged the messages to be > > > replicated. The documentation suggests to monitor the > > > ProducerRequestPurgatory size metrics , but it only applies if you're > > using > > > ack=-1, otherwise, this value will always be 0. > > > > > > For consumer requests, they'll be in purgatory (delayed) until the max > > > allowed time to respond has been reached, unless it has enough messages > > to > > > fill the buffer before that. The request will not end up in the > purgatory > > > if you're making a blocking request (max wait <= 0). > > > > > > Not sure about the configuration interval though. > > > > > > marc > > > > > > > > > On Thu, Oct 31, 2013 at 12:41 PM, Priya Matpadi < > > > priya.matp...@ecofactor.com > > > > wrote: > > > > > > > Hello, > > > > What is purgatory? I believe the following two properties relate to > > > > consumer and producer respectively. > > > > Could someone please explain the significance of these? > > > > fetch.purgatory.purge.interval.requests=100 > > > > producer.purgatory.purge.interval.requests=100 > > > > > > > > Thanks, > > > > Priya > > > > > > > > > > > > > > > -- > > -- Guozhang > > >
