Hi,
I did not check above values for "PurgatorySize". Here is the output of
free -m when test was running.
total used free shared buffers cached
Mem: 48356 48217 139 0 419 44584
-/+ buffers/cache: 3212 45144
Swap: 16383 4 16379
..LCassa
On Mon, Mar 24, 2014 at 8:38 PM, Jun Rao <jun...@gmail.com> wrote:
> Are the following jmx values high on the brokers?
>
> "kafka.server":name="PurgatorySize",type="ProducerRequestPurgatory"
> "kafka.server":name="PurgatorySize",type="FetchRequestPurgatory"
>
> Thanks,
>
> Jun
>
> On Mon, Mar 24, 2014 at 12:38 PM, Cassa L <lcas...@gmail.com> wrote:
>
> > Hi,
> > We have been doing some evaluation testing against Kafka.We have 48GB
> RAM
> > on each broker. I created 3 broker cluster and one zookeeper and sent
> > 10,000Messages/seconds to this cluster, continuously. Payload is very
> > small, less than Kilobyte. Consumer was readind 5,000Messages per second.
> >
> > What I noticed that, after couple of hours, entire memory was being
> used.
> > There was no free memory available on those machines. I believe Kafka
> does
> > use Off-Heap memory, how do reduce this consumption? How can I avoid
> > OutOfMemory problem if this happens ?
> > Are there any setting I need to do? I was playing with some configuration
> > parameters and here are they -
> > log.flush.interval.messages=100000 (default
> > 10000)
> > log.flush.interval.ms=60000
> > (default 1000)
> > log.default.flush.scheduler.interval.ms=10000 (default
> 3000)
> > log.cleanup.interval.mins=120
> (default
> > 10)
> >
> > Thanks,
> > LCassa
> >
>
