Hello Prabhjot, Actually what I meant in previous email is that there are 200 topics, with 1 partition each so there are 200 total partitions. Is there any rule of thumb regarding this matter? Can you share your configuration (including spec, jvm memory, etc) for kafka cluster?
Thank you, On Thu, Nov 5, 2015 at 4:07 PM, Prabhjot Bharaj <prabhbha...@gmail.com> wrote: > Hi, > > Not sure. But, I had hit OOM when using too many partitions and many topics > with a lesser heap size assigned to the jvm > > Regards, > Prabhjot > > On Thu, Nov 5, 2015 at 10:40 AM, Muqtafi Akhmad <muqt...@traveloka.com> > wrote: > > > Hello all, > > Recently I got incident with kafka cluster, I found OutOfMemoryError in > > kafka server log > > > > > WARN [ReplicaFetcherThread-0-2], Error in fetch Name: FetchRequest; > > > Version: 0; CorrelationId: 7041716; ClientId: ReplicaFetcherThread-0-2; > > > ReplicaId: 0; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo:[. . . . > > .]. > > > Possible cause: java.lang.OutOfMemoryError: Java heap space > > > (kafka.server.ReplicaFetcherThread) > > > > And at the same time there are socket timeout exceptions in kafka > consumer. > > Is there any hypothesis about what might be wrong? > > > > Here is the cluster spec : > > 3 hosts, @ with memory 16 G > > partitions : ~200 > > replication factor : 3 > > rate in : 300 K/s > > rate out : 800 K/s > > > > Thank you, > > > > -- > > Muqtafi Akhmad > > Software Engineer > > Traveloka > > > > > > -- > --------------------------------------------------------- > "There are only 10 types of people in the world: Those who understand > binary, and those who don't" > -- Muqtafi Akhmad Software Engineer Traveloka
