Re: Heartbeat failed w/ multi-group/consumers

2017-05-17 Thread Guozhang Wang
Did you have any heavy logic in the rebalance callback? There are some observed issues that have been fixed in later versions such as: https://cwiki.apache.org/confluence/display/KAFKA/KIP-62%3A+Allow+consumer+to+send+heartbeats+from+a+background+thread (do not falls out of the rebalance during r

Re: Heartbeat failed w/ multi-group/consumers

2017-05-17 Thread Abhimanyu Nagrath
I am also facing the same issue . Regards, Abhimanyu On Thu, May 18, 2017 at 5:29 AM, Cédric Chantepie < c.chante...@yahoo.fr.invalid> wrote: > Hi, > > I have a test app using Java lib for consumers with Kafka 0.10, using > Kafka storage for offset. > > This app is managing 190 consumers, accro

Heartbeat failed w/ multi-group/consumers

2017-05-17 Thread Cédric Chantepie
Hi, I have a test app using Java lib for consumers with Kafka 0.10, using Kafka storage for offset. This app is managing 190 consumers, accross 19 different consumer group, against 12 distinct topics (details bellow). When one app instance is starting, with 40 partitions per topic, it takes ~1