Hi Phil, I tested my code and it is correct. I do see heartbeats getting missed sometimes and causing session time out for consumer where generation is marked dead. I see that there are long time windows where there is no heartbeat whereas i do commit in between these time windows and there is no reason to skip a heartbeat as the last hearbeat was way before configured heartbeat interval. I ran tests with kafka client jar build from trunk (0.10.1.0) and it works fine due to latest changes where each commitSync acts as a heartbeat.
Calling commitSync after each record processing works fine because it triggers more number of heartbeat requests in a timeinterval making chances extremely small where heartbeats can be missed for more than session time out. You wrote in a previous mail that "I thought having commitSync throw an explicit 'rebalance in progress' type > exception rather than just a KafkaException would allow this to be easily > identified and handled." I noticed that in case where a consumer is marked dead or a rebalance is in progress, kafka throws CommitFailedException. A KafkaException is thrown only when something unknown has happened which is not yet categorized. Regards, Vinay On Wed, Apr 27, 2016 at 7:52 AM, vinay sharma <vinsharma.t...@gmail.com> wrote: > Hi Phil, > > This sounds great. Thanks for trying these serrings. This means probably > something wrong in my code or setup. I will check what is causing this > issue in my case. > > I have a 3 broker 1 zk cluster and my topic has 3 partitions with > replication factor 3. > > Regards, > Vinay Sharma >
