RE: Frequent Consumer Rebalance/ Commit fail exception

2016-10-11 Thread Misra, Rahul
Thanks Guozhang for the detailed explanation. It was really helpful. Regards, Rahul Misra -Original Message- From: Guozhang Wang [mailto:wangg...@gmail.com] Sent: Wednesday, October 12, 2016 6:25 AM To: users@kafka.apache.org Subject: Re: Frequent Consumer Rebalance/ Commit fail

Re: Frequent Consumer Rebalance/ Commit fail exception

2016-10-11 Thread Guozhang Wang
Hello Rahul, This "CommitFailedException" usually means that the consumer group coordinator that sits on the server side has decided that this consumer is "failed" from the heartbeat protocol and hence kicked out of the group, and later when it sees a commit-offset request from this consumer it wi

Frequent Consumer Rebalance/ Commit fail exception

2016-10-10 Thread Misra, Rahul
Hi, I have a custom Kafka consumer which reads messages from a topic, hands over the processing of the messages to a different thread, and while the messages are being processed, it pauses the topic and keeps polling the Kafka topic (to maintain heartbeats) and also commits offsets using commi