Hi All, I have a Kafka Consumer that polls the data and gets *paused* for 15-20 mins for the post-processing of the polled records. However, during the time of pause, the broker assumes that the consumer group is dead(check below log) and rebalances the consumer group.
*Application Log:* k8s-worker403: 2020-03-20 21:41:14.147 [kafka-coordinator-heartbeat-thread | <consumer_group>] INFO o.a.k.c.c.i.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=<consumer_group>] Marking the coordinator <broker>:9093 (id: 2147483480 rack: null) dead *Broker Log:* Member consumer-1-4eb939a6-c37d-4853-96dc-4255ec8663ff in group <consumer group> has failed, removing it from the group When rebalancing happens, consumer is automatically *resumed* and starts polling for more records though I am expecting it to be paused. This is causing unexpected behavior. When reading through Kafka docs, it looks like tuning up request.timeout.ms/session.timeout.ms will help avoid running into this problem. Please correct if my understanding is incorrect. However, I am not willing to tweak the configurations, instead handle the problem at the code level. *Question:* 1. Is this a bug in Kafka Java API letting the consumer be resumed after rebalancing even when it is initially paused? If so, details of the bug would be helpful. 2. Is there anyother way to avoid running into this problem without tweaking configurations? *Kafka Version Using:* *1.0.1* Any suggestions would be of great help. Thanks, Ravi
