Re: Consumer Rebalancing Issues

2020-03-21 Thread Ravi Kanth
Lukasz, Thanks for the reply. I set max.poll.interval.ms to 1 hour and my consumer is polling with no issues every 500 ms and still I'm seeing the problem. I'm not worried about consumer dying and rebalancing instead after the rebalance it forgets it's previous state. Thanks On Fri, Mar 20, 2020

Re: Consumer Rebalancing Issues

2020-03-20 Thread Łukasz Antoniak
Ravi, It is not a bug. Broker assumes that your consumer faced live-lock. You need to tune property max.poll.interval.ms to increase expected interval between poll() on consumer side. -- Lukasz sob., 21 mar 2020 o 02:19 Ravi Kanth napisał(a): > Hi All, > > I have a Kafka Consumer that polls th

Consumer Rebalancing Issues

2020-03-20 Thread Ravi Kanth
Hi All, I have a Kafka Consumer that polls the data and gets *paused* for 15-20 mins for the post-processing of the polled records. However, during the time of pause, the broker assumes that the consumer group is dead(check below log) and rebalances the consumer group. *Application Log:* k8s-work