Hi, If you choose not to close consumer, but only poll as a background thread it, the rebalance wouldn’t take place. This is only if you are sending heartbeats (via Poll() ) within the timeout period. Also, if one of the pod dies, rebalance has to happen. You’d want a fare share of work.
An example code is also provided on KafkaConsumer docs. You may want to check that. The other option is to use manual partition assignment, but that means you need to manually assign and manage who gets the partition. Thanks, On Wed, 9 Oct 2019 at 20:21, Gopinath <gop...@gmail.com> wrote: > Hi, > > I have 2 partition in a topic and 3 pods/instances of my microservice > running in my k8s cluster. I wanted all 3 pods to pull message from 2 > partitions. (I'm using same group id for all 3 pods/instances). > > I have achieved this by closing consumer as soon as i pulled message from > partition. kafkaconsumer.close(); in this way i see i could see 3 pods can > able to pull message from this 2 partition. > > But what i observed is, when ever i close suing kafkaconsumer.close(); i > see rebalancing is happening in partition. I do not want rebalancing to > happen as another pod is ready to pick message from this partition. > > Can you please suggest a way to stop rebalancing. >
