The pattern you apply seems to be an anti-pattern. If you have only 2 partitions, only 2 consumer instances within a group can consumer data -- hence, you should either only use 2 pods, or you should increase the number of partitions.
What you try to do atm, won't work smoothly no matter what you do. -Matthias On 10/9/19 11:05 AM, Gopinath wrote: > Hi, > > I have 2 partition in a topic and 3 pods/instances of my microservice > running in my k8s cluster. I wanted all 3 pods to pull message from 2 > partitions. (I'm using same group id for all 3 pods/instances). > > I have achieved this by closing consumer as soon as i pulled message from > partition. kafkaconsumer.close(); in this way i see i could see 3 pods can > able to pull message from this 2 partition. > > But what i observed is, when ever i close suing kafkaconsumer.close(); i > see rebalancing is happening in partition. I do not want rebalancing to > happen as another pod is ready to pick message from this partition. > > Can you please suggest a way to stop rebalancing. >
signature.asc
Description: OpenPGP digital signature
