Hi Luke, I did end up with a situation where I had two instances connecting to the same consumer group and they ended up in a rebalance trade-off. All partitions kept going back and forth between the two microservice instances. That was a test case where I'd removed the Group Instance Id setting to see what would happen. I stabilized that one by reducing it to a single consumer after 20+ rebalances.
The other issue I'm seeing may be a bug in the Functional Scala `fs2-kafka` wrapper where I see the partitions cleanly assigned but one or more instances isn't ingesting. I found out that they recently added support for the cooperative sticky assignor for the stream recreation since they were assuming a full revocation of the partitions. So I basically wanted to make sure I wasn't missing any settings beyond the partition assignment strategy and the group instance id. -Richard -Richard On Tue, Mar 15, 2022 at 11:27 PM Luke Chen <show...@gmail.com> wrote: > Hi Richard, > > To use `CooperativeStickyAssignor`, no other special configuration is > required. > > I'm not sure what does `make the rebalance happen cleanly` mean. > Did you find any problem during group rebalance? > > Thank you. > Luke > > On Wed, Mar 16, 2022 at 1:00 PM Richard Ney <richard....@lookout.com > .invalid> > wrote: > > > Trying to find a good sample of what consumer settings besides setting > > > > ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG to > > org.apache.kafka.clients.consumer.CooperativeStickyAssignor > > > > is needed to make the rebalance happen cleanly. Unable to find and decent > > documentation or code samples. I have set the Group Instance Id to the > EC2 > > instance id based on one blog write up I found. > > > > Any help would be appreciated > > > > -Richard > > >
