Re: Imbalance in Commit Messages Across __consumer_offsets Topic Partitions

2023-05-09 Thread Alexandre Dupriez
Hi Fares, What is the rate of offset commits for the group? How often do you need to commit offsets for consumers in this group? Thanks, Alexandre Le mar. 9 mai 2023 à 18:34, Fares Oueslati a écrit : > > Hello Richard, > > Thank you for your answer. > > Upon examining the `__consumer_offsets` t

Re: Imbalance in Commit Messages Across __consumer_offsets Topic Partitions

2023-05-09 Thread Fares Oueslati
Hello Richard, Thank you for your answer. Upon examining the `__consumer_offsets` topic, it seems that all commit messages for a given consumer `group.id` go to the same partition. So, there's nothing much to do if we have a dominant consumer group reading from all topics. The only solution woul

Re: Imbalance in Commit Messages Across __consumer_offsets Topic Partitions

2023-05-01 Thread Richard Bosch
Hi Fares, You're right in your description of the contents of the __consumer_offsets topic, and how they are stored. The most common reason for an uneven load on the consumer offsets are. 1. Configuration of offset commits in the client 2. Load on topic being consumed If a topic has 10 partition