[ https://issues.apache.org/jira/browse/KAFKA-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Quah updated KAFKA-19424: ------------------------------ Description: {{GroupCoordinatorService}} captures the number of partitions of {{\_\_consumer_offsets}} at startup. This is used to map group ids to {{\_\_consumer_offsets}} partitions in {{GroupCoordinatorService.partitionFor}}. When adding partitions to {{\_\_consumer_offsets}}, {{GroupCoordinatorService}} doesn't update its cached partition count. This means that newly started brokers will map group ids to partitions differently to existing brokers and FindCoordinator requests can return different results depending on which broker the client asks. It's proposed to make brokers consistent by updating the cached partition count in {{GroupMetadataService.onNewMetadataImage}}. We should additionally log a warning when the partition count changes, since adding partitions to {{\_\_consumer_offsets}} is not truly supported, as any existing group info will remain on the old partition and become un-findable. was: {{GroupCoordinatorService}} captures the number of partitions of {{__consumer_offsets}} at startup. This is used to map group ids to {{__consumer_offsets}} partitions in {{GroupCoordinatorService.partitionFor}}. When adding partitions to {{__consumer_offsets}}, {{GroupCoordinatorService}} doesn't update its cached partition count. This means that newly started brokers will map group ids to partitions differently to existing brokers and FindCoordinator requests can return different results depending on which broker the client asks. It's proposed to make brokers consistent by updating the cached partition count in {{GroupMetadataService.onNewMetadataImage}}. We should additionally log a warning when the partition count changes, since adding partitions to {{__consumer_offsets}} is not truly supported, as any existing group info will remain on the old partition and become un-findable. > FindCoordinator inconsistent across brokers when __consumer_offsets partition > count is increased > ------------------------------------------------------------------------------------------------ > > Key: KAFKA-19424 > URL: https://issues.apache.org/jira/browse/KAFKA-19424 > Project: Kafka > Issue Type: Bug > Components: group-coordinator > Reporter: Sean Quah > Priority: Major > > {{GroupCoordinatorService}} captures the number of partitions of > {{\_\_consumer_offsets}} at startup. This is used to map group ids to > {{\_\_consumer_offsets}} partitions in > {{GroupCoordinatorService.partitionFor}}. > When adding partitions to {{\_\_consumer_offsets}}, > {{GroupCoordinatorService}} doesn't update its cached partition count. This > means that newly started brokers will map group ids to partitions differently > to existing brokers and FindCoordinator requests can return different results > depending on which broker the client asks. > It's proposed to make brokers consistent by updating the cached partition > count in {{GroupMetadataService.onNewMetadataImage}}. > We should additionally log a warning when the partition count changes, since > adding partitions to {{\_\_consumer_offsets}} is not truly supported, as any > existing group info will remain on the old partition and become un-findable. -- This message was sent by Atlassian Jira (v8.20.10#820010)