[ 
https://issues.apache.org/jira/browse/KAFKA-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Quah updated KAFKA-19424:
------------------------------
    Description: 
{{GroupCoordinatorService}} captures the number of partitions of 
{{\_\_consumer_offsets}} at startup. This is used to map group ids to 
{{\_\_consumer_offsets}} partitions in {{GroupCoordinatorService.partitionFor}}.

When adding partitions to {{\_\_consumer_offsets}}, {{GroupCoordinatorService}} 
doesn't update its cached partition count. This means that newly started 
brokers will map group ids to partitions differently to existing brokers and 
FindCoordinator requests can return different results depending on which broker 
the client asks.

It's proposed to make brokers consistent by updating the cached partition count 
in {{GroupMetadataService.onNewMetadataImage}}.

We should additionally log a warning when the partition count changes, since 
adding partitions to {{\_\_consumer_offsets}} is not truly supported, as any 
existing group info will remain on the old partition and become un-findable. 

  was:
{{GroupCoordinatorService}} captures the number of partitions of 
{{__consumer_offsets}} at startup. This is used to map group ids to 
{{__consumer_offsets}} partitions in {{GroupCoordinatorService.partitionFor}}.

When adding partitions to {{__consumer_offsets}}, {{GroupCoordinatorService}} 
doesn't update its cached partition count. This means that newly started 
brokers will map group ids to partitions differently to existing brokers and 
FindCoordinator requests can return different results depending on which broker 
the client asks.

It's proposed to make brokers consistent by updating the cached partition count 
in {{GroupMetadataService.onNewMetadataImage}}.

We should additionally log a warning when the partition count changes, since 
adding partitions to {{__consumer_offsets}} is not truly supported, as any 
existing group info will remain on the old partition and become un-findable. 


> FindCoordinator inconsistent across brokers when __consumer_offsets partition 
> count is increased
> ------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-19424
>                 URL: https://issues.apache.org/jira/browse/KAFKA-19424
>             Project: Kafka
>          Issue Type: Bug
>          Components: group-coordinator
>            Reporter: Sean Quah
>            Priority: Major
>
> {{GroupCoordinatorService}} captures the number of partitions of 
> {{\_\_consumer_offsets}} at startup. This is used to map group ids to 
> {{\_\_consumer_offsets}} partitions in 
> {{GroupCoordinatorService.partitionFor}}.
> When adding partitions to {{\_\_consumer_offsets}}, 
> {{GroupCoordinatorService}} doesn't update its cached partition count. This 
> means that newly started brokers will map group ids to partitions differently 
> to existing brokers and FindCoordinator requests can return different results 
> depending on which broker the client asks.
> It's proposed to make brokers consistent by updating the cached partition 
> count in {{GroupMetadataService.onNewMetadataImage}}.
> We should additionally log a warning when the partition count changes, since 
> adding partitions to {{\_\_consumer_offsets}} is not truly supported, as any 
> existing group info will remain on the old partition and become un-findable. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to