[
https://issues.apache.org/jira/browse/KAFKA-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14552582#comment-14552582
]
Guozhang Wang edited comment on KAFKA-2017 at 5/20/15 4:18 PM:
---------------------------------------------------------------
This patch used the second ZK data structure proposed above to reduce #. ZK
reads.
1. The consumer group registry path only contains the members list, not the
assignment, which can always be calculated on-the-fly from the member
registries.
2. The group registry will be updated every time when the prepare-rebalance
phase has completed, and hence the new members has finalized; if the group has
no members, it will be removed as well.
3. When a consumer group was first created, it is not immediately registered in
ZK but only kept in coordinator's metadata cache; only when the new group has
finished the first prepare-rebalance it will then be registered in ZK.
4. When a consumer group cannot be found in the metadata cache, the coordinator
will always try to read it form ZK.
Pinging [~onurkaraman], [~junrao] for reviews.
was (Author: guozhang):
This patch used the second ZK data structure proposed above to reduce #. ZK
reads.
1. The consumer group registry path only contains the members list, not the
assignment, which can always be calculated on-the-fly from the member
registries.
2. The group registry will be updated every time when the prepare-rebalance
phase has completed, and hence the new members has finalized; if the group has
no members, it will be removed as well.
3. When a consumer group was first created, it is not immediately registered in
ZK but only kept in coordinator's metadata cache; only when the new group has
finished the first prepare-rebalance it will then be registered in ZK.
4. When a consumer group cannot be found in the metadata cache, the coordinator
will always try to read it form ZK.
> Persist Coordinator State for Coordinator Failover
> --------------------------------------------------
>
> Key: KAFKA-2017
> URL: https://issues.apache.org/jira/browse/KAFKA-2017
> Project: Kafka
> Issue Type: Sub-task
> Components: consumer
> Affects Versions: 0.9.0
> Reporter: Onur Karaman
> Assignee: Guozhang Wang
> Attachments: KAFKA-2017.patch, KAFKA-2017_2015-05-20_09:13:39.patch
>
>
> When a coordinator fails, the group membership protocol tries to failover to
> a new coordinator without forcing all the consumers rejoin their groups. This
> is possible if the coordinator persists its state so that the state can be
> transferred during coordinator failover. This state consists of most of the
> information in GroupRegistry and ConsumerRegistry.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
