[
https://issues.apache.org/jira/browse/KAFKA-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13750826#comment-13750826
]
Guozhang Wang commented on KAFKA-1020:
--------------------------------------
Proposed Approach:
1. Currently the partitionReplicaAssignment cache has two point of
inconsistency with the ZK data: 1) in handling replica state change to
NonExistentReplica we remove the replica in partitionReplicaAssignment before
we update ZK; 2) in handling replica state change from New to Online we add the
replica to partitionReplicaAssignment before we update ZK. Since
partitionReplicaAssignment will be updated later in ZK, we can safely remove
these two updates.
2. Then we can safely change the getAllReplicasOnBroker to reading from
partitionReplicaAssignment instead of from ZK.
3. Some minor issues such as comments will also be fixed in this JIRA.
> Remove getAllReplicasOnBroker from KafkaController
> --------------------------------------------------
>
> Key: KAFKA-1020
> URL: https://issues.apache.org/jira/browse/KAFKA-1020
> Project: Kafka
> Issue Type: Bug
> Reporter: Guozhang Wang
> Assignee: Guozhang Wang
>
> Today KafkaController call getAllReplicasOnBroker on broker failure and new
> broker start up to get all the replicas that broker is holding (or suppose to
> hold). This function actually issue a read on each topic's partition znodes.
> With large number of topic/partitions this could seriously increase the
> latency of handling broker failure and new broker startup.
> On the other hand, ControllerContext maintains a partitionReplicaAssignment
> cache, which is designed to keep the most updated partition replica
> assignment according to ZK. So instead of reading from ZK, we could just read
> from the local cache, given that partitionReplicaAssignment is guaranteed to
> be up-to-date.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
