[
https://issues.apache.org/jira/browse/KAFKA-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591976#comment-14591976
]
Manikumar Reddy edited comment on KAFKA-2265 at 6/18/15 3:36 PM:
-----------------------------------------------------------------
For each partition, we are parsing /brokers/topic/ zk node for latest
partition replica assignment data. This will take time when we have large
number of partitions. In TopicChangeListener , we are adding latest partition
replica assignment to controllerContext.partitionReplicaAssignment. So we no
need to read this partition info again. Similar code added to
AddPartitionsListener. Removed assignReplicasToPartitions method and its usage.
was (Author: omkreddy):
For each partition, we are parsing /brokers/topic/ zk node for latest
partition assignment data. This will take time when we have large number of
partitions. In TopicChangeListener , we are adding latest partition Assignment
to controllerContext.partitionReplicaAssignment. So we no need to read this
partition info again. Similar code added to AddPartitionsListener. Removed
assignReplicasToPartitions method and its usage.
> creating a topic with large number of partitions takes a long time
> ------------------------------------------------------------------
>
> Key: KAFKA-2265
> URL: https://issues.apache.org/jira/browse/KAFKA-2265
> Project: Kafka
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.8.2.1
> Reporter: Jun Rao
> Assignee: Manikumar Reddy
> Fix For: 0.8.3
>
> Attachments: KAFKA-2265.patch
>
>
> Currently, creating a topic with 3K partitions can take 15 mins. We should be
> able to do that much faster. There is perhaps some redundant accesses to ZK
> during topic creation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
