----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14041/#review26022 -----------------------------------------------------------
Following up on the performance concerns that Neha had raised - this will be a significant bottleneck for tools such as the mirror-maker as the rebalance latency will almost certainly multiply. We could consider utilizing the returned children information in ZkRebalancerListener's handleChildChange - i.e., curChilds will be the partitions if the parentPath is a topic. It won't be a trivial change then - we currently recompute the full map of topic/partitions during each rebalance. Now, we will need to "maintain" that data structure. There is also an initial bootstrap problem (that can also be addressed). This makes me wonder if it is better to implement selective rebalance - that way the overhead of going to ZK for partition information (for the affected topics alone) should be acceptable. It will also give us the added benefit of reducing the overall time for rebalance on a topic event. - Joel Koshy On Sept. 10, 2013, 6:29 p.m., Guozhang Wang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/14041/ > ----------------------------------------------------------- > > (Updated Sept. 10, 2013, 6:29 p.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1030 > https://issues.apache.org/jira/browse/KAFKA-1030 > > > Repository: kafka > > > Description > ------- > > Using the approach of reading directly from ZK. > > > Diffs > ----- > > core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala > e7a692a1d23ca5a9ecf86e3cb34be418b9c0c943 > > Diff: https://reviews.apache.org/r/14041/diff/ > > > Testing > ------- > > unit tests > > > Thanks, > > Guozhang Wang > >
