[
https://issues.apache.org/jira/browse/KAFKA-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13753146#comment-13753146
]
Guozhang Wang commented on KAFKA-1030:
--------------------------------------
I think the first option might work just okay, since the consumer do not
actually needs the partition leader id, etc. All it needs is the map of topic
-> list of partition ids. This can be done by just reading one ZK path per
topic: /brokers/topics/[topic]. Of course this will put more pressure on
MirrorMaker though, but we should really not do full rebalance for added
partition or topic anyways..
> Addition of partitions requires bouncing all the consumers of that topic
> ------------------------------------------------------------------------
>
> Key: KAFKA-1030
> URL: https://issues.apache.org/jira/browse/KAFKA-1030
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Swapnil Ghike
> Assignee: Swapnil Ghike
> Priority: Blocker
> Fix For: 0.8
>
>
> Consumer may not notice new partitions because the propagation of the
> metadata to servers can be delayed.
> Options:
> 1. As Jun suggested on KAFKA-956, the easiest fix would be to read the new
> partition data from zookeeper instead of a kafka server.
> 2. Run a fetch metadata loop in consumer, and set auto.offset.reset to
> smallest once the consumer has started.
> 1 sounds easier to do. If 1 causes long delays in reading all partitions at
> the start of every rebalance, 2 may be worth considering.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
