[
https://issues.apache.org/jira/browse/KAFKA-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769994#comment-13769994
]
Neha Narkhede commented on KAFKA-1030:
--------------------------------------
Thanks for the updated patch and the performance comparison analysis. I agree
that the ideal change might prove to be too large for 0.8 and will require
non-trivial amount of time stabilizing it since it is fairly tricky. We can
just do it properly on trunk and live with this minor performance hit for
consumer rebalancing on 0.8.
> Addition of partitions requires bouncing all the consumers of that topic
> ------------------------------------------------------------------------
>
> Key: KAFKA-1030
> URL: https://issues.apache.org/jira/browse/KAFKA-1030
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Swapnil Ghike
> Assignee: Guozhang Wang
> Priority: Blocker
> Fix For: 0.8
>
> Attachments: KAFKA-1030-v1.patch
>
>
> Consumer may not notice new partitions because the propagation of the
> metadata to servers can be delayed.
> Options:
> 1. As Jun suggested on KAFKA-956, the easiest fix would be to read the new
> partition data from zookeeper instead of a kafka server.
> 2. Run a fetch metadata loop in consumer, and set auto.offset.reset to
> smallest once the consumer has started.
> 1 sounds easier to do. If 1 causes long delays in reading all partitions at
> the start of every rebalance, 2 may be worth considering.
>
> The same issue affects MirrorMaker when new topics are created, MirrorMaker
> may not notice all partitions of the new topics until the next rebalance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
