[
https://issues.apache.org/jira/browse/KAFKA-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752619#comment-13752619
]
Neha Narkhede commented on KAFKA-1030:
--------------------------------------
One alternative is to change the topic metadata request handling on the
controller broker and let the consumer re-issue a metadata request to the
controller broker when the partition change listener fires. If the broker is
the controller, it should serve the metadata request by reading from the
controller cache directly. If not, it can rely on leaderCache that is updated
via the UpdateMetadata request. Upside of this approach is that it won't kill
performance and will solve the problem. Downside is that it might make the
metadata request handling on the controller broker somewhat slower since it
invovlves locking on the controller lock.
> Addition of partitions requires bouncing all the consumers of that topic
> ------------------------------------------------------------------------
>
> Key: KAFKA-1030
> URL: https://issues.apache.org/jira/browse/KAFKA-1030
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Swapnil Ghike
> Assignee: Swapnil Ghike
> Priority: Blocker
> Fix For: 0.8
>
>
> Consumer may not notice new partitions because the propagation of the
> metadata to servers can be delayed.
> Options:
> 1. As Jun suggested on KAFKA-956, the easiest fix would be to read the new
> partition data from zookeeper instead of a kafka server.
> 2. Run a fetch metadata loop in consumer, and set auto.offset.reset to
> smallest once the consumer has started.
> 1 sounds easier to do. If 1 causes long delays in reading all partitions at
> the start of every rebalance, 2 may be worth considering.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
