[
https://issues.apache.org/jira/browse/KAFKA-3358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15216760#comment-15216760
]
Jason Gustafson edited comment on KAFKA-3358 at 3/29/16 8:16 PM:
-----------------------------------------------------------------
The main problem is that the current topic metadata request doesn't give us the
capability to actually fetch empty topic metadata (a fix for this problem was
proposed in the patch for KAFKA-3306). If we had that, then this issue would
resolve itself: if the Metadata contains no topics, we would only retrieve
broker metadata. To fix this problem currently, we have to skip the metadata
fetch entirely until we have a non-empty topic list. That would mean that the
bootstrap broker list would stay around until that time, which may or may not
be a problem depending on how the user has configured it (see discussion in
KAFKA-3068). I wonder if we are better off waiting until the "empty means
empty" topic metadata fix is checked in rather than attempting a short-term
workaround?
[~junrao] Any thoughts?
was (Author: hachikuji):
The main problem is that the current topic metadata request doesn't give us the
capability to actually fetch empty topic metadata (a fix for this problem was
proposed in the patch for KAFKA-3306). If we had that, then this patch would
resolve itself: if the Metadata contains no topics, we would only retrieve
broker metadata. To fix this problem currently, we have to skip the metadata
fetch entirely until we have a non-empty topic list. That would mean that the
bootstrap broker list would stay around until that time, which may or may not
be a problem depending on how the user has configured it (see discussion in
KAFKA-3068). I wonder if we are better off waiting until the "empty means
empty" topic metadata fix is checked in rather than attempting a short-term
workaround?
[~junrao] Any thoughts?
> Only request metadata updates once we have topics or a pattern subscription
> ---------------------------------------------------------------------------
>
> Key: KAFKA-3358
> URL: https://issues.apache.org/jira/browse/KAFKA-3358
> Project: Kafka
> Issue Type: Improvement
> Components: clients
> Affects Versions: 0.9.0.0, 0.9.0.1
> Reporter: Ismael Juma
> Assignee: Jason Gustafson
> Priority: Critical
> Fix For: 0.10.1.0
>
>
> The current code requests a metadata update for _all_ topics which can cause
> major load issues in large clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
