[
https://issues.apache.org/jira/browse/KAFKA-9372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009933#comment-17009933
]
Brian Byrne commented on KAFKA-9372:
------------------------------------
Hi [~Jiao-zhang] - we actually have a KIP in progress to add just this in the
form of metadata.eviction.period.ms:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-526%3A+Reduce+Producer+Metadata+Lookups+for+Large+Number+of+Topics
I'll take ownership of the ticket and update it once implemented. Thanks!
> Add producer config to make topicExpiry configurable
> ----------------------------------------------------
>
> Key: KAFKA-9372
> URL: https://issues.apache.org/jira/browse/KAFKA-9372
> Project: Kafka
> Issue Type: Improvement
> Components: producer
> Affects Versions: 1.1.0
> Reporter: Jiao Zhang
> Priority: Minor
>
> Sometimes we got error "org.apache.kafka.common.errors.TimeoutException:
> Failed to update metadata after 1000 ms" on producer side. We did the
> investigation and found
> # our producer produced messages in really low rate, the interval is more
> than 10 minutes
> # by default, producer would expire topics after TOPIC_EXPIRY_MS, after
> topic expired if no data produce before next metadata update (automatically
> triggered by metadata.max.age.ms) partitions entry for the topic would
> disappear from the Metadata cache As a result, almost for every time's
> produce, producer need fetch metadata which could possibly end with timeout.
> To solve this, we propose to add a new config metadata.topic.expiry for
> producer to make topicExpiry configurable. Topic expiry is good only when
> producer is long-lived and is used for producing variable counts of topics.
> But in the case that producers are bounded to single or few fixed topics,
> there is no need to expire topics at all.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
