[
https://issues.apache.org/jira/browse/KAFKA-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15842831#comment-15842831
]
Stevo Slavic commented on KAFKA-4385:
-------------------------------------
{{UnknownTopicOrPartitionException}} extends {{InvalidMetadataException}} which
extends {{RetriableException}}.
This shows root cause of the problem - by current Kafka design
{{UnknownTopicOrPartitionException}} is considered to be retriable exception at
all times. IMO, when auto topic creation is disabled,
{{UnknownTopicOrPartitionException}} should not be considered as retriable.
Besides of unnecessary metadata retrieval retries, I've found that
{{KafkaProducer}}, in 0.9.0.1 and 0.10.1.1 with auto topic creation disabled,
when one tries to send to non-existing topic, registered callback will not be
completed with {{UnknownTopicOrPartitionException}}. Instead
{noformat}
"org.apache.kafka.common.errors.TimeoutException: Failed to update metadata
after X ms."
{noformat}
will be thrown.
> producer is sending too many unnecessary meta data request if the meta data
> for a topic is not available and "auto.create.topics.enable" =false
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-4385
> URL: https://issues.apache.org/jira/browse/KAFKA-4385
> Project: Kafka
> Issue Type: Bug
> Reporter: Jun Yao
>
> All current kafka-client producer implementation (<= 0.10.1.0),
> When sending a msg to a topic, it will first check if meta data for this
> topic is available or not,
> when not available, it will set "metadata.requestUpdate()" and wait for meta
> data from brokers,
> The thing is inside "org.apache.kafka.clients.Metadata.awaitUpdate()", it's
> already doing a "while (this.version <= lastVersion)" loop waiting for new
> version response,
> So the loop inside
> "org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata() is not
> needed,
> When "auto.create.topics.enable" is false, sending msgs to a non-exist topic
> will trigger too many meta requests, everytime a metadata response is
> returned, because it does not contain the metadata for the topic, it's going
> to try again until TimeoutException is thrown;
> This is a waste and sometimes causes too much overhead when unexpected msgs
> are arrived.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
