I have a producer that fails to get metadata when it first attempts to send
a record to a certain topic. It fails on
ClusterAndWaitTime clusterAndWaitTime = waitOnMetadata(record.topic(),
record.partition(), maxBlockTimeMs); [0]
and yields:
org.apache.kafka.common.errors.TimeoutException: Failed to update
metadata after 30000 ms.
After the first attempt, however, the producer works fine.
This problem does not occur when I isolate the producer in a test.
However, when embedded in my service layer—which includes things like a
Kafka Streams application and a gRPC service—the above behavior occurs. I
played with this for a while and found that this occurs specifically when
the producer producers to another topic just before it produces to the one
that fails. If I remove the producing to the first topic, producing to the
second topic succeeds, even on the first try.
Please let me know what data I could provide to help debug this. I looked
at logs at DEBUG level for the NetworkClient but nothing stuck out at me. I
am on 0.10.2.1, but this happens on 0.10.2.0 and 0.10.1.0 as well. I am not
using topic auto creation.
Thank you,
Dmitry
[0]
https://github.com/apache/kafka/blob/0.10.2.1/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L450
