Hi
Our scenario -
One of the brokers in the kafka cluster is down.
The cluster has got rebalanced, but the metadata refresh does not bring the
same detail every time.
The metadata refresh waits indefinitely sometimes.
We are using kafka-clients jar version 0.8.2.1 to send data to kafka cluster
which is in 0.9.0.
The kafka producer thread starts fine and starts pumping data. But all of a
sudden it halts waiting indefinitely to do the metadata refresh.
Below is the jstack for one of the kafka producer threads (it is the same with
all the producer threads as well) -
"KAFKA-PUBLISHER_1" prio=10 tid=0x00007fb050880800 nid=0x68d in Object.wait()
[0x00007fafc44f3000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x000000060fb7c1f0> (a
org.apache.kafka.clients.producer.internals.Metadata)
at
org.apache.kafka.clients.producer.internals.Metadata.awaitUpdate(Metadata.java:107)
- locked <0x000000060fb7c1f0> (a
org.apache.kafka.clients.producer.internals.Metadata)
at
org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:378)
Can you tell us why the producer threads remain waiting given the below ?
* One of the brokers is down but the cluster has rebalanced and is up and
running
* The connectivity from the producer box to the cluster is good.
* We have not provided any value for "topic.metadata.refresh.interval.ms".
The default value of 10 mins should be taken.
Thanks
Shobana
