Aleksandar Stojadinovic created KAFKA-3334:
----------------------------------------------
Summary: First message on new topic not actually being sent, no
exception thrown
Key: KAFKA-3334
URL: https://issues.apache.org/jira/browse/KAFKA-3334
Project: Kafka
Issue Type: Bug
Affects Versions: 0.9.0.0
Environment: Linux, Java
Reporter: Aleksandar Stojadinovic
Although I've seen this issue pop around the internet in a few forms, I'm not
sure it is yet properly fixed.
When publishing to a new topic, with auto create-enabled, the java client
(0.9.0) shows this WARN message in the log, and the message is not sent
obviously:
org.apache.kafka.clients.NetworkClient - Error while fetching metadata with
correlation id 0 : {file.topic=LEADER_NOT_AVAILABLE}
In the meantime I see in the console the message that a log for partition is
created. The next messages are patched through normally, but the first one is
never sent. No exception is ever thrown, either by calling get on the future,
or with the async usage, like everything is perfect.
I notice when I leave my application blocked on the get call, in the debugger,
then the message may be processed, but with significant delay. This is
consistent with another issue I found for the python client. Also, if I call
partitionsFor previously, the topic is created and the message is sent. But it
seems silly to call it every time, just to mitigate this issue.
{code}
Future<RecordMetadata> recordMetadataFuture = producer.send(new
ProducerRecord<>(topic, key, file));
RecordMetadata recordMetadata = recordMetadataFuture.get(30,
TimeUnit.SECONDS);
{code}
I hope I'm clear enough.
Related similar (but not same) issues:
https://issues.apache.org/jira/browse/KAFKA-1124
https://github.com/dpkp/kafka-python/issues/150
http://stackoverflow.com/questions/35187933/how-to-resolve-leader-not-available-kafka-error-when-trying-to-consume
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
