[
https://issues.apache.org/jira/browse/KAFKA-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15183559#comment-15183559
]
ASF GitHub Bot commented on KAFKA-3334:
---------------------------------------
GitHub user SinghAsDev opened a pull request:
https://github.com/apache/kafka/pull/1022
KAFKA-3334: Add a note on potential message loss while using auto top…
…ics creation.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/SinghAsDev/kafka KAFKA-3334
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1022.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1022
----
commit d78df396dfbcc0b5e13765dfff33f54156cb9c5e
Author: Ashish Singh <asi...@cloudera.com>
Date: 2016-03-07T19:46:58Z
KAFKA-3334: Add a note on potential message loss while using auto topics
creation.
----
> First message on new topic not actually being sent, no exception thrown
> -----------------------------------------------------------------------
>
> Key: KAFKA-3334
> URL: https://issues.apache.org/jira/browse/KAFKA-3334
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.9.0.0
> Environment: Linux, Java
> Reporter: Aleksandar Stojadinovic
> Assignee: Ashish K Singh
>
> Although I've seen this issue pop around the internet in a few forms, I'm not
> sure it is yet properly fixed.
> When publishing to a new topic, with auto create-enabled, the java client
> (0.9.0) shows this WARN message in the log, and the message is not sent
> obviously:
> org.apache.kafka.clients.NetworkClient - Error while fetching metadata with
> correlation id 0 : {file.topic=LEADER_NOT_AVAILABLE}
> In the meantime I see in the console the message that a log for partition is
> created. The next messages are patched through normally, but the first one is
> never sent. No exception is ever thrown, either by calling get on the future,
> or with the async usage, like everything is perfect.
> I notice when I leave my application blocked on the get call, in the
> debugger, then the message may be processed, but with significant delay. This
> is consistent with another issue I found for the python client. Also, if I
> call partitionsFor previously, the topic is created and the message is sent.
> But it seems silly to call it every time, just to mitigate this issue.
> {code}
> Future<RecordMetadata> recordMetadataFuture = producer.send(new
> ProducerRecord<>(topic, key, file));
> RecordMetadata recordMetadata = recordMetadataFuture.get(30,
> TimeUnit.SECONDS);
> {code}
> I hope I'm clear enough.
> Related similar (but not same) issues:
> https://issues.apache.org/jira/browse/KAFKA-1124
> https://github.com/dpkp/kafka-python/issues/150
> http://stackoverflow.com/questions/35187933/how-to-resolve-leader-not-available-kafka-error-when-trying-to-consume
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
