[
https://issues.apache.org/jira/browse/KAFKA-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jun Rao updated KAFKA-1301:
---------------------------
Description:
The problem is that the producer doesn't drain the unsent data properly on
close. The problem is in the following code in Sender.run(). It's possible for
this loop to exit with unfinished requests.
// okay we stopped accepting requests but there may still be
// requests in the accumulator or waiting for acknowledgment,
// wait until these are completed.
int unsent = 0;
do {
try {
unsent = run(time.milliseconds());
} catch (Exception e) {
log.error("Uncaught error in kafka producer I/O thread: ", e);
}
} while (unsent > 0 || this.inFlightRequests.totalInFlightRequests() >
0);
Suppose that all produce requests are being sent, but the sender is waiting for
responses. Then the broker failed. In handling disconnects, we cleared all
inflight requests. When we check the condition in the while clause, there is no
unsent data and no in flight requests. However, failed records have been added
to RecordAccumulator and are ready to be sent in the next iteration.
was:
The problem is that the producer doesn't drain the unsent data properly on
close. The problem is in the following code. It's possible for this loop to
exit with unfinished requests.
// okay we stopped accepting requests but there may still be
// requests in the accumulator or waiting for acknowledgment,
// wait until these are completed.
int unsent = 0;
do {
try {
unsent = run(time.milliseconds());
} catch (Exception e) {
log.error("Uncaught error in kafka producer I/O thread: ", e);
}
} while (unsent > 0 || this.inFlightRequests.totalInFlightRequests() >
0);
Suppose that all produce requests are being sent, but the sender is waiting for
responses. Then the broker failed. In handling disconnects, we cleared all
inflight requests. When we check the condition in the while clause, there is no
unsent data and no in flight requests. However, failed records have been added
to RecordAccumulator and are ready to be sent in the next iteration.
> system testcase_0206 fails using the new producer
> -------------------------------------------------
>
> Key: KAFKA-1301
> URL: https://issues.apache.org/jira/browse/KAFKA-1301
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8.2
> Reporter: Jun Rao
>
> The problem is that the producer doesn't drain the unsent data properly on
> close. The problem is in the following code in Sender.run(). It's possible
> for this loop to exit with unfinished requests.
> // okay we stopped accepting requests but there may still be
> // requests in the accumulator or waiting for acknowledgment,
> // wait until these are completed.
> int unsent = 0;
> do {
> try {
> unsent = run(time.milliseconds());
> } catch (Exception e) {
> log.error("Uncaught error in kafka producer I/O thread: ", e);
> }
> } while (unsent > 0 || this.inFlightRequests.totalInFlightRequests()
> > 0);
> Suppose that all produce requests are being sent, but the sender is waiting
> for responses. Then the broker failed. In handling disconnects, we cleared
> all inflight requests. When we check the condition in the while clause, there
> is no unsent data and no in flight requests. However, failed records have
> been added to RecordAccumulator and are ready to be sent in the next
> iteration.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
