Guozhang Wang created KAFKA-2799:
------------------------------------

             Summary: WakupException thrown in the followup poll() could lead 
to data loss
                 Key: KAFKA-2799
                 URL: https://issues.apache.org/jira/browse/KAFKA-2799
             Project: Kafka
          Issue Type: Bug
            Reporter: Guozhang Wang
            Assignee: Guozhang Wang
            Priority: Blocker
             Fix For: 0.9.0.0


The common pattern of the new consumer:

{code}
try {
   records = consumer.poll();
   // process records
} catch (WakeupException) {
   consumer.close()
}
{code}

in which the close() can commit offsets. But since in the poll() call, we do 
the following order:

1) trigger client.poll().
2) possibly update consumed position if there are some data from fetch response.
3) before return the records, possibly trigger another client.poll()

And if wakeup exception is thrown in 3) it will lead to not-returned messages 
to be committed hence data loss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to