Guozhang Wang created KAFKA-2799: ------------------------------------ Summary: WakupException thrown in the followup poll() could lead to data loss Key: KAFKA-2799 URL: https://issues.apache.org/jira/browse/KAFKA-2799 Project: Kafka Issue Type: Bug Reporter: Guozhang Wang Assignee: Guozhang Wang Priority: Blocker Fix For: 0.9.0.0
The common pattern of the new consumer: {code} try { records = consumer.poll(); // process records } catch (WakeupException) { consumer.close() } {code} in which the close() can commit offsets. But since in the poll() call, we do the following order: 1) trigger client.poll(). 2) possibly update consumed position if there are some data from fetch response. 3) before return the records, possibly trigger another client.poll() And if wakeup exception is thrown in 3) it will lead to not-returned messages to be committed hence data loss. -- This message was sent by Atlassian JIRA (v6.3.4#6332)