[
https://issues.apache.org/jira/browse/KAFKA-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14715362#comment-14715362
]
Dmitry Stratiychuk commented on KAFKA-2478:
-------------------------------------------
Or flush the buffer and commit at the end of processing all records.
Is there a way to control how many records are returned by the poll() method?
> KafkaConsumer javadoc example seems wrong
> -----------------------------------------
>
> Key: KAFKA-2478
> URL: https://issues.apache.org/jira/browse/KAFKA-2478
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.8.3
> Reporter: Dmitry Stratiychuk
> Assignee: Neha Narkhede
>
> I was looking at this KafkaConsumer example in the javadoc:
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L199
> As I understand, commit() method commits the maximum offsets returned by the
> most recent invocation of poll() method.
> In this example, there's a danger of losing the data.
> Imagine the case where 300 records are returned by consumer.poll()
> The commit will happen after inserting 200 records into the database.
> But it will also commit the offsets for 100 records that are still
> unprocessed.
> So if consumer fails before buffer is dumped into the database again,
> then those 100 records will never be processed.
> If I'm wrong, could you please clarify the behaviour of commit() method?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
