[
https://issues.apache.org/jira/browse/KAFKA-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431812#comment-15431812
]
ASF GitHub Bot commented on KAFKA-2894:
---------------------------------------
GitHub user kkonstantine opened a pull request:
https://github.com/apache/kafka/pull/1771
KAFKA-2894: WorkerSinkTask should rewind offsets on rebalance
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kkonstantine/kafka
KAFKA-2894-rewind-offsets-on-rebalance
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1771.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1771
----
commit d14eff6084bafcf5014c8309703faafd96fe7071
Author: Konstantine Karantasis <k.karanta...@gmail.com>
Date: 2016-08-22T23:30:27Z
KAFKA-2894: WorkerSinkTask should rewind offsets on rebalance
----
> WorkerSinkTask doesn't handle rewinding offsets on rebalance
> ------------------------------------------------------------
>
> Key: KAFKA-2894
> URL: https://issues.apache.org/jira/browse/KAFKA-2894
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 0.9.0.0
> Reporter: Ewen Cheslack-Postava
> Assignee: Liquan Pei
> Priority: Blocker
> Fix For: 0.10.1.0
>
>
> rewind() is only invoked at the beginning of each poll(). This means that if
> a rebalance occurs in the poll, it's feasible to get data that doesn't match
> a request to change offsets during the rebalance. I think the consumer will
> hold on to consumer data across the rebalance if it is reassigned the same
> offset, so there may already be data ready to be delivered. Additionally we
> may already have data in an incomplete messageBatch that should be discarded
> when the rewind is requested.
> While connectors that care about this (i.e. ones that manage their own
> offsets) can handle this correctly by tracking the offsets they're expecting
> to see, it's a hassle, error prone, an pretty unintuitive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
