GitHub user rhauch opened a pull request:
https://github.com/apache/kafka/pull/3717
KAFKA-5731 Corrected how the sink task worker updates the last committed
offsets (0.10.2)
Prior to this change, it was possible for the synchronous consumer commit
request to be handled before previously-submitted asynchronous commit requests.
If that happened, the out-of-order handlers improperly set the last committed
offsets, which then became inconsistent with the offsets the connector task is
working with.
This change ensures that the last committed offsets are updated only for
the most recent commit request, even if the consumer reorders the calls to the
callbacks.
This change also backports the fix for KAFKA-4942, which was minimal that
caused the new tests to fail.
**This is for the `0.10.2` branch; see #3662 for the equivalent and
already-approved PR for `trunk` and #3672 for the equivalent and
already-approved PR for the `0.11.0` branch.**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rhauch/kafka kafka-5731-0.10.2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/3717.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3717
----
commit 4e39118ae302980e6d8b75cb09534172d99d5b7a
Author: Randall Hauch <[email protected]>
Date: 2017-08-12T00:42:06Z
KAFKA-5731 Corrected how the sink task worker updates the last committed
offsets
Prior to this change, it was possible for the synchronous consumer commit
request to be handled before previously-submitted asynchronous commit requests.
If that happened, the out-of-order handlers improperly set the last committed
offsets, which then became inconsistent with the offsets the connector task is
working with.
This change ensures that the last committed offsets are updated only for
the most recent commit request, even if the consumer reorders the calls to the
callbacks.
commit 6f74ef8e89f1193a5deb66870022591d51eb6580
Author: Randall Hauch <[email protected]>
Date: 2017-08-14T19:11:08Z
KAFKA-5731 Corrected mock consumer behavior during rebalance
Corrects the test case added in the previous commit to properly revoke the
existing partition assignments before adding new partition assigments.
commit 3c2531b1abdaf3cdaac3781a45597a616652ff1c
Author: Randall Hauch <[email protected]>
Date: 2017-08-14T19:11:45Z
KAFKA-5731 Added expected call that was missing in another test
commit 05567b1677e7f5a39ca0f20d86773c872193da0b
Author: Randall Hauch <[email protected]>
Date: 2017-08-14T22:24:35Z
KAFKA-5731 Improved log messages related to offset commits
# Conflicts:
#
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java
commit 60eba0ce024f3a211f33bf20bc04febbebb7d1c4
Author: Randall Hauch <[email protected]>
Date: 2017-08-15T14:47:05Z
KAFKA-5731 More cleanup of log messages related to offset commits
commit ff123bfb910742e3a5c320fff6b23ff645ef62a2
Author: Randall Hauch <[email protected]>
Date: 2017-08-15T16:21:52Z
KAFKA-5731 More improvements to the log messages in WorkerSinkTask
# Conflicts:
#
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java
commit d5f1b29a4cb41c139094cbed9b78cf51594f861c
Author: Randall Hauch <[email protected]>
Date: 2017-08-15T16:31:28Z
KAFKA-5731 Removed unnecessary log message
commit f2b02cda83876b7d55f331cf89d9a306ab2b467f
Author: Randall Hauch <[email protected]>
Date: 2017-08-15T17:54:16Z
KAFKA-5731 Additional tweaks to debug and trace log messages to ensure
clarity and usefulness
commit fa427b7557b93a600e0007e1a4adfb4aa38f526b
Author: Randall Hauch <[email protected]>
Date: 2017-08-15T19:30:09Z
KAFKA-5731 Use the correct value in trace messages
commit 957f0acac6ffff154fda6b522e89c00df3dcb299
Author: Randall Hauch <[email protected]>
Date: 2017-08-22T23:29:52Z
KAFKA-4942 Fix commitTimeoutMs being set before the commit actually started
Backported the fix for this issue, which was fixed in 0.11.0.0
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
