Hi all, It's been a little over eight months since KIP-618 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-618%3A+Exactly-Once+Support+for+Source+Connectors) was approved, and nearly as long since a feature- and test-complete implementation (initially published in https://github.com/apache/kafka/pull/10907) has been ready for review.
Since then, there have been two rounds of review that, combined, cover less than half of the code base. At this rate, I estimate it won't receive a complete review--let alone be approved and merged--until October this year, at the earliest. I understand that review for moderately large features like this one (the current diff stacks up at a little over 10K additions) is time-consuming, but would it be possible to expedite things a little here? I've taken the suggestion of Mickael Maison and split the old, monolithic PR up into several smaller ones that should be easier to review. These PRs are ready to review right now: - Producer fencing API for admin client: https://github.com/apache/kafka/pull/11777 - Metrics for source task transactions: https://github.com/apache/kafka/pull/11772 - Source connector APIs: https://github.com/apache/kafka/pull/11773 - Utility methods used in downstream PRs: https://github.com/apache/kafka/pull/11774 These require changes either from PRs in the above list, or from others in this list: - User-facing config properties: https://github.com/apache/kafka/pull/11775 - Preflight validation: https://github.com/apache/kafka/pull/11776 - Protected config topic writes by the leader: https://github.com/apache/kafka/pull/11778 - Zombie fencing: https://github.com/apache/kafka/pull/11779 - Transactional source tasks: https://github.com/apache/kafka/pull/11780 - Per-connector offsets topics: https://github.com/apache/kafka/pull/11781 And these bring together all of the above and add end-to-end testing: - Embedded integration tests: https://github.com/apache/kafka/pull/11782 - System tests: https://github.com/apache/kafka/pull/11783 I'd like it if we could try to get this feature merged in time for the upcoming 3.2 release. If anyone believes this isn't possible, please let me know. Cheers, Chris
