Hi Chris, Thanks for all the PRs (wow, 10K lines of changes). EOS on source connector is a great feature in Kafka connect!
I will start to review this PR next week: Producer fencing API for admin client: https://github.com/apache/kafka/pull/11777 Thank you. Luke On Fri, Feb 18, 2022 at 2:36 AM Chris Egerton <chr...@confluent.io.invalid> wrote: > Hi all, > > It's been a little over eight months since KIP-618 ( > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-618%3A+Exactly-Once+Support+for+Source+Connectors > ) > was approved, and nearly as long since a feature- and test-complete > implementation (initially published in > https://github.com/apache/kafka/pull/10907) has been ready for review. > > Since then, there have been two rounds of review that, combined, cover less > than half of the code base. At this rate, I estimate it won't receive a > complete review--let alone be approved and merged--until October this year, > at the earliest. > > I understand that review for moderately large features like this one (the > current diff stacks up at a little over 10K additions) is time-consuming, > but would it be possible to expedite things a little here? > > I've taken the suggestion of Mickael Maison and split the old, monolithic > PR up into several smaller ones that should be easier to review. > > These PRs are ready to review right now: > - Producer fencing API for admin client: > https://github.com/apache/kafka/pull/11777 > - Metrics for source task transactions: > https://github.com/apache/kafka/pull/11772 > - Source connector APIs: https://github.com/apache/kafka/pull/11773 > - Utility methods used in downstream PRs: > https://github.com/apache/kafka/pull/11774 > > These require changes either from PRs in the above list, or from others in > this list: > - User-facing config properties: > https://github.com/apache/kafka/pull/11775 > - Preflight validation: https://github.com/apache/kafka/pull/11776 > - Protected config topic writes by the leader: > https://github.com/apache/kafka/pull/11778 > - Zombie fencing: https://github.com/apache/kafka/pull/11779 > - Transactional source tasks: https://github.com/apache/kafka/pull/11780 > - Per-connector offsets topics: https://github.com/apache/kafka/pull/11781 > > And these bring together all of the above and add end-to-end testing: > - Embedded integration tests: https://github.com/apache/kafka/pull/11782 > - System tests: https://github.com/apache/kafka/pull/11783 > > I'd like it if we could try to get this feature merged in time for the > upcoming 3.2 release. If anyone believes this isn't possible, please let me > know. > > Cheers, > > Chris >
