Hi Chris, Thanks for taking the time to split this up. It would indeed be nice to get this in 3.2.
I've merged a couple of them already, and I'll make another pass in the next few days. Thanks, Mickael On Sat, Feb 19, 2022 at 12:32 AM Chris Egerton <chr...@confluent.io.invalid> wrote: > > Thanks Luke! Appreciate it. > > On Thu, Feb 17, 2022 at 10:02 PM Luke Chen <show...@gmail.com> wrote: > > > Hi Chris, > > > > Thanks for all the PRs (wow, 10K lines of changes). > > EOS on source connector is a great feature in Kafka connect! > > > > I will start to review this PR next week: > > Producer fencing API for admin client: > > https://github.com/apache/kafka/pull/11777 > > > > Thank you. > > Luke > > > > On Fri, Feb 18, 2022 at 2:36 AM Chris Egerton <chr...@confluent.io.invalid > > > > > wrote: > > > > > Hi all, > > > > > > It's been a little over eight months since KIP-618 ( > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-618%3A+Exactly-Once+Support+for+Source+Connectors > > > ) > > > was approved, and nearly as long since a feature- and test-complete > > > implementation (initially published in > > > https://github.com/apache/kafka/pull/10907) has been ready for review. > > > > > > Since then, there have been two rounds of review that, combined, cover > > less > > > than half of the code base. At this rate, I estimate it won't receive a > > > complete review--let alone be approved and merged--until October this > > year, > > > at the earliest. > > > > > > I understand that review for moderately large features like this one (the > > > current diff stacks up at a little over 10K additions) is time-consuming, > > > but would it be possible to expedite things a little here? > > > > > > I've taken the suggestion of Mickael Maison and split the old, monolithic > > > PR up into several smaller ones that should be easier to review. > > > > > > These PRs are ready to review right now: > > > - Producer fencing API for admin client: > > > https://github.com/apache/kafka/pull/11777 > > > - Metrics for source task transactions: > > > https://github.com/apache/kafka/pull/11772 > > > - Source connector APIs: https://github.com/apache/kafka/pull/11773 > > > - Utility methods used in downstream PRs: > > > https://github.com/apache/kafka/pull/11774 > > > > > > These require changes either from PRs in the above list, or from others > > in > > > this list: > > > - User-facing config properties: > > > https://github.com/apache/kafka/pull/11775 > > > - Preflight validation: https://github.com/apache/kafka/pull/11776 > > > - Protected config topic writes by the leader: > > > https://github.com/apache/kafka/pull/11778 > > > - Zombie fencing: https://github.com/apache/kafka/pull/11779 > > > - Transactional source tasks: https://github.com/apache/kafka/pull/11780 > > > - Per-connector offsets topics: > > https://github.com/apache/kafka/pull/11781 > > > > > > And these bring together all of the above and add end-to-end testing: > > > - Embedded integration tests: https://github.com/apache/kafka/pull/11782 > > > - System tests: https://github.com/apache/kafka/pull/11783 > > > > > > I'd like it if we could try to get this feature merged in time for the > > > upcoming 3.2 release. If anyone believes this isn't possible, please let > > me > > > know. > > > > > > Cheers, > > > > > > Chris > > > > >
