Thanks Luke! Appreciate it. On Thu, Feb 17, 2022 at 10:02 PM Luke Chen <show...@gmail.com> wrote:
> Hi Chris, > > Thanks for all the PRs (wow, 10K lines of changes). > EOS on source connector is a great feature in Kafka connect! > > I will start to review this PR next week: > Producer fencing API for admin client: > https://github.com/apache/kafka/pull/11777 > > Thank you. > Luke > > On Fri, Feb 18, 2022 at 2:36 AM Chris Egerton <chr...@confluent.io.invalid > > > wrote: > > > Hi all, > > > > It's been a little over eight months since KIP-618 ( > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-618%3A+Exactly-Once+Support+for+Source+Connectors > > ) > > was approved, and nearly as long since a feature- and test-complete > > implementation (initially published in > > https://github.com/apache/kafka/pull/10907) has been ready for review. > > > > Since then, there have been two rounds of review that, combined, cover > less > > than half of the code base. At this rate, I estimate it won't receive a > > complete review--let alone be approved and merged--until October this > year, > > at the earliest. > > > > I understand that review for moderately large features like this one (the > > current diff stacks up at a little over 10K additions) is time-consuming, > > but would it be possible to expedite things a little here? > > > > I've taken the suggestion of Mickael Maison and split the old, monolithic > > PR up into several smaller ones that should be easier to review. > > > > These PRs are ready to review right now: > > - Producer fencing API for admin client: > > https://github.com/apache/kafka/pull/11777 > > - Metrics for source task transactions: > > https://github.com/apache/kafka/pull/11772 > > - Source connector APIs: https://github.com/apache/kafka/pull/11773 > > - Utility methods used in downstream PRs: > > https://github.com/apache/kafka/pull/11774 > > > > These require changes either from PRs in the above list, or from others > in > > this list: > > - User-facing config properties: > > https://github.com/apache/kafka/pull/11775 > > - Preflight validation: https://github.com/apache/kafka/pull/11776 > > - Protected config topic writes by the leader: > > https://github.com/apache/kafka/pull/11778 > > - Zombie fencing: https://github.com/apache/kafka/pull/11779 > > - Transactional source tasks: https://github.com/apache/kafka/pull/11780 > > - Per-connector offsets topics: > https://github.com/apache/kafka/pull/11781 > > > > And these bring together all of the above and add end-to-end testing: > > - Embedded integration tests: https://github.com/apache/kafka/pull/11782 > > - System tests: https://github.com/apache/kafka/pull/11783 > > > > I'd like it if we could try to get this feature merged in time for the > > upcoming 3.2 release. If anyone believes this isn't possible, please let > me > > know. > > > > Cheers, > > > > Chris > > >
