Awesome ! Let me do a new pass, but I'm in favor of merging it. It would be great to have it for Iceberg 1.5.0.
Thanks ! Regards JB On Thu, Jan 4, 2024 at 10:22 PM Bryan Keller <brya...@gmail.com> wrote: > > Hey JB, > > I went ahead and rebased with main. I believe all feedback has been addressed > so the PR is just waiting on those who might want another pass. > > -Bryan > > > On Jan 4, 2024, at 9:36 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > > > > Hi Bryan > > > > I did a new pass on https://github.com/apache/iceberg/pull/8701 and it > > looks good. Eduard, Fokko, Dan, and Ajantha also approved the PR. > > > > Maybe worth to rebase and merge on main, thoughts ? > > > > I'm also doing a pass/test with Kafka 3.6.1. > > > > Maybe we can sync together on Slack to move forward about that. > > > > Thanks ! > > Regards > > JB > > > > On Mon, Nov 13, 2023 at 3:31 PM Bryan Keller <brya...@gmail.com> wrote: > >> > >> Hey JB, > >> > >> Smaller PRs to the current repo are welcome, though I’m trying to keep > >> disruptive changes to a minimum during the submission process. I will have > >> the next PR out as soon as the first is merged. > >> > >> Bryan > >> > >>> On Nov 9, 2023, at 9:30 AM, Jean-Baptiste Onofré <j...@nanthrax.net> > >>> wrote: > >>> > >>> Hi Bryan > >>> > >>> I would like to follow up about kafka connect sink donation. > >>> > >>> If you don't mind, I would like to propose a PR (on the tabular repo) > >>> with the changes we would need for the donation into the Apache > >>> Iceberg repo. > >>> I should have time to work on this next week. > >>> > >>> Thoughts ? > >>> > >>> Thanks > >>> Regards > >>> JB > >>> > >>>> On Thu, Oct 19, 2023 at 11:13 AM Bryan Keller <brya...@gmail.com> wrote: > >>>> > >>>> Hi JB, > >>>> > >>>> The plan is to move forward, unless there are concerns from anyone. I > >>>> got a little bit sidetracked but will be working on addressing comments > >>>> in the first PR then following up with more PRs after that one is > >>>> merged, so stay tuned for those. > >>>> > >>>> Thanks, > >>>> Bryan > >>>> > >>>> > >>>> > >>>>>> On Oct 18, 2023, at 7:13 AM, Jean-Baptiste Onofré <j...@nanthrax.net> > >>>>>> wrote: > >>>>> > >>>>> Hi Bryan, > >>>>> > >>>>> Any update on this thread ? Can I help somehow ? > >>>>> > >>>>> Thanks, > >>>>> Regards > >>>>> JB > >>>>> > >>>>>> On Mon, Oct 2, 2023 at 7:39 PM Bryan Keller <brya...@gmail.com> wrote: > >>>>>> > >>>>>> Hi all, > >>>>>> > >>>>>> We at Tabular would like to contribute our Kafka Connect Iceberg sink > >>>>>> to the Iceberg project. It would be great to give Iceberg users > >>>>>> another option for landing data from Kafka into Iceberg tables that is > >>>>>> supported by the Iceberg community. Kafka Connect is a part of systems > >>>>>> from AWS, Confluent, Redpanda, and so on, so it can make landing data > >>>>>> from Kafka into Iceberg much easier for those without a Flink or Spark > >>>>>> infrastructure. > >>>>>> > >>>>>> There are a few Iceberg sink implementations out there for Kafka > >>>>>> Connect, but we feel this one covers most of the features users have > >>>>>> requested, such as exactly-once processing, schema evolution, and > >>>>>> multi-table fanout. And having the sink backed by the Iceberg > >>>>>> community will help it to evolve and improve over time. > >>>>>> > >>>>>> If this sounds like something everyone would like to see added to > >>>>>> Iceberg, I've opened a PR that includes some initial pieces of the > >>>>>> sink. The thought was to break up the submission into parts so each > >>>>>> could be reviewed more easily. Some design docs and notes can be found > >>>>>> in the original repo here: > >>>>>> https://github.com/tabular-io/iceberg-kafka-connect/tree/main/docs > >>>>>> > >>>>>> We'd like to get feedback if others approve of moving forward with > >>>>>> this or not. > >>>>>> > >>>>>> Thanks, > >>>>>> Bryan > >>>>>> >
