Hi Bryan I would like to follow up about kafka connect sink donation.
If you don't mind, I would like to propose a PR (on the tabular repo) with the changes we would need for the donation into the Apache Iceberg repo. I should have time to work on this next week. Thoughts ? Thanks Regards JB On Thu, Oct 19, 2023 at 11:13 AM Bryan Keller <brya...@gmail.com> wrote: > > Hi JB, > > The plan is to move forward, unless there are concerns from anyone. I got a > little bit sidetracked but will be working on addressing comments in the > first PR then following up with more PRs after that one is merged, so stay > tuned for those. > > Thanks, > Bryan > > > > > On Oct 18, 2023, at 7:13 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > > > > Hi Bryan, > > > > Any update on this thread ? Can I help somehow ? > > > > Thanks, > > Regards > > JB > > > >> On Mon, Oct 2, 2023 at 7:39 PM Bryan Keller <brya...@gmail.com> wrote: > >> > >> Hi all, > >> > >> We at Tabular would like to contribute our Kafka Connect Iceberg sink to > >> the Iceberg project. It would be great to give Iceberg users another > >> option for landing data from Kafka into Iceberg tables that is supported > >> by the Iceberg community. Kafka Connect is a part of systems from AWS, > >> Confluent, Redpanda, and so on, so it can make landing data from Kafka > >> into Iceberg much easier for those without a Flink or Spark infrastructure. > >> > >> There are a few Iceberg sink implementations out there for Kafka Connect, > >> but we feel this one covers most of the features users have requested, > >> such as exactly-once processing, schema evolution, and multi-table fanout. > >> And having the sink backed by the Iceberg community will help it to evolve > >> and improve over time. > >> > >> If this sounds like something everyone would like to see added to Iceberg, > >> I've opened a PR that includes some initial pieces of the sink. The > >> thought was to break up the submission into parts so each could be > >> reviewed more easily. Some design docs and notes can be found in the > >> original repo here: > >> https://github.com/tabular-io/iceberg-kafka-connect/tree/main/docs > >> > >> We'd like to get feedback if others approve of moving forward with this or > >> not. > >> > >> Thanks, > >> Bryan > >>
