Hi all, We at Tabular would like to contribute our Kafka Connect Iceberg sink to the Iceberg project. It would be great to give Iceberg users another option for landing data from Kafka into Iceberg tables that is supported by the Iceberg community. Kafka Connect is a part of systems from AWS, Confluent, Redpanda, and so on, so it can make landing data from Kafka into Iceberg much easier for those without a Flink or Spark infrastructure.
There are a few Iceberg sink implementations out there for Kafka Connect, but we feel this one covers most of the features users have requested, such as exactly-once processing, schema evolution, and multi-table fanout. And having the sink backed by the Iceberg community will help it to evolve and improve over time. If this sounds like something everyone would like to see added to Iceberg, I've opened a PR that includes some initial pieces of the sink. The thought was to break up the submission into parts so each could be reviewed more easily. Some design docs and notes can be found in the original repo here: https://github.com/tabular-io/iceberg-kafka-connect/tree/main/docs We'd like to get feedback if others approve of moving forward with this or not. Thanks, Bryan
