Re: Ingestion Layer?

2023-11-09 Thread Jean-Baptiste Onofré
Hi Austin I agree. The idea is to have a ingestion layer with a kind of mix of Apache Camel (for EIPs), Apache Beam like IOs, etc. I started a PoC powered by Apache Karaf Minho like runtime. I should be able to share a first more concrete proposal by next week (sorry still no electricity at home

Ingestion Layer?

2023-11-09 Thread Austin Bennett
Just a little comment that I imagine there would be massive value from an ingestion layer. Making it easier to add more integrations will be a great benefit for the ecosystem, adoption. Concretely, FWIW, I'm evaluating Iceberg [ and alternatives ] for an enterprise adoption, and existing integra

Re: Kafka Connect sink

2023-11-09 Thread Jean-Baptiste Onofré
Hi Bryan I would like to follow up about kafka connect sink donation. If you don't mind, I would like to propose a PR (on the tabular repo) with the changes we would need for the donation into the Apache Iceberg repo. I should have time to work on this next week. Thoughts ? Thanks Regards JB O

Re: Updating the Iceberg table architecture diagram

2023-11-09 Thread Jean-Baptiste Onofré
Hi I think we have to keep it "clear and simple" as possible. I would prefer to have one diagram per spec version (to be clear in the scope). So, I would rather keep the current diagram (working for v1) and add a new one v2 centric. It would be great to have a "side by side" presentation, somethi