Re: MOR CDC view support

Hi Micah, Brian Olsen proposed a way to publish or archive the slack chat to Discourse Forum . It's not done yet AFAIK. Yufei On Tue, Nov 21, 2023 at 10:47 AM Micah Kornfield wrote: > Slightly side topic: Are slack channels archived anywhere for offline > consumption (

Re: MOR CDC view support

Slightly side topic: Are slack channels archived anywhere for offline consumption (apologies if I missed it on the community page)? Thanks, Micah On Tue, Nov 21, 2023 at 6:07 AM Renjie Liu wrote: > Thanks for sharing. > > On Tue, Nov 21, 2023 at 21:52 Walaa Eldin Moustafa > wrote: > >> We met

Re: MOR CDC view support

Thanks for sharing. On Tue, Nov 21, 2023 at 21:52 Walaa Eldin Moustafa wrote: > We met on Wednesday and created the channel #cdc-read on Iceberg Slack. A > summary of the meeting discussion points is there. > > Thanks, > Walaa. > > On Tue, Nov 21, 2023 at 8:06 AM Renjie Liu > wrote: > >> Hi: >>

Re: MOR CDC view support

We met on Wednesday and created the channel #cdc-read on Iceberg Slack. A summary of the meeting discussion points is there. Thanks, Walaa. On Tue, Nov 21, 2023 at 8:06 AM Renjie Liu wrote: > Hi: > > Is there any update on this topic? > > On Tue, Nov 14, 2023 at 07:25 Yufei Gu wrote: > >> Hi f

Re: MOR CDC view support

Hi: Is there any update on this topic? On Tue, Nov 14, 2023 at 07:25 Yufei Gu wrote: > Hi folks, > > We will discuss it this Wednesday(11/15) at 9 am PST. Feel free to join if > you are interested. > > Sync-up for Iceberg CDC View on MOR > Wednesday, November 15 · 9:00 – 10:00am > Time zone: Am

Re: MOR CDC view support

Hi folks, We will discuss it this Wednesday(11/15) at 9 am PST. Feel free to join if you are interested. Sync-up for Iceberg CDC View on MOR Wednesday, November 15 · 9:00 – 10:00am Time zone: America/Los_Angeles Google Meet joining info Video call link: https://meet.google.com/zef-grqu-cqy Yufei

Re: MOR CDC view support

Hi Team, I was thinking about the possible implementations of a streaming read of MOR tables from Flink. I was checking the Spark code, and found that the feature is also missing from Spark. As Yufei mentioned, the building blocks are there, but the feature is not implemented yet. It would be good

Re: MOR CDC view support

Hi Pucheng, In short, we can reuse front-end infrastructure, including the changelog view procedure and iterators. We need some work from the reader side, it is not a trivial one, but some essential building blocks, like the `_deleted` metadata column, are there already. To get row-level deletes,

Re: MOR CDC view support

Feature request ticket: https://github.com/apache/iceberg/issues/8975 On Thu, Nov 2, 2023 at 9:16 PM Pucheng Yang wrote: > Hi community, > > I wonder if anyone is interested in having a MOR CDC view feature? My > organization is interested in using Flink upsert (MOR) into the Iceberg > table, bu

MOR CDC view support

Hi community, I wonder if anyone is interested in having a MOR CDC view feature? My organization is interested in using Flink upsert (MOR) into the Iceberg table, but currently the MOR CDC view is not supported. If we were to support it, do you know how much work it will be? How difficult will th