Re: Materialized view integration with REST spec

2024-03-25 Thread Walaa Eldin Moustafa
Thanks Jan! I am not sure if you would like to make suggestions to revise the options themselves or the current options pros and cons. In either case, as mentioned earlier, we can do that on the doc and once we agree on the options and their pros and cons we can move forward. How does that sound?

Re: Materialized view integration with REST spec

2024-03-25 Thread Jan Kaul
I have the feeling that the current pros and cons from the summary target a version of the MV spec that wasn't really part of the discussion. The current arguments target a completely new specification for materialized views which we agreed on, is out of scope. Instead of a completely new speci

Equality deletes with Flink - design question

2024-03-25 Thread Gabor Kaszab
Hey Iceberg Community, I've recently had the chance to examine Iceberg's equality delete support in a multi-engine perspective (Flink, Hive, Impala, Spark). I started exploring how *Flink* can be used for writing and I observed that there is a restriction that the users are forced to add the *part

Re: Materialized view integration with REST spec

2024-03-25 Thread Benny Chow
Hi Manu This is Walaa's Spark implementation for option 1: https://github.com/apache/iceberg/pull/9830/files/a9e1bee3b5bf5914e5330d3b195042aea33868c9 There's no code for option 2 yet. Best Benny On Mon, Mar 25, 2024 at 12:37 AM Manu Zhang wrote: > Thanks Walaa for the summary. It's unclear to

Re: Materialized view integration with REST spec

2024-03-25 Thread Manu Zhang
Thanks Walaa for the summary. It's unclear to me which are the reference implementation for option 1 and reference MV spec for option 2 from the context. I can find some links in the References section but not sure which should be referred to respectively. On Mon, Mar 25, 2024 at 3:38 AM Walaa Eld