Thanks Jan. I think we moved on to more alignment steps beyond that doc a while ago. After that doc, we have discussed the topic further in 2 dev list threads and one more doc <https://docs.google.com/document/d/1zg0wQ5bVKTckf7-K_cdwF4mlRi6sixLcyEh6jErpGYY/edit?pli=1> (with strictly two options for the storage model to consider). Moreover, the original doc grew to 14 pages long with one section comparing 5 design alternatives, which made things harder to reach consensus. The lack of consensus is what partly led up to the subsequent discussions and call for a more focused approach to reach consensus. If we already have a consensus on the storage model (separate tables and views), I think we should take things further and have continued focused discussions on the specific metadata in the form of a PR. I have included all previous discussions including the original doc and issue as references in the PR description. Please let me know if this works. Happy to hear others' thoughts on the best way to move forward.
Thanks, Walaa. On Wed, May 8, 2024 at 12:56 AM Jan Kaul <jank...@mailbox.org.invalid> wrote: > Thanks Walaa for trying to move things along. However I don't think it's a > good idea to start a separate discussion about the metadata for > materialized views because we already had this discussion and reached > consensus in this google doc: > > > https://docs.google.com/document/d/1UnhldHhe3Grz8JBngwXPA6ZZord1xMedY5ukEhZYF-A/edit?usp=sharing > > Once the draft is finalized we can adopt the PR to reflect the consensus > from the google doc. > > Best wishes, > > Jan > On 07.05.24 19:11, Walaa Eldin Moustafa wrote: > > Thanks Steven. I feel it is needed so the MV spec is not scattered across > the table and view spec pages. We may add a reference in each respective > properties section. > > On Tue, May 7, 2024 at 10:04 AM Steven Wu <stevenz...@gmail.com> wrote: > >> Walaa, thanks for initiating the next step. >> >> With the agreed model of separate view and storage table, I am wondering >> if a separate materialized view spec page is needed. E.g., the new view >> metadata (view-materialized and view-storage-table) is probably good to be >> added to the view page directly to avoid information scattering. The same >> can be said about the storage table metadata. >> >> We may keep the separate materialized view page to document motivation, >> freshness semantics, etc.. >> >> On Mon, May 6, 2024 at 10:58 PM Walaa Eldin Moustafa < >> wa.moust...@gmail.com> wrote: >> >>> Hi Everyone, >>> >>> Thanks again for participating in the modeling discussion [1]. Since the >>> outcome of this discussion was to model materialized views as separate >>> objects, an Iceberg view and a table, I think the next step should be >>> discussing the metadata details for each object. I have created a PR >>> https://github.com/apache/iceberg/pull/10280 with an initial spec >>> improvement. Please feel free to review it and leave feedback there. >>> >>> [1] https://lists.apache.org/thread/rotmqzmwk5jrcsyxhzjhrvcjs5v3yjcc >>> >>> Thanks, >>> Walaa. >>> >>>
