Thanks Benny! We discussed this option during the meeting but we did not prefer it because we did not want to leak the SQL identifiers to the storage table since SQL identifiers are view concepts and fit better with the view.
Thanks, Walaa. On Thu, Aug 8, 2024 at 4:12 PM Benny Chow <btc...@gmail.com> wrote: > Maybe a third option is to decouple the view lineage and materialization > state. > > The view lineage can just list out the SQL identifiers+ref... we can still > decide whether this is just direct children or fully expanded. > The materialization state doesn't have to depend on the view lineage > (through either sequence ID or UUIDs). It's a direct mapping of SQL > identifiers (possibly with refs) to their corresponding snapshot id or view > versions. > > The materialization state was our original design until Dan brought up the > idea to also capture the view lineage. So, what I am suggesting is that we > do both but not couple them together through any sort of sequence ID or > UUID. > > Thanks > Benny > > On Thu, Aug 8, 2024 at 2:04 PM Walaa Eldin Moustafa <wa.moust...@gmail.com> > wrote: > >> Hi Everyone, >> >> In the last community sync on Materialized Views [1], we agreed to split >> the information that is used to determine the materialized view staleness >> to two parts: Lineage Information and State Information. We have made a lot >> of progress on representing both but one issue remains open: >> >> Both lineage information and state information express facts about the >> child objects of the MV, and to correlate those facts we need some sort of >> an ID for those objects. >> The open question is whether the ID should be of type sequence ID or UUID >> (i.e., the Iceberg UUID of the object). >> >> I have described the issue in more detail in this doc [2]. I have also >> added some thoughts on the advantages of using UUIDs in the same doc. >> Further, there is a discussion thread on that topic in the MV spec doc [3]. >> >> Please feel free to comment on the docs or share your thoughts in the >> thread. After the discussion we can move forward to a vote. >> >> Thanks, >> Walaa. >> >> [1] https://lists.apache.org/thread/bhmxo1w1bdp1p2hh842kpm2gy1g5rscp >> [2] >> https://docs.google.com/document/d/1-OaPqm8ahVT3_OCbVdAPQ_wZ8I3ToeqU3RLUjcyKQM0/edit >> [3] >> https://docs.google.com/document/d/1UnhldHhe3Grz8JBngwXPA6ZZord1xMedY5ukEhZYF-A/edit?disco=AAABPUgvras >> >
