+1 as well for separate objects. I think Netflix has proven this model works well. Exposure of the storage table can be handled either through naming convention or be hidden by the catalog, but that's more of an implementation detail.
-Dan On Mon, Apr 22, 2024 at 3:00 PM Ryan Blue <b...@tabular.io> wrote: > +1 for separate table and view objects and not needing to introduce > unnecessary combined APIs. > > On Mon, Apr 22, 2024 at 1:51 PM Szehon Ho <szehon.apa...@gmail.com> wrote: > >> +1 for the approach given it reduces the work. On this, as it exposes >> storage tables to user catalog, I was mainly thinking we should have a >> common suffix/naming pattern for storage table across catalog. The netflix >> approach sounds good to me. >> >> Hope we can continue the proposal, as there's still decisions on how to >> standardize other metadata like how MV lineages. >> >> Thanks, >> Szehon >> >> On Fri, Apr 19, 2024 at 6:17 PM John Zhuge <jzh...@apache.org> wrote: >> >>> +1 on separate view and table metadata >>> >>> I'd like to share our experience of such a design at Netflix for years. >>> The changes to the view spec are minimal and there are no changes to the >>> Iceberg table metadata other than tracking an additional table property for >>> capturing freshness. The storage tables have a specific suffix and a naming >>> pattern. It is convenient to use existing toolings on these tables. We have >>> not encountered any fundamental issues with this modeling. >>> >>> On Fri, Apr 19, 2024 at 5:49 AM Renjie Liu <liurenjie2...@gmail.com> >>> wrote: >>> >>>> +1 for this proposal. >>>> >>>> On Fri, Apr 19, 2024 at 3:40 PM Ajantha Bhat <ajanthab...@gmail.com> >>>> wrote: >>>> >>>>> +1 for the proposal. >>>>> >>>>> - Ajantha >>>>> >>>>> On Fri, Apr 19, 2024 at 7:29 AM Benny Chow <btc...@gmail.com> wrote: >>>>> >>>>>> +1 for separate view and table objects. Walaa's Spark >>>>>> implementation demonstrates how little change it takes on the Iceberg >>>>>> APIs >>>>>> to start sharing MVs between engines. >>>>>> >>>>>> Thanks >>>>>> Benny >>>>>> >>>>>> On Thu, Apr 18, 2024 at 9:52 AM Walaa Eldin Moustafa < >>>>>> wa.moust...@gmail.com> wrote: >>>>>> >>>>>>> Hi everyone, >>>>>>> >>>>>>> I would like to make a proposal for issue [1] to support >>>>>>> materialized views in Iceberg. The support leverages two separate >>>>>>> objects, >>>>>>> an Iceberg view and an Iceberg table to implement materialized views. >>>>>>> Each >>>>>>> object retains relevant metadata to support the MV operations. An >>>>>>> initial >>>>>>> design, which we can refine, is detailed in the description section of >>>>>>> this >>>>>>> PR [2]. >>>>>>> >>>>>>> This proposal is the outcome of extensive community discussions in >>>>>>> various forums [3, 4, 5, 6, 7]. >>>>>>> >>>>>>> Please respond with your recommendation: >>>>>>> +1 if you support moving forward with the two separate objects model. >>>>>>> 0 if you are neutral. >>>>>>> -1 if you disagree with the two separate objects model. >>>>>>> >>>>>>> Thanks, >>>>>>> Walaa. >>>>>>> >>>>>>> [1] https://github.com/apache/iceberg/issues/10043 >>>>>>> [2] https://github.com/apache/iceberg/pull/9830 >>>>>>> [3] >>>>>>> https://docs.google.com/document/d/1zg0wQ5bVKTckf7-K_cdwF4mlRi6sixLcyEh6jErpGYY >>>>>>> [4] https://github.com/apache/iceberg/issues/6420 >>>>>>> [5] >>>>>>> https://docs.google.com/document/d/1UnhldHhe3Grz8JBngwXPA6ZZord1xMedY5ukEhZYF >>>>>>> [6] https://lists.apache.org/thread/tb3wcs7czjvjbq9y1qtr87g9s95ky5zh >>>>>>> [7] https://lists.apache.org/thread/l6cvrp4r1001k08cy2ypybzy2kgxpt1y >>>>>>> >>>>>> >>> >>> -- >>> John Zhuge >>> >> > > -- > Ryan Blue > Tabular >
