Hi Marc, Could you clarify where you store the view definitions in this case, and how the syntax looks like?
Thanks, Walaa. On Tue, Nov 15, 2022 at 2:34 PM Ryan Blue <b...@tabular.io> wrote: > Hi Marc, > > This is expected. Although the ViewCatalog SPIP was approved by the Spark > community, the implementation hasn't made it in yet for v2. > > Ryan > > On Tue, Nov 15, 2022 at 11:38 AM Marc Laforet <mlafor...@gmail.com> wrote: > >> Hi Iceberg folks, >> >> I'm working on a project where we're migrating tables from hive to >> iceberg. We are revamping our ingestion pipeline in parallel from batch to >> stream. Originally, our plan was to have two separate tables, a backfill >> table and a live table, that would be stitched together via a view for >> downstream consumers. This is proving rather difficult. In the absence of >> engine agnostic views we were going to prepend views with the engine type >> (ie trino_my_table and spark_my_table) but I receive a >> org.apache.spark.sql.AnalysisException: >> Catalog iceberg_catalog does not support views error when trying to >> create the spark view. With the ongoing work towards engine agnostic views >> I'm unsure if this limitation is expected or easily surpassed with some >> config/spark change? >> >> Thank you for your time, >> >> Marc >> > > > -- > Ryan Blue > Tabular >
