View support in Spark requires the implementation of the View DataSource V2 SPIP <https://docs.google.com/document/d/1XOxFtloiMuW24iqJ-zJnDzHl2KMxipTjJoxleJFz66A/edit#heading=h.c08703sbaxxl> (see associated Spark Jira SPARK-31357 <https://issues.apache.org/jira/browse/SPARK-31357>). The interface PR <https://github.com/apache/spark/pull/37556> of that work has already been merged, but this integration PR <https://github.com/apache/spark/pull/39796> needs to be merged as well. This is the umbrella PR <https://github.com/apache/spark/pull/35636> (with some comments for backward compatibility with V1 views). We have that setup at LinkedIn (after adapting the PRs and merging them internally), and we create/query views stored in the Hive Metastore (HMS), while the underlying tables are either in the HMS or the Iceberg catalog.
How did you create the views in the first place? Where does their definition live? Thanks, Walaa. On Tue, Sep 26, 2023 at 12:40 AM Eduard Tudenhoefner <edu...@tabular.io> wrote: > Views are currently not supported by Iceberg, but are under active > development. You can check the current status of Iceberg View support here > <https://github.com/apache/iceberg/projects/29>. > > Eduard > > On Mon, Sep 25, 2023 at 8:39 PM Agrawal, Sanket > <sankeagra...@deloitte.com.invalid> wrote: > >> Hello Everyone, >> >> >> >> We have setup spark and setup everything as mentioned at >> https://iceberg.apache.org/docs/latest/aws/ to integrate Spark, Iceberg, >> and AWS Glue Catalog. We are able to read tables through this but we are >> unable to read data through views. PFB, the error: >> >> >> >> pyspark.errors.exceptions.captured.AnalysisException: >> [TABLE_OR_VIEW_NOT_FOUND] The table or view `db`.`vw_name ` cannot be >> found. Verify the spelling and correctness of the schema and catalog. >> >> If you did not qualify the name with a schema, verify the >> current_schema() output, or qualify the name with the correct schema and >> catalog. >> >> To tolerate the error on drop use DROP VIEW IF EXISTS or DROP TABLE IF >> EXISTS.; line 1 pos 14; >> >> 'GlobalLimit 1 >> >> +- 'LocalLimit 1 >> >> +- 'Project [*] >> >> +- 'UnresolvedRelation [db, vw_name], [], false >> >> >> >> >> >> Any help or guidance would be of great help. >> >> >> >> Thank You, >> >> Sanket A. >> >> This message (including any attachments) contains confidential >> information intended for a specific individual and purpose, and is >> protected by law. If you are not the intended recipient, you should delete >> this message and any disclosure, copying, or distribution of this message, >> or the taking of any action based on it, by you is strictly prohibited. >> >> Deloitte refers to a Deloitte member firm, one of its related entities, >> or Deloitte Touche Tohmatsu Limited ("DTTL"). Each Deloitte member firm is >> a separate legal entity and a member of DTTL. DTTL does not provide >> services to clients. Please see www.deloitte.com/about to learn more. >> >> v.E.1 >> >
