Then the above setup should work (assuming you create the view from Spark).
Thanks, Walaa. On Tue, Sep 26, 2023 at 7:50 AM Agrawal, Sanket <sankeagra...@deloitte.com.invalid> wrote: > It’s in the AWS Glue catalog. AWS Glue Catalog allows to create views on > top of Iceberg tables. > > > > Regards, > > Sanket A. > > > > *From:* Walaa Eldin Moustafa <wa.moust...@gmail.com> > *Sent:* Tuesday, September 26, 2023 8:00 PM > *To:* dev@iceberg.apache.org > *Subject:* [EXT] Re: Reading Glue catalog Views created on top of Iceberg > tables. > > > > View support in Spark requires the implementation of the View DataSource > V2 SPIP > <https://docs.google.com/document/d/1XOxFtloiMuW24iqJ-zJnDzHl2KMxipTjJoxleJFz66A/edit#heading=h.c08703sbaxxl> > (see > associated Spark Jira SPARK-31357 > <https://secure-web.cisco.com/1pAlkVXwFmqHaOB6tIW2tPJOQmPqMcGVYnUGt2CNc9JLIX7CS2lfh4i43zx1_AmXd0RnuOTZKv5u2K3LMlmuCfC1smgQ28ps8g__4YcuE_RnMSABpDXuSum-bJJXP3shd6MNfOhGQhhk02noHfD9tY4_vfpWPoJrRjqeV2NjIyR2Q0JZdx3sId1NHwhNve15_nAa_gKo5ExhPPe4bqrY_PlitSvKb9ZIGL46e2-t1jZ62y1ylQ5TZGjhwzajkbxJy_YfQrZISNceNaRNvnlCeZonMUVANq8nHyEJgomakmg-YczakhFWMtGC4ULCmEFyDQGnZbCXrd86ErwKOhyG_CMxjdT51LH_04LEzwOHvSg_a1qsCkREla6ykfHqytWsY5-RjR4HJCHUsUEoYO72ss_mOOw728JdhVj4-QzhrsK_GY7zcOBma57eNY-Gb9x5GVPqVbhfZzaqKZ1by87NOc3wwBcMt0UdgFYwjFuN24OYEznaAfoxBrVMV-crcnTgRjIsoPkkB_vxSY2nDvGBgLg/https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSPARK-31357>). > The interface PR > <https://secure-web.cisco.com/1qYTroKl7W3a3drWnbxd6jhSdG8438ooiFOvMGKSmZLhK8mc5rjnGWXZdVxcacv2VSEENi8ITgYcJLkDveL-8KdpS8Vz-qOJe2S1vephtIa27hJn6agz9sz6hAItwY3kpTM386AilDvynOZ2IdkqVpecnp4iL5C9E8-SgGndiQ0UEYXdoCfAkxVjO8GL8aHFwvq10IWhIkfieAr8HznelxMIwgEcwERNe9twsU3mofd4wY_AeNqCnneEsDdi83K2yQm6gu2CzzSoipImR5nInOSqspUVNR7S1oWbw1NGxe-0X85DQ-bVs1XN53zX3iUJ7Wlcm98FUo0DIcDi2s38Q_w13SOK8jDunM3gqvZ6sH4McxrrDOkhNwwJeCZUFt1ZaGy5OSHcC3FIBqMQSlOf1VODX6-EoGD4yUWWkoczJWa9Cz0Wlrjm-AHgaYHnJfEiQxzIsXH08zxyySxb3Q3EK3982D8sfblReuhA_K1FNZJN2sRMZi-lMdse6onRDV2gVD1nPTpGsvXAD5XaAsKUr5g/https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F37556> > of that work has already been merged, but this integration PR > <https://secure-web.cisco.com/1-38yC1JhdXUIKq6wcMkIjWWiGoMzn8Ps8ZnqkB4fQXmaordymgYcvFo1h3MwqxiC4ErjRTmOiHscsRYqL9ZEVLY4GLiOVNn_AHJojBr2js9HA60KsD9fOKSBAYxni_xvRhDYglIucL0d0xJy_TyLeVMZrtFLKpC-EE3LBDtihH7PKELmM7-H5PCC2VunXgDKbWnzeiZxFDi8-PqYxGf_WbGqxspQs7a_AlklEq6CI0zG84EjefxYLApC30-FXcddvom7KYVETecwvcRzpKT_WaOZm7DrKZWeXZao_LawZwW08Zz_ecJZoiULwYZSkFhvXrkxaHc7Ociu6ofB9NrY_K1sBdwbBTcNxpLFjZR_OY4gE5Y_XncU_4xhXKgwUIcqhogTf1m9RblmGs-ZHaSPdzxBiVVThsH_zLF6KeGnKeFpiToP8noP2GeGxyIUJ2YS1H4a6-bXULClUitlchBYFNMljay_7HEzlUFfXMuEKtNLpRM0M5tQ0VUPNabYYPIAB508-pwYmtdc0-Be5oJIHg/https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F39796> > needs to be merged as well. This is the umbrella PR > <https://secure-web.cisco.com/1ZepFBlGqtvD7ybnemRHFYjhVNg5BzsSaanifvYZz3OTr96CO3Khvk_3nlhFTGh5iyWc71cTyRYIMa64ufD2IvmFI4Y0r2udaBS56_842auwZm3J_flQRrvzJaIYKssjSS63YZD_SOLLPM4UyE-8coCazTfCQXVjHZLRGmfLa9-vCwkIdLmkfNm-skFwgUMduZgX0Qc6xJ845YwXDVBpnz14Ok-Is3O_Co02L1Se0jJUJspjXGIWUY13u6RAd7jtWbNry-gsAsAly6NpoJjuVhEuC__tJWqTzaA8zKCnlTWliz9oMirr8FDTSxHASoExw9pHJoYja0LfeT3l_HQEk2I8Awpx07GaqzoV6KGg9zIk5Kw5Gl3vnAzlvX93FLdaIDk2Q9bvpc-5OEt7haR7HbzAUuVX9IrX_3cjFTwiUqUeWKu4cFvabnvYirqU3A1aCjhNwUhrRRhMwGvXYlbMfz5JFx32xjUlqnyzAUepf5THrHy-D8Q6UTUjluTD0IuuE6KE9t4-DTlF9cEX2DpJFnQ/https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fpull%2F35636> > (with > some comments for backward compatibility with V1 views). We have that setup > at LinkedIn (after adapting the PRs and merging them internally), and we > create/query views stored in the Hive Metastore (HMS), while the underlying > tables are either in the HMS or the Iceberg catalog. > > > > How did you create the views in the first place? Where does their > definition live? > > > > Thanks, > > Walaa. > > > > > > On Tue, Sep 26, 2023 at 12:40 AM Eduard Tudenhoefner <edu...@tabular.io> > wrote: > > Views are currently not supported by Iceberg, but are under active > development. You can check the current status of Iceberg View support here > <https://secure-web.cisco.com/1IzMhQTaiE7pvXDxGWQhl6rBKThhC3JORDd2CAs5bglmywcSL87exOblvMAgOGT-ij6ALoJxP0ovX3yoeabCBRZPohqMkiP38jy58mL9rOL5r81hWsIBSAiz2-sCLpq2PkFz17XQMv_pPLMxU0q5QlPPqZOBxtpaYorLnqa0HIhKi9ea1seaGMQHGgdszR9ZpLNKP1oWo7rHFTu7a4p1K5dgdsoXo2k9bWRHE4bBKsIcHj-uP9nSU4f0ndm7OSkBo2VOEIMLB3ofS95syqan_RxPDxLWlJImpwkpeixwYSZ6mdJAiSfdR-J5KH4ou4eLpoNWMaDMisZ-_pXRPmcMKJulg_xnYTG1838jShzrx8dcszL6jsr0dpsoN9G6EwyUl3fp86CoIdTN2YgLqUT3GHFprJBvdZPMTf82y9DDobkWTKXepAK6qzyCi1JHSQJGV6QRU0rkDVuAoiPxbkXKkUlMx1iHGQdIYW9uydEiQ9PxxgD1VtTg27-pUu5E7fZsn-eQR0HdOisyiX6YirQ_58g/https%3A%2F%2Fgithub.com%2Fapache%2Ficeberg%2Fprojects%2F29> > . > > > > Eduard > > > > On Mon, Sep 25, 2023 at 8:39 PM Agrawal, Sanket < > sankeagra...@deloitte.com.invalid> wrote: > > Hello Everyone, > > > > We have setup spark and setup everything as mentioned at > https://iceberg.apache.org/docs/latest/aws/ > <https://secure-web.cisco.com/1boIMcWOkE2owBmEs8DNd4rEzN_YUXzGWlmjG8nTZiIPbDGzjjhOEAYFPcMWEqqntDCz5UH9v_cC2IsIgmJU9WEvPR3YIoBdfKKqE1A3do3QPPkCcu0UTs_dcs2D0fjzUW9h2Lz3-vMb-8vhuCgoCimlW3gmjBqlxmluPvLlQNp8lo9L5mTe2TMhkytvEHgS2H6ePA4MJ8yijtgZUy1_ywTwXtFfS0bLxDB27txFCnu3fXXYN0Z4yPIZTiWt44LgIqTT5qGfY0pt8AzjB8Uc4X_2I1CwbU9ICeIsdMKchAeQsAW7S_4Wu6XGlEYotCW-AALcwW1_Rerq626WD1TqzxTkKo6TnuFfom3mX0RympfD4FMAVFJ6gpWf7AN9eCZ509IHgW5nLKvs9mQADhyh36huK0JhmY3FO_VkO7Ksq-COU2IZb6vqfaJuVS7v_PQP6nLwNvaF_y3Wf67BilxmSpfbEsjkU-q3fezoxvFjF-fanmil0zNNScjXGUZNATW6hc0nsIKGlnlFHC1XVRhDCwg/https%3A%2F%2Ficeberg.apache.org%2Fdocs%2Flatest%2Faws%2F> > to integrate Spark, Iceberg, and AWS Glue Catalog. We are able to read > tables through this but we are unable to read data through views. PFB, the > error: > > > > pyspark.errors.exceptions.captured.AnalysisException: > [TABLE_OR_VIEW_NOT_FOUND] The table or view `db`.`vw_name ` cannot be > found. Verify the spelling and correctness of the schema and catalog. > > If you did not qualify the name with a schema, verify the current_schema() > output, or qualify the name with the correct schema and catalog. > > To tolerate the error on drop use DROP VIEW IF EXISTS or DROP TABLE IF > EXISTS.; line 1 pos 14; > > 'GlobalLimit 1 > > +- 'LocalLimit 1 > > +- 'Project [*] > > +- 'UnresolvedRelation [db, vw_name], [], false > > > > > > Any help or guidance would be of great help. > > > > Thank You, > > Sanket A. > > This message (including any attachments) contains confidential information > intended for a specific individual and purpose, and is protected by law. If > you are not the intended recipient, you should delete this message and any > disclosure, copying, or distribution of this message, or the taking of any > action based on it, by you is strictly prohibited. > > Deloitte refers to a Deloitte member firm, one of its related entities, or > Deloitte Touche Tohmatsu Limited ("DTTL"). Each Deloitte member firm is a > separate legal entity and a member of DTTL. DTTL does not provide services > to clients. Please see www.deloitte.com/about to learn more. > > v.E.1 > >
