Thanks for the improvement. The syntax looks much more intuitive than in Spark, where the catalog has to be configured with the "spark.sql" prefix (even for DataFrame operations), which can be confusing—especially when working with multiple catalogs.
A few questions on the new CATALOG entity: 1. Catalog persistence — When we run "*CREATE CATALOG myRestCatalog*" with configuration options, will the catalog be stored and persisted beyond the current session? In Spark and other engines, the catalog implementation and configuration usually last only for the active session. Since we are querying external tables, I’m not sure if storing catalog details is necessary. Also, AWS roles and STS credentials expire after some time, which would require catalog updates. 2. Table referencing — How do we plan to reference tables? Will it be a three-part notation -- might be clearer when working across multiple catalogs? For example: *SELECT *FROM glue_catalog.namespace1.iceberg_table1 AINNER JOIN unity_catalog.namespace2.delta_lake_table1 B ON A.id = B.id;* 3. It looks like this APE proposes two features: 1. The new CATALOG entity 2. DQL and DDL support for Iceberg tables using various catalog implementations. Would it make sense to split these into separate APEs? Thanks Hari Kishore On Fri, Aug 8, 2025 at 9:39 AM Hussain Towaileb <hussai...@gmail.com> wrote: > Initiating discussion for adding improved support for Apache Iceberg > Feature: *Improved Support for Apache Iceberg* > Details: Apache Iceberg would provide support for reading Iceberg tables. > This APE discusses adding improved support to the current Apache Iceberg > support by introducing the Catalog entity to AsterixDB Metadata, adding > support to different types of Iceberg catalogs, and introducing other > features like time travel. > > APE: > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/ASTERIXDB/APE*25*3A*Apache*Iceberg*Support__;KyUrKys!!CzAuKJ42GuquVTTmVmPViYEvSg!J81u58s9FyWRyedF8qV0TL-QjZrvS9vCviVHuCte1wGJ-y3qgzG087UwfC-ii0LKkEI3c5Iw7CG6yxA62A$ > > -- > Regards, > Hussain Towaileb >
