Re: Iceberg View Spec Improvements

Hi Steven, Assumption 1 in "Portable SQL table identifiers" states: *All engines resolve a fully specified SQL identifier x.y.z to the same storage table identifier b.c in the same catalog a.* I think this assumption encodes the 4th assumption you shared. Assuming "x.y.z" resolves to "b.c" in st

Re: Iceberg View Spec Improvements

Walaa, it doesn't seem to me that the doc captured Russel's idea. there could be a new assumption 4. If the catalog name is part of the table identifier, it should be consistent across engines. catalog federation can achieve the normalization/standardization of the catalog names On Tue, Oct 8, 2

[VOTE] Table V3 Spec: Row Lineage

Hi Y'all! I think we are more or less in agreement on adding Row Lineage to the spec apart from a few details which may change a bit during implementation. Because of this, I'd like to call for an overall vote on whether or not Row-Lineage as described in PR 11130

Re: Iceberg View Spec Improvements

Just opened Comment access to the doc. Link here again for convenience [1]. [1] https://docs.google.com/document/d/1e5orD_sBv0VlNNLZRgUtalVUllGuztnAGTtqo8J0UG8/edit Thanks, Walaa. On Tue, Oct 8, 2024 at 10:42 AM Walaa Eldin Moustafa wrote: > Thanks Steven! I think this fits in the framework o

[Discuss] Replace Hadoop Catalog Examples with JDBC Catalog in Documentation

Hi all, I wanted to bring up a suggestion regarding our current documentation. The existing examples for Iceberg often use the Hadoop catalog, as seen in: - Adding a Catalog - Spark Quickstart [1] - Adding Catalogs - Spark Getting Started [2] Since we generally advise against using Hadoop

Re: Iceberg View Spec Improvements

Thanks Steven! I think this fits in the framework of "portable table identifiers" in the doc. I have stated the assumptions that should be added to the Iceberg spec in that case (in the doc they are more abstract/generic than the version you shared). Would be great to provide your feedback on the a

Re: Iceberg View Spec Improvements

I like to follow up on Russel's suggestion of using a federated catalog for resolving the catalog name/alias problem. I think Russel's idea is that the federated catalog standardizes the catalog names (for referencing). That could solve the problem. There are two cases/ (1) single catalog: there i

[Discuss] Iceberg community maintaining the docker images

Hello everyone, Now that the test fixtures are in [1],we can create a runtime JAR for the REST catalog adapter [2] from the TCK. Following that, we can build and maintain the Docker image based on it [3]. I also envision the Iceberg community maintaining some quick-start Docker images, such as sp