Re: Catalog Questions

Hey Taher, You're right! Iceberg uses a catalog among others to maintain consistency, you can read more about it here . The choice of a catalog depends on your organization and how your setup is organized. For example, if you don't use Hive metastore

Catalog Questions

Hi All, I have a question about which catalog to be using with Iceberg for our use case. We are on Kubernetes running Spark and Minio for storage. We use spark and write data to Minio as s3 and we use something like a third party Data Catalog to write the table location and create data

RE: Iceberg catalog questions

: Iceberg catalog questions For your subsequent questions: 2. mapping namespace name to the file path is only a convention, and can be overridden at both namespace and table level. The table root path can be customized to be at any location, and we actually recommend that for cloud storage use

Re: Iceberg catalog questions

Hi Mayur, Did you try Hive Metastore local mode? In that case, you have your postgres as the catalog DB, you can use all HMS's functionalities. However, you still need to handle Table access permission(your 3rd point) by yourself. Best, Yufei `This is not a contribution` On Tue, May 11, 2021

Re: Iceberg catalog questions

For your subsequent questions: 2. mapping namespace name to the file path is only a convention, and can be overridden at both namespace and table level. The table root path can be customized to be at any location, and we actually recommend that for cloud storage use cases to reduce throttling. 3.

Re: Iceberg catalog questions

Yes there is one, but unfortunately we lost attention after some time: https://github.com/apache/iceberg/pull/1870 I think the PR is close to be merged with quite a few rounds of review already, we should add it as a milestone of 0.12. -Jack On Tue, May 11, 2021 at 11:46 AM Mayur Srivastava < ma

Iceberg catalog questions

Hi, I'm looking to use/implement a PostgreSQL based Iceberg catalog. I'm wondering if one already exists and also have a few questions. I would really appreciate any help I can get with the questions. 1. Does Iceberg have a catalog that is compatible with PostgreSQL (or any storage backen