FANNG1 commented on code in PR #4273:
URL: https://github.com/apache/gravitino/pull/4273#discussion_r1714750014
##########
docs/iceberg-rest-service.md:
##########
@@ -147,6 +147,51 @@ If you have a JDBC Iceberg catalog prior, you must set
`catalog-backend-name` to
You must download the corresponding JDBC driver to the
`iceberg-rest-server/libs` directory.
:::
+#### Multi catalog support
+
+Gravitino Iceberg REST server supports multiple catalogs. Users can enable it
by setting configuration entries in the style
`gravitino.iceberg-rest.catalog.<catalog name>.<param name>=<value>`.
Review Comment:
The Gravitino Iceberg REST server supports multiple catalogs and offers a
configuration-based catalog management system.
##########
docs/iceberg-rest-service.md:
##########
@@ -147,6 +147,51 @@ If you have a JDBC Iceberg catalog prior, you must set
`catalog-backend-name` to
You must download the corresponding JDBC driver to the
`iceberg-rest-server/libs` directory.
:::
+#### Multi catalog support
+
+Gravitino Iceberg REST server supports multiple catalogs. Users can enable it
by setting configuration entries in the style
`gravitino.iceberg-rest.catalog.<catalog name>.<param name>=<value>`.
+
+| Configuration item | Description
| Default value
| Required | Since Version |
+|--------------------------------------------------|------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|----------|---------------|
+| `gravitino.iceberg-rest.catalog-provider-impl` | The implementation of
IcebergTableOpsProvider defines how the Iceberg REST catalog server gets
Iceberg catalogs. |
`org.apache.gravitino.iceberg.common.ops.ConfigBasedIcebergTableOpsProvider`
| No | 0.7.0 |
+
+Using `ConfigBasedIcebergTableOpsProvider`, the default value, could be
backward compatible with configurations without catalog names. It constructs a
default catalog using those configurations for clients not setting `prefix`.
Review Comment:
When using a config-based catalog provider, you can configure the default
catalog with `gravitino.iceberg-rest.catalog.<param name>=<value>`. For
specific catalogs, use the format `gravitino.iceberg-rest.catalog.<catalog
name>.<param name>=<value>`.
##########
docs/iceberg-rest-service.md:
##########
@@ -147,6 +147,51 @@ If you have a JDBC Iceberg catalog prior, you must set
`catalog-backend-name` to
You must download the corresponding JDBC driver to the
`iceberg-rest-server/libs` directory.
:::
+#### Multi catalog support
+
+Gravitino Iceberg REST server supports multiple catalogs. Users can enable it
by setting configuration entries in the style
`gravitino.iceberg-rest.catalog.<catalog name>.<param name>=<value>`.
+
+| Configuration item | Description
| Default value
| Required | Since Version |
+|--------------------------------------------------|------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|----------|---------------|
+| `gravitino.iceberg-rest.catalog-provider-impl` | The implementation of
IcebergTableOpsProvider defines how the Iceberg REST catalog server gets
Iceberg catalogs. |
`org.apache.gravitino.iceberg.common.ops.ConfigBasedIcebergTableOpsProvider`
| No | 0.7.0 |
+
+Using `ConfigBasedIcebergTableOpsProvider`, the default value, could be
backward compatible with configurations without catalog names. It constructs a
default catalog using those configurations for clients not setting `prefix`.
+
+For instance, we can configure three different catalogs, which one is
anonymous and the others named `hive_backend` and `jdbc_backend`:
Review Comment:
For instance, you can configure three different catalogs, the default
catalog and the specific `hive_backend` and `jdbc_backend` catalogs separately.
##########
iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/IcebergConfig.java:
##########
@@ -158,6 +158,15 @@ public class IcebergConfig extends Config implements
OverwriteDefaultConfig {
.stringConf()
.create();
+ public static final ConfigEntry<String> ICEBERG_REST_CATALOG_PROVIDER =
+ new ConfigBuilder(IcebergConstants.ICEBERG_REST_CATALOG_PROVIDER)
+ .doc(
+ "The implementation of IcebergTableOpsProvider defines how the
Iceberg REST catalog server gets Iceberg catalogs.")
+ .version(ConfigConstants.VERSION_0_7_0)
+ .stringConf()
+ .createWithDefault(
+
"org.apache.gravitino.iceberg.common.ops.ConfigBasedIcebergTableOpsProvider");
Review Comment:
Could you use a simple default value like `config-base-provider`? like the
implement in `IcebergMetricsManager.loadIcebergMetricsStore()`
##########
docs/iceberg-rest-service.md:
##########
@@ -147,6 +147,51 @@ If you have a JDBC Iceberg catalog prior, you must set
`catalog-backend-name` to
You must download the corresponding JDBC driver to the
`iceberg-rest-server/libs` directory.
:::
+#### Multi catalog support
+
+Gravitino Iceberg REST server supports multiple catalogs. Users can enable it
by setting configuration entries in the style
`gravitino.iceberg-rest.catalog.<catalog name>.<param name>=<value>`.
+
+| Configuration item | Description
| Default value
| Required | Since Version |
+|--------------------------------------------------|------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|----------|---------------|
+| `gravitino.iceberg-rest.catalog-provider-impl` | The implementation of
IcebergTableOpsProvider defines how the Iceberg REST catalog server gets
Iceberg catalogs. |
`org.apache.gravitino.iceberg.common.ops.ConfigBasedIcebergTableOpsProvider`
| No | 0.7.0 |
+
+Using `ConfigBasedIcebergTableOpsProvider`, the default value, could be
backward compatible with configurations without catalog names. It constructs a
default catalog using those configurations for clients not setting `prefix`.
+
+For instance, we can configure three different catalogs, which one is
anonymous and the others named `hive_backend` and `jdbc_backend`:
+
+```text
+gravitino.iceberg-rest.catalog-backend = jdbc
+gravitino.iceberg-rest.uri = jdbc:postgresql://127.0.0.1:5432
+gravitino.iceberg-rest.warehouse =
hdfs://127.0.0.1:9000/user/hive/warehouse-postgresql
+...
+gravitino.iceberg-rest.catalog.hive_backend.catalog-backend = hive
+gravitino.iceberg-rest.catalog.hive_backend.uri = thrift://127.0.0.1:9084
+gravitino.iceberg-rest.catalog.hive_backend.warehouse =
/user/hive/warehouse-hive/
+...
+gravitino.iceberg-rest.catalog.jdbc_backend.catalog-backend = jdbc
+gravitino.iceberg-rest.catalog.jdbc_backend.uri = jdbc:mysql://127.0.0.1:3306/
+gravitino.iceberg-rest.catalog.jdbc_backend.warehouse =
hdfs://127.0.0.1:9000/user/hive/warehouse-mysql
+...
+```
+
+Then we can set `prefix` in client config to use those catalogs. Take Spark
SQL as an example:
Review Comment:
You can access different catalogs by setting the `prefix` to the specific
catalog name in the Iceberg REST client configuration. The default catalog will
be used if you do not specify a prefix. For instance, consider the case of
SparkSQL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
