Hi all,
Iceberg REST GET /v1/config is how clients bootstrap a
catalog [1]. After #5246 [2], a 200 still returns the full
catalog properties bag; the endpoint gate is off by default.
The CHANGELOG says that default will not flip until
client-facing catalog configuration is defined [3]. I'd like
to define it.
Today a 200 puts CatalogEntity.getPropertiesAsMap() into
ConfigResponse.defaults. Overrides already carry prefix and the
namespace separator; endpoints and the optional
idempotency-key-lifetime are separate fields. Storage config,
connection config and secrets live in internal properties and
are already omitted.
What still goes out in defaults is the whole public properties
map: default-base-location, operator custom keys, Iceberg
catalog properties, and polaris.config.* catalog overrides
(drop-with-purge, unstructured locations, list pagination,
credential vending, and so on).
Management GET catalog is a separate API, gated by
CATALOG_READ_PROPERTIES. Iceberg GET /v1/config exists for REST
clients to configure the catalog and HTTP client [1]. I read
polaris.config.* as server/operator policy, not Iceberg client
configuration. Stripping keys Spark or other clients actually
consume would be a breaking change, and I'd rather not add a
second privilege (GET_CATALOG_CONFIG_PROPERTIES). One authorized
200, with a defined bag.
The plan I have in mind:
Keep default-base-location, custom keys and Iceberg catalog
properties in /v1/config defaults..
Drop polaris.config.* keys from the Iceberg response; they
stay available via Management GET catalog.
Leave overrides, endpoints and idempotency-key-lifetime as is.
Document the contract, then consider flipping
ENFORCE_CATALOG_CONFIG_AUTHORIZATION [3]..
If any polaris.config.* key is consumed by an Iceberg client
today, it should stay in defaults; please name it here.
Thoughts?
Thanks,
*Vignesh*
[1] Iceberg REST spec, GET /v1/config (defaults, then client
config, then overrides)
[2] https://github.com/apache/polaris/pull/5246
[3] https://github.com/apache/polaris/issues/5684
