lasdf1234 commented on PR #14300: URL: https://github.com/apache/gravitino/pull/14300#issuecomment-6468271988
### Design notes (for reviewers) This PR has two related pieces: **1. Shared Iceberg catalog-wrapper cache (metalake ↔ embedded IRC)** In aux mode, metalake `lakehouse-iceberg` used to keep a long-lived `IcebergCatalogWrapper` separate from IRC’s cache. That duplicated backends/FileIO and meant GCS tokens refreshed by IRC cache eviction never reached lakehouse ops. Now both paths go through `IcebergCatalogWrapperCache` via `CatalogWrapperManager`: - aux → borrow IRC’s manager - non-aux → `StandaloneCatalogWrapperManager` (same eviction / token policy) Lakehouse ops resolve the wrapper **per operation** (`IcebergCatalogWrapperProvider.get()`), with a single fail-fast `get()` at `initialize`. After cache eviction (e.g. GCS token near expiry), the next op rebuilds the backend; `loadCatalogBackend` semantics are unchanged, only call frequency after eviction. When `catalog_meta.current_version` changes, the cache key is `(name, version)` so alterCatalog gets a new entry; the previous wrapper stays open until leases release (`close` is idempotent). Static IRC providers skip the versioned path. **2. TMS policy expand: import only missing entities** `NearestPolicyTableEnumerator` lists live schemas/tables from `SchemaDispatcher`/`TableDispatcher`, compares against the entity store, and calls `loadSchema`/`loadTable` only for missing names (`importSchemaIfMissing` / `importTableIfMissing`). On list failure it falls back to relational meta listing. Direct table tags still use the stored id. No new user-facing config. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
