LuciferYang opened a new issue, #13675: URL: https://github.com/apache/gravitino/issues/13675
### Version main branch ### Describe what's wrong `GravitinoCatalogManager` caches each `GravitinoClient` with `expireAfterAccess` and `maximumSize`, and the removal listener closes the evicted client because it owns an HTTP connection pool. That is right for a client nobody holds, but the catalogs handed out keep using it: `BaseCatalog` resolves the Gravitino catalog once in its constructor and stores it in a field for the life of the Spark catalog, then issues REST calls through it on every operation (`listTables`, `listSchemas`, `loadSchema`, `createSchema`, `alterSchema`, `dropSchema`, the table paths, and `supportsSecrets().getSecrets()`). None of those calls touch the `clients` cache, so they never refresh the entry. Once `spark.sql.gravitino.clientCacheTtlSec` elapses (3600 by default) with no new catalog lookup for that identity, or `spark.sql.gravitino.clientCacheMaxSize` (100 by default) evicts the entry, the client is closed while the Spark catalog still holds it, and the next operation goes to a shut-down connection pool. ### Error message and/or stacktrace Not captured yet. The failure surfaces as a request against a closed Apache HttpClient5 connection pool on the first catalog operation after the entry is evicted. ### How to reproduce Set `spark.sql.gravitino.clientCacheTtlSec` to a few seconds, run a statement that loads a catalog, idle past the TTL without triggering another catalog lookup, then run `SHOW TABLES` against the same Spark catalog. Alternatively set `spark.sql.gravitino.clientCacheMaxSize=1` and alternate statements between two identities in token mode. ### Additional context #13166 makes the client's `close()` idempotent, which stops the shutdown double close from breaking tests, but it does not help here: the problem is closing a client that is still referenced, not closing it twice. Two directions worth weighing: keep the client alive while a `BaseCatalog` holds it (reference counting, or pinning the entry for as long as a live catalog uses it), or have `BaseCatalog` re-resolve the client from the manager per operation so that cache access refreshes the entry. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
