Hi folks, As Polaris expands its support for external services, such as federated catalogs and cloud storage, it needs to securely access systems like AWS S3, AWS Glue, Azure Storage, and others. This external access requires Polaris to handle credentials correctly, whether they’re long-lived credentials in self-managed deployments or temporary credentials in multi-tenant SaaS setups.
We've had several ongoing discussions about how credential handling should evolve, especially in light of the work around SigV4 Auth for catalog federation. * [PR#1191] Fix updating the storage config <https://github.com/apache/polaris/pull/1191> * [PR#1506] Spec: Add SigV4 Auth Support for Catalog Federation <https://github.com/apache/polaris/pull/1506> * [Spec] Add SigV4 Auth Support for Catalog Federation <https://lists.apache.org/thread/rlbxvw0xmzvlfm7pdh97bs3xvq7o8lmy> To frame the problem and proposed solutions, I’ve drafted a design doc: Apache Polaris Creds Management Proposal <https://docs.google.com/document/d/1MAW87DtyHWPPNIEkUCRVUKBGjhh5bPn0GbtV7fifm30/edit?usp=sharing> The proposal breaks the problem into four key areas: 1. How Polaris gets vendor-specific service identity and credentials (e.g., from server config and service context registry) 2. How Polaris surfaces service identity info to users (e.g., exposing userArn or consentUrl for trust policy setup) 3. How Polaris injects service-managed identity fields into catalog or storage configs (e.g., using entity mutators at creation time) 4. How Polaris retrieves temporary credentials to access external services (e.g., via STS, with caching support) The goal is to unify credential handling across storage and connection configs, support both SaaS and self-managed deployments, and cleanly separate user-provided config from Polaris-managed properties. Would love to hear your thoughts, feedback, and suggestions. Happy to refine based on feedback! Thanks, Rulin
