Hi Alex, Thanks for starting this discussion!
>From my POV, the SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION flag represents legacy use cases that are not well-defined in the current Polaris codebase. The flag can certainly be used if you know the code well, but from the end user's perspective, I believe the related behaviour changes are very obscure. I'd propose to avoid using this flag for main use cases, but keep it for backward compatibility for a few releases, aiming to eventually remove it. That said, even with subscoping "on", but STS disabled, the behaviour is still quite obscure. However, IIRC the static S3 credentials will actually be used for FileIO instances inside Polaris servers. I think it's a good idea to use StorageAccessConfig on all call paths, both internal and when vending credentials to clients. Having distinct methods like getClientAccessConfig() and getServerAccessConfig() in AccessConfigProvider sounds like a reasonable approach to implementing this. Cheers, Dmitri. On Tue, Nov 4, 2025 at 8:11 AM Alexandre Dutra <[email protected]> wrote: > Hi all, > > I'm having trouble understanding how AWS system-wide credentials work > when subscoping or STS is unavailable. My apologies if this has been > asked previously, but I would greatly appreciate it if someone could > provide an explanation. > > Currently, Polaris honors AWS system-wide credentials (set via > polaris.storage.aws.access-key and polaris.storage.aws.secret-key) > only when credentials subscoping is enabled and STS is available. > > In this scenario, Polaris generates subscoped credentials using these > static credentials for every STS request [1]. > > However, if either credentials subscoping or STS is unavailable, these > system-wide credentials are ignored, afaict. > > The core problem, in my opinion, is that the current code mixes > credentials subscoping (for the server itself) and credentials vending > (for clients). While it's appropriate not to vend these long-lived > credentials directly to clients, I believe they should be used when > the server itself needs to access the remote storage. > > This leads to issues in two specific S3 setups: > > 1) System-wide credentials with credentials subscoping OFF (STS status > is irrelevant). > > 2) System-wide credentials with credentials subscoping ON, but STS > marked unavailable. > > In both cases, the generated AccessConfig doesn't contain credentials, > which means that the FileIO instance that Polaris creates will rely on > the default provider chain for credentials. > > I wonder if the AccessConfig should instead contain the keys > s3.access-key-id and s3.secret-access-key, populated with the static, > system-wide credentials? > > If I were a user using an S3-like service without STS, I would have > expected that those static credentials would be used for FileIO > instance creation (and also, one day, for S3 remote signing). > > To address this, I was thinking of adding a dedicated method to > AccessConfigProvider. Something like getClientAccessConfig() vs > getServerAccessConfig(). The latter would return a specialized > AccessConfig for server-side usage, ensuring that if static > credentials are present, these are used whenever subscoping is off or > STS is unavailable. > > What do you all think? > > Thanks, > Alex > > [1]: > https://github.com/apache/polaris/blob/f97c5eb50016489129575aab62d5efb3efb7552e/polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L99-L100 >
