Hi Dmitri, Thanks for the clarification.
I am going to experiment more with the idea of distinguishing client and server AccessConfigs, and if that goes well, I will open a PR. Thanks, Alex On Wed, Nov 5, 2025 at 6:50 PM Dmitri Bourlatchkov <[email protected]> wrote: > > > with subscoping "on", but STS disabled [...] the static S3 credentials > will actually be used for FileIO instances inside Polaris servers. > > That's not the behavior I'm observing. Common properties like region, > endpoint and path-style access do get included in the AccessConfig, > but no credential is present [1]. > > > Oh, I agree that credentials are not in AccessConfig in this case. I meant > that FileIO instances will use static credentials from the AWS profile or > env. vars in this case. Sorry about the confusion. > > This is what I leveraged in the MinIO example [1]. > > [1] > https://github.com/apache/polaris/blob/main/getting-started/minio/docker-compose.yml#L57 > > Cheers, > Dmitri. > > On Wed, Nov 5, 2025 at 12:21 PM Alexandre Dutra <[email protected]> wrote: > > > Hi Dmitri, > > > > > SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION flag represents legacy use cases > > that are not well-defined in the current Polaris codebase. > > > > Thanks for confirming, that was my impression as well. I tried to set > > this flag on, but Polaris started failing in many surprising ways. > > > > > with subscoping "on", but STS disabled [...] the static S3 credentials > > will actually be used for FileIO instances inside Polaris servers. > > > > That's not the behavior I'm observing. Common properties like region, > > endpoint and path-style access do get included in the AccessConfig, > > but no credential is present [1]. > > > > Thanks, > > Alex > > > > [1]: > > https://github.com/apache/polaris/blob/f33646b564c813979a8a7f0d96d09dd12fd569f5/polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L124 > > > > > > On Tue, Nov 4, 2025 at 7:38 PM Dmitri Bourlatchkov <[email protected]> > > wrote: > > > > > > Hi Alex, > > > > > > Thanks for starting this discussion! > > > > > > From my POV, the SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION flag represents > > > legacy use cases that are not well-defined in the current Polaris > > > codebase. The flag can certainly be used if you know the code well, but > > > from the end user's perspective, I believe the related behaviour changes > > > are very obscure. I'd propose to avoid using this flag for main use > > cases, > > > but keep it for backward compatibility for a few releases, aiming to > > > eventually remove it. > > > > > > That said, even with subscoping "on", but STS disabled, the behaviour is > > > still quite obscure. However, IIRC the static S3 credentials will > > actually > > > be used for FileIO instances inside Polaris servers. > > > > > > I think it's a good idea to use StorageAccessConfig on all call paths, > > both > > > internal and when vending credentials to clients. > > > > > > Having distinct methods like getClientAccessConfig() > > > and getServerAccessConfig() in AccessConfigProvider sounds like a > > > reasonable approach to implementing this. > > > > > > Cheers, > > > Dmitri. > > > > > > On Tue, Nov 4, 2025 at 8:11 AM Alexandre Dutra <[email protected]> > > wrote: > > > > > > > Hi all, > > > > > > > > I'm having trouble understanding how AWS system-wide credentials work > > > > when subscoping or STS is unavailable. My apologies if this has been > > > > asked previously, but I would greatly appreciate it if someone could > > > > provide an explanation. > > > > > > > > Currently, Polaris honors AWS system-wide credentials (set via > > > > polaris.storage.aws.access-key and polaris.storage.aws.secret-key) > > > > only when credentials subscoping is enabled and STS is available. > > > > > > > > In this scenario, Polaris generates subscoped credentials using these > > > > static credentials for every STS request [1]. > > > > > > > > However, if either credentials subscoping or STS is unavailable, these > > > > system-wide credentials are ignored, afaict. > > > > > > > > The core problem, in my opinion, is that the current code mixes > > > > credentials subscoping (for the server itself) and credentials vending > > > > (for clients). While it's appropriate not to vend these long-lived > > > > credentials directly to clients, I believe they should be used when > > > > the server itself needs to access the remote storage. > > > > > > > > This leads to issues in two specific S3 setups: > > > > > > > > 1) System-wide credentials with credentials subscoping OFF (STS status > > > > is irrelevant). > > > > > > > > 2) System-wide credentials with credentials subscoping ON, but STS > > > > marked unavailable. > > > > > > > > In both cases, the generated AccessConfig doesn't contain credentials, > > > > which means that the FileIO instance that Polaris creates will rely on > > > > the default provider chain for credentials. > > > > > > > > I wonder if the AccessConfig should instead contain the keys > > > > s3.access-key-id and s3.secret-access-key, populated with the static, > > > > system-wide credentials? > > > > > > > > If I were a user using an S3-like service without STS, I would have > > > > expected that those static credentials would be used for FileIO > > > > instance creation (and also, one day, for S3 remote signing). > > > > > > > > To address this, I was thinking of adding a dedicated method to > > > > AccessConfigProvider. Something like getClientAccessConfig() vs > > > > getServerAccessConfig(). The latter would return a specialized > > > > AccessConfig for server-side usage, ensuring that if static > > > > credentials are present, these are used whenever subscoping is off or > > > > STS is unavailable. > > > > > > > > What do you all think? > > > > > > > > Thanks, > > > > Alex > > > > > > > > [1]: > > > > > > https://github.com/apache/polaris/blob/f97c5eb50016489129575aab62d5efb3efb7552e/polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L99-L100 > > > > > >
