Kousuke Saruta created SPARK-57898:
--------------------------------------
Summary: Implement AwsStsCredentialProvider
Key: SPARK-57898
URL: https://issues.apache.org/jira/browse/SPARK-57898
Project: Spark
Issue Type: Sub-task
Components: Spark Core
Affects Versions: 4.3.0
Reporter: Kousuke Saruta
This sub-task is part of the umbrella
[SPARK-57703|https://issues.apache.org/jira/browse/SPARK-57703] (OIDC
Credential Propagation).
h2. Problem
No {{CredentialProvider}} implementation exists to exchange an OIDC token for
AWS STS temporary credentials.
h2. Goal
Implement the S3/STS reference provider that calls
{{AssumeRoleWithWebIdentity}} and returns S3A-compatible credentials.
h2. Scope
* Implement {{CredentialProvider}} in {{connector/credential-aws}}
* Calls {{sts:AssumeRoleWithWebIdentity}} with {{UserContext.rawToken}} and
configured role ARN
* Returns {{ServiceCredential}} with keys: {{fs.s3a.access.key}},
{{fs.s3a.secret.key}}, {{fs.s3a.session.token}}
* Configuration: {{spark.security.credentials.aws.roleArn}} (required),
{{sessionName}}, {{durationSeconds}}, {{region}}, {{stsEndpoint}}
* Works with any STS-compatible endpoint (AWS, MinIO, Ceph)
h2. Acceptance criteria
* Given a valid OIDC token and role ARN, provider returns a
{{ServiceCredential}} with S3A keys.
* Configuration for non-AWS STS endpoints (MinIO, Ceph) works correctly.
* Unit tests with mocked STS client cover success, failure, and configuration
variations.
Reference: [SPIP design document Appendix
B|https://docs.google.com/document/d/1usJKncCPMiyFUg7aIdpZ0HQsklXIHow_sU_6dfFMjN0/edit].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
