This is an automated email from the ASF dual-hosted git repository.
roryqi pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new 026a660350 [#13927] fix(credential): handle AWS IRSA fallback and ADLS
paths (#13928)
026a660350 is described below
commit 026a660350666bb740c4f53c1b34b8e2707b2d88
Author: roryqi <[email protected]>
AuthorDate: Mon Oct 19 22:40:25 2026 +0800
[#13927] fix(credential): handle AWS IRSA fallback and ADLS paths (#13928)
### What changes were proposed in this pull request?
- Allow the AWS IRSA credential generator to initialize without
`s3-role-arn`
and fall back to `AWS_ROLE_ARN`, including when the configured value is
blank.
- Reduce equivalent ADLS locations and child paths to the one requested
parent
location, then generate one directory-scoped SAS.
- Reject ADLS paths that do not resolve to one requested root location,
or
belong to different storage accounts or containers.
- Add unit tests for the AWS fallback and ADLS multi-path behavior.
- Update the credential-vending documentation to describe AWS fallback
and
ADLS path scope accurately.
### Why are the changes needed?
AWS IRSA currently validates `s3-role-arn` before its
environment-variable
fallback can run. ADLS credential vending rejects Iceberg tables that
define
`write.data.path` or `write.metadata.path` because it requires exactly
one
unique path.
Fix: #13927
### Does this PR introduce _any_ user-facing change?
Yes.
- `aws-irsa` can use the pod's `AWS_ROLE_ARN` when `s3-role-arn` is
unset
or blank. If neither ARN is available, the error occurs when credentials
are
first requested because the environment fallback is evaluated then.
- Path-based IRSA requests remain scoped by a session policy;
catalog-level
requests do not include one.
- `adls-token` supports multiple table paths when one of the requested
paths
is their parent and can be covered by one directory-scoped SAS.
- Equivalent ADLS spellings that differ by `abfs`/`abfss`, account-name
case, or trailing slashes are treated as one location.
- ADLS paths without one requested root, or from different accounts or
containers, produce a clearer error.
No property keys or public APIs are added or removed.
### How was this patch tested?
```shell
./gradlew :bundles:aws:test :bundles:azure:test :docs:build
```
---
.../s3/credential/AwsIrsaCredentialGenerator.java | 18 +-
.../credential/TestAwsIrsaCredentialGenerator.java | 56 ++++++
.../abs/credential/ADLSTokenGenerator.java | 111 +++++++++---
.../abs/credential/TestADLSTokenGenerator.java | 193 +++++++++++++++++++++
docs/security/credential-vending.md | 18 +-
5 files changed, 360 insertions(+), 36 deletions(-)
diff --git
a/bundles/aws/src/main/java/org/apache/gravitino/s3/credential/AwsIrsaCredentialGenerator.java
b/bundles/aws/src/main/java/org/apache/gravitino/s3/credential/AwsIrsaCredentialGenerator.java
index 2d5fe0f6ba..2026afbb56 100644
---
a/bundles/aws/src/main/java/org/apache/gravitino/s3/credential/AwsIrsaCredentialGenerator.java
+++
b/bundles/aws/src/main/java/org/apache/gravitino/s3/credential/AwsIrsaCredentialGenerator.java
@@ -28,6 +28,7 @@ import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
+import java.util.function.Supplier;
import java.util.stream.Stream;
import org.apache.commons.lang3.StringUtils;
import org.apache.gravitino.credential.AwsIrsaCredential;
@@ -55,6 +56,7 @@ import software.amazon.awssdk.services.sts.model.Credentials;
/** Generate AWS IRSA credentials according to the read and write paths. */
public class AwsIrsaCredentialGenerator implements
CredentialGenerator<AwsIrsaCredential> {
+ private final Supplier<String> environmentRoleArnSupplier;
private WebIdentityTokenSource tokenSource;
private String roleArn;
private int tokenExpireSecs;
@@ -64,12 +66,24 @@ public class AwsIrsaCredentialGenerator implements
CredentialGenerator<AwsIrsaCr
private StsClient basicModeStsClient;
private Map<String, StsAssumeRoleWithWebIdentityCredentialsProvider>
basicModeProviders;
+ /** Creates an AWS IRSA credential generator. */
+ public AwsIrsaCredentialGenerator() {
+ this(() -> System.getenv("AWS_ROLE_ARN"));
+ }
+
+ AwsIrsaCredentialGenerator(Supplier<String> environmentRoleArnSupplier) {
+ this.environmentRoleArnSupplier =
Objects.requireNonNull(environmentRoleArnSupplier);
+ }
+
@Override
public void initialize(Map<String, String> properties) {
this.tokenSource = WebIdentityTokenSources.create(properties);
S3CredentialConfig s3CredentialConfig = new S3CredentialConfig(properties);
- this.roleArn = s3CredentialConfig.s3RoleArn();
+ this.roleArn =
+
StringUtils.isNotBlank(properties.get(S3CredentialConfig.S3_ROLE_ARN.getKey()))
+ ? s3CredentialConfig.s3RoleArn()
+ : null;
this.tokenExpireSecs = s3CredentialConfig.tokenExpireInSecs();
this.region = s3CredentialConfig.region();
this.stsEndpoint = s3CredentialConfig.stsEndpoint();
@@ -330,7 +344,7 @@ public class AwsIrsaCredentialGenerator implements
CredentialGenerator<AwsIrsaCr
private String getValidatedRoleArn(String configRoleArn) {
String effectiveRoleArn =
- StringUtils.isNotBlank(configRoleArn) ? configRoleArn :
System.getenv("AWS_ROLE_ARN");
+ StringUtils.isNotBlank(configRoleArn) ? configRoleArn :
environmentRoleArnSupplier.get();
if (StringUtils.isBlank(effectiveRoleArn)) {
throw new IllegalStateException(
"No role ARN available. Either configure s3-role-arn or ensure
AWS_ROLE_ARN environment variable is set.");
diff --git
a/bundles/aws/src/test/java/org/apache/gravitino/s3/credential/TestAwsIrsaCredentialGenerator.java
b/bundles/aws/src/test/java/org/apache/gravitino/s3/credential/TestAwsIrsaCredentialGenerator.java
index f8c48b5c0d..ca261f02d8 100644
---
a/bundles/aws/src/test/java/org/apache/gravitino/s3/credential/TestAwsIrsaCredentialGenerator.java
+++
b/bundles/aws/src/test/java/org/apache/gravitino/s3/credential/TestAwsIrsaCredentialGenerator.java
@@ -84,6 +84,62 @@ class TestAwsIrsaCredentialGenerator {
assertFalse(body.contains("Policy="));
}
+ @Test
+ void
basicModeFallsBackToEnvironmentRoleArnWhenConfiguredRoleIsBlank(@TempDir Path
dir)
+ throws IOException {
+ Map<String, String> properties = createProperties(dir);
+ properties.put("s3-role-arn", " ");
+
+ try (AwsIrsaCredentialGenerator generator =
+ new AwsIrsaCredentialGenerator(() ->
"arn:aws:iam::123456789012:role/environment-role")) {
+ generator.initialize(properties);
+
+ AwsIrsaCredential credential = generator.generate(() -> "test-user");
+
+ assertEquals("access-key", credential.accessKeyId());
+ }
+
+ String body = URLDecoder.decode(requestBody.get(),
StandardCharsets.UTF_8.name());
+
assertTrue(body.contains("RoleArn=arn:aws:iam::123456789012:role/environment-role"));
+ }
+
+ @Test
+ void pathModeFallsBackToEnvironmentRoleArn(@TempDir Path dir) throws
IOException {
+ Map<String, String> properties = createProperties(dir);
+ properties.remove("s3-role-arn");
+
+ try (AwsIrsaCredentialGenerator generator =
+ new AwsIrsaCredentialGenerator(() ->
"arn:aws:iam::123456789012:role/environment-role")) {
+ generator.initialize(properties);
+ PathBasedCredentialContext context =
+ new PathBasedCredentialContext(
+ "test-user", Collections.singleton("s3://bucket/table"),
Collections.emptySet());
+
+ AwsIrsaCredential credential = generator.generate(context);
+
+ assertEquals("access-key", credential.accessKeyId());
+ }
+
+ String body = URLDecoder.decode(requestBody.get(),
StandardCharsets.UTF_8.name());
+
assertTrue(body.contains("RoleArn=arn:aws:iam::123456789012:role/environment-role"));
+ assertTrue(body.contains("Policy="));
+ }
+
+ @Test
+ void basicModeFailsWhenNoRoleArnIsAvailable(@TempDir Path dir) throws
IOException {
+ Map<String, String> properties = createProperties(dir);
+ properties.remove("s3-role-arn");
+
+ try (AwsIrsaCredentialGenerator generator = new
AwsIrsaCredentialGenerator(() -> null)) {
+ generator.initialize(properties);
+
+ IllegalStateException exception =
+ assertThrows(IllegalStateException.class, () ->
generator.generate(() -> "test-user"));
+
+ assertTrue(exception.getMessage().contains("No role ARN available"));
+ }
+ }
+
@Test
void basicModeReusesCachedStsCredentials(@TempDir Path dir) throws
IOException {
try (AwsIrsaCredentialGenerator generator = new
AwsIrsaCredentialGenerator()) {
diff --git
a/bundles/azure/src/main/java/org/apache/gravitino/abs/credential/ADLSTokenGenerator.java
b/bundles/azure/src/main/java/org/apache/gravitino/abs/credential/ADLSTokenGenerator.java
index f3cbb5b1fe..fbc69c343a 100644
---
a/bundles/azure/src/main/java/org/apache/gravitino/abs/credential/ADLSTokenGenerator.java
+++
b/bundles/azure/src/main/java/org/apache/gravitino/abs/credential/ADLSTokenGenerator.java
@@ -29,9 +29,13 @@ import
com.azure.storage.file.datalake.sas.DataLakeServiceSasSignatureValues;
import com.azure.storage.file.datalake.sas.PathSasPermission;
import java.io.IOException;
import java.time.OffsetDateTime;
-import java.util.HashSet;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
import java.util.Map;
import java.util.Set;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
import org.apache.gravitino.credential.ADLSTokenCredential;
import org.apache.gravitino.credential.CredentialContext;
import org.apache.gravitino.credential.CredentialGenerator;
@@ -70,34 +74,14 @@ public class ADLSTokenGenerator implements
CredentialGenerator<ADLSTokenCredenti
Set<String> writePaths = pathContext.getWritePaths();
Set<String> readPaths = pathContext.getReadPaths();
- Set<String> combinedPaths = new HashSet<>(writePaths);
+ Set<String> combinedPaths = new TreeSet<>(writePaths);
combinedPaths.addAll(readPaths);
- if (combinedPaths.size() != 1) {
- throw new IllegalArgumentException(
- "ADLS should contain exactly one unique path, but found: "
- + combinedPaths.size()
- + " paths: "
- + combinedPaths);
- }
- String uniquePath = combinedPaths.iterator().next();
-
- ClientSecretCredential clientSecretCredential =
- new ClientSecretCredentialBuilder()
- .tenantId(tenantId)
- .clientId(clientId)
- .clientSecret(clientSecret)
- .build();
-
- DataLakeServiceClient dataLakeServiceClient =
- new DataLakeServiceClientBuilder()
- .endpoint(endpoint)
- .credential(clientSecretCredential)
- .buildClient();
+ ADLSLocationUtils.ADLSLocationParts locationParts =
findCredentialLocation(combinedPaths);
OffsetDateTime start = OffsetDateTime.now();
OffsetDateTime expiry = OffsetDateTime.now().plusSeconds(tokenExpireSecs);
- UserDelegationKey userDelegationKey =
dataLakeServiceClient.getUserDelegationKey(start, expiry);
+ UserDelegationKey userDelegationKey = getUserDelegationKey(start, expiry);
PathSasPermission pathSasPermission =
new
PathSasPermission().setReadPermission(true).setListPermission(true);
@@ -111,7 +95,6 @@ public class ADLSTokenGenerator implements
CredentialGenerator<ADLSTokenCredenti
DataLakeServiceSasSignatureValues signatureValues =
new DataLakeServiceSasSignatureValues(expiry, pathSasPermission);
- ADLSLocationUtils.ADLSLocationParts locationParts =
ADLSLocationUtils.parseLocation(uniquePath);
String sasToken =
new DataLakeSasImplUtil(
signatureValues,
@@ -125,6 +108,84 @@ public class ADLSTokenGenerator implements
CredentialGenerator<ADLSTokenCredenti
locationParts.getAccountName(), sasToken,
expiry.toInstant().toEpochMilli());
}
+ UserDelegationKey getUserDelegationKey(OffsetDateTime start, OffsetDateTime
expiry) {
+ ClientSecretCredential clientSecretCredential =
+ new ClientSecretCredentialBuilder()
+ .tenantId(tenantId)
+ .clientId(clientId)
+ .clientSecret(clientSecret)
+ .build();
+
+ DataLakeServiceClient dataLakeServiceClient =
+ new DataLakeServiceClientBuilder()
+ .endpoint(endpoint)
+ .credential(clientSecretCredential)
+ .buildClient();
+ return dataLakeServiceClient.getUserDelegationKey(start, expiry);
+ }
+
+ private ADLSLocationUtils.ADLSLocationParts
findCredentialLocation(Set<String> paths) {
+ if (paths.isEmpty()) {
+ throw new IllegalArgumentException(
+ "ADLS credential context should contain at least one path");
+ }
+
+ List<ADLSLocationUtils.ADLSLocationParts> parsedLocations =
+
paths.stream().map(ADLSLocationUtils::parseLocation).collect(Collectors.toList());
+ ADLSLocationUtils.ADLSLocationParts first = parsedLocations.get(0);
+ boolean hasDifferentScope =
+ parsedLocations.stream()
+ .anyMatch(
+ location ->
+
!storageAccountName.equalsIgnoreCase(location.getAccountName())
+ ||
!first.getAccountName().equalsIgnoreCase(location.getAccountName())
+ ||
!first.getContainer().equalsIgnoreCase(location.getContainer()));
+ if (hasDifferentScope) {
+ throw new IllegalArgumentException(
+ "ADLS paths must belong to the configured storage account and a
single container, but found: "
+ + paths);
+ }
+
+ Map<String, ADLSLocationUtils.ADLSLocationParts> uniqueLocations = new
LinkedHashMap<>();
+ parsedLocations.forEach(
+ location ->
+ uniqueLocations.putIfAbsent(
+ location.getContainer() + "\0" +
ADLSLocationUtils.trimSlashes(location.getPath()),
+ location));
+ List<ADLSLocationUtils.ADLSLocationParts> locations = new
ArrayList<>(uniqueLocations.values());
+
+ List<ADLSLocationUtils.ADLSLocationParts> rootLocations =
+ locations.stream()
+ .filter(
+ location ->
+ locations.stream()
+ .noneMatch(
+ possibleParent ->
+ possibleParent != location
+ && isDescendantOf(location,
possibleParent)))
+ .collect(Collectors.toList());
+ if (rootLocations.size() != 1) {
+ throw new IllegalArgumentException(
+ "ADLS credential paths must resolve to exactly one root location,
but found: "
+ + rootLocations.size()
+ + " roots in paths: "
+ + paths);
+ }
+
+ return rootLocations.get(0);
+ }
+
+ private static boolean isDescendantOf(
+ ADLSLocationUtils.ADLSLocationParts location,
+ ADLSLocationUtils.ADLSLocationParts possibleParent) {
+ String path = ADLSLocationUtils.trimSlashes(location.getPath());
+ String parentPath =
ADLSLocationUtils.trimSlashes(possibleParent.getPath());
+ if (parentPath.isEmpty()) {
+ return !path.isEmpty();
+ }
+ return path.startsWith(parentPath + "/");
+ }
+
@Override
public void close() throws IOException {}
}
diff --git
a/bundles/azure/src/test/java/org/apache/gravitino/abs/credential/TestADLSTokenGenerator.java
b/bundles/azure/src/test/java/org/apache/gravitino/abs/credential/TestADLSTokenGenerator.java
new file mode 100644
index 0000000000..98bd98c56d
--- /dev/null
+++
b/bundles/azure/src/test/java/org/apache/gravitino/abs/credential/TestADLSTokenGenerator.java
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.abs.credential;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import com.azure.storage.file.datalake.models.UserDelegationKey;
+import java.time.OffsetDateTime;
+import java.util.Arrays;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import org.apache.gravitino.credential.ADLSTokenCredential;
+import org.apache.gravitino.credential.PathBasedCredentialContext;
+import org.apache.gravitino.storage.AzureProperties;
+import org.junit.jupiter.api.Test;
+
+class TestADLSTokenGenerator {
+
+ @Test
+ void generatesTokenForParentDirectoryOfMultiplePaths() throws Exception {
+ try (ADLSTokenGenerator generator = new TestableADLSTokenGenerator()) {
+ generator.initialize(createProperties());
+ PathBasedCredentialContext context =
+ new PathBasedCredentialContext(
+ "test-user",
+ Collections.emptySet(),
+ new HashSet<>(
+ Arrays.asList(
+
"abfss://[email protected]/warehouse/table",
+
"abfss://[email protected]/warehouse/table/data",
+
"abfss://[email protected]/warehouse/table/metadata")));
+
+ ADLSTokenCredential credential = generator.generate(context);
+
+ assertEquals("account", credential.accountName());
+ assertTrue(credential.sasToken().contains("sr=d"));
+ assertTrue(credential.sasToken().contains("sdd=2"));
+ }
+ }
+
+ @Test
+ void treatsEquivalentLocationsAsOneRoot() throws Exception {
+ try (ADLSTokenGenerator generator = new TestableADLSTokenGenerator()) {
+ generator.initialize(createProperties());
+ PathBasedCredentialContext context =
+ new PathBasedCredentialContext(
+ "test-user",
+ Collections.emptySet(),
+ new HashSet<>(
+ Arrays.asList(
+
"abfss://[email protected]/warehouse/table",
+
"abfs://[email protected]/warehouse/table/",
+
"abfss://[email protected]/warehouse/table/data")));
+
+ ADLSTokenCredential credential = generator.generate(context);
+
+ assertTrue(credential.sasToken().contains("sr=d"));
+ assertTrue(credential.sasToken().contains("sdd=2"));
+ }
+ }
+
+ @Test
+ void rejectsPathsWithoutProvidedCommonRoot() throws Exception {
+ String first =
"abfss://[email protected]/warehouse/table/data";
+ String second =
"abfss://[email protected]/warehouse/table/metadata";
+ try (ADLSTokenGenerator generator = new TestableADLSTokenGenerator()) {
+ generator.initialize(createProperties());
+ PathBasedCredentialContext context =
+ new PathBasedCredentialContext(
+ "test-user", Collections.emptySet(), new
HashSet<>(Arrays.asList(first, second)));
+
+ IllegalArgumentException exception =
+ assertThrows(IllegalArgumentException.class, () ->
generator.generate(context));
+
+ assertTrue(exception.getMessage().contains("exactly one root location"));
+ assertTrue(exception.getMessage().contains(first));
+ assertTrue(exception.getMessage().contains(second));
+ }
+ }
+
+ @Test
+ void rejectsSiblingPathsWithCommonStringPrefix() throws Exception {
+ String first =
"abfss://[email protected]/warehouse/table";
+ String second =
"abfss://[email protected]/warehouse/table2";
+ try (ADLSTokenGenerator generator = new TestableADLSTokenGenerator()) {
+ generator.initialize(createProperties());
+ PathBasedCredentialContext context =
+ new PathBasedCredentialContext(
+ "test-user", Collections.emptySet(), new
HashSet<>(Arrays.asList(first, second)));
+
+ IllegalArgumentException exception =
+ assertThrows(IllegalArgumentException.class, () ->
generator.generate(context));
+
+ assertTrue(exception.getMessage().contains("exactly one root location"));
+ }
+ }
+
+ @Test
+ void rejectsPathsFromDifferentContainers() throws Exception {
+ String first =
"abfss://[email protected]/warehouse/table";
+ String second =
"abfss://[email protected]/warehouse/table/data";
+ try (ADLSTokenGenerator generator = new TestableADLSTokenGenerator()) {
+ generator.initialize(createProperties());
+ PathBasedCredentialContext context =
+ new PathBasedCredentialContext(
+ "test-user", Collections.emptySet(), new
HashSet<>(Arrays.asList(first, second)));
+
+ IllegalArgumentException exception =
+ assertThrows(IllegalArgumentException.class, () ->
generator.generate(context));
+
+ assertTrue(exception.getMessage().contains(first));
+ assertTrue(exception.getMessage().contains(second));
+ }
+ }
+
+ @Test
+ void rejectsPathsFromDifferentStorageAccounts() throws Exception {
+ String first =
"abfss://[email protected]/warehouse/table";
+ String second =
"abfss://[email protected]/warehouse/table/data";
+ try (ADLSTokenGenerator generator = new TestableADLSTokenGenerator()) {
+ generator.initialize(createProperties());
+ PathBasedCredentialContext context =
+ new PathBasedCredentialContext(
+ "test-user", Collections.emptySet(), new
HashSet<>(Arrays.asList(first, second)));
+
+ IllegalArgumentException exception =
+ assertThrows(IllegalArgumentException.class, () ->
generator.generate(context));
+
+ assertTrue(exception.getMessage().contains(first));
+ assertTrue(exception.getMessage().contains(second));
+ }
+ }
+
+ @Test
+ void rejectsSinglePathFromUnconfiguredStorageAccount() throws Exception {
+ String path =
"abfss://[email protected]/warehouse/table";
+ try (ADLSTokenGenerator generator = new TestableADLSTokenGenerator()) {
+ generator.initialize(createProperties());
+ PathBasedCredentialContext context =
+ new PathBasedCredentialContext(
+ "test-user", Collections.emptySet(),
Collections.singleton(path));
+
+ IllegalArgumentException exception =
+ assertThrows(IllegalArgumentException.class, () ->
generator.generate(context));
+
+ assertTrue(exception.getMessage().contains(path));
+ }
+ }
+
+ private static Map<String, String> createProperties() {
+ Map<String, String> properties = new HashMap<>();
+ properties.put(AzureProperties.GRAVITINO_AZURE_STORAGE_ACCOUNT_NAME,
"account");
+ properties.put(AzureProperties.GRAVITINO_AZURE_TENANT_ID, "tenant");
+ properties.put(AzureProperties.GRAVITINO_AZURE_CLIENT_ID, "client");
+ properties.put(AzureProperties.GRAVITINO_AZURE_CLIENT_SECRET, "secret");
+ return properties;
+ }
+
+ private static class TestableADLSTokenGenerator extends ADLSTokenGenerator {
+ @Override
+ UserDelegationKey getUserDelegationKey(OffsetDateTime start,
OffsetDateTime expiry) {
+ return new UserDelegationKey()
+ .setSignedObjectId("object")
+ .setSignedTenantId("tenant")
+ .setSignedStart(start)
+ .setSignedExpiry(expiry)
+ .setSignedService("b")
+ .setSignedVersion("2020-02-10")
+ .setValue(Base64.getEncoder().encodeToString(new byte[32]));
+ }
+ }
+}
diff --git a/docs/security/credential-vending.md
b/docs/security/credential-vending.md
index 0db656e06d..b323ab3587 100755
--- a/docs/security/credential-vending.md
+++ b/docs/security/credential-vending.md
@@ -220,16 +220,16 @@ The vended credentials inherit this policy, narrowed to
the table path. Without
For Gravitino running on EKS. Instead of an access key, Gravitino uses its
pod's IAM role to call AssumeRole, so no static keys exist anywhere in the
setup.
-| Property | Description
| Default value | Required
|
-|-----------------------------|------------------------------------------------------------------------------------------------|---------------|-----------------------------|
-| `s3-role-arn` | ARN of the role to assume, in the form
`arn:aws:iam::{account_id}:role/{role_name}`. | (none) | For
path-scoped credentials |
-| `s3-region` | AWS region for STS operations.
| (none) | No
|
-| `s3-token-expire-in-secs` | Session lifetime of the vended credentials.
Cannot exceed the role's maximum session duration. | 3600 | No
|
-| `s3-token-service-endpoint` | Alternative STS endpoint, for S3-compatible
storage. | (none) | No
|
+| Property | Description
| Default value | Required |
+|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------|---------------|----------|
+| `s3-role-arn` | ARN of the role to assume. When omitted,
Gravitino uses the pod's `AWS_ROLE_ARN`.
| (none) | No |
+| `s3-region` | AWS region for STS operations.
| (none) | No |
+| `s3-token-expire-in-secs` | Session lifetime of the vended credentials.
Cannot exceed the role's maximum session duration.
| 3600 | No |
+| `s3-token-service-endpoint` | Alternative STS endpoint, for S3-compatible
storage.
| (none) | No |
-Set `s3-role-arn` to get credentials scoped to the table path, with an IAM
policy generated per table covering its data, metadata, and write locations.
Without it, the vended credentials carry the full permissions of the pod's role.
+For path-based requests, Gravitino always scopes the credentials with a
session policy covering the requested data, metadata, and write locations.
Catalog-level requests do not include a session policy and inherit the selected
role's permissions. The role is selected from `s3-role-arn` when configured, or
from the pod's `AWS_ROLE_ARN` otherwise.
-The role in `s3-role-arn` needs the same two policies as `s3-token` above,
except the trust policy names the pod's IAM role rather than an IAM user.
+The selected role needs the same permission policy as `s3-token` above. Its
trust policy must allow `sts:AssumeRoleWithWebIdentity` from the pod's
Kubernetes service account identity.
IRSA itself must already be configured on the pod's Kubernetes service
account. When it is, EKS injects a signed service account token into the pod
and sets `AWS_WEB_IDENTITY_TOKEN_FILE` to its path, which is what the AWS SDK
uses to obtain credentials. If vending fails, check that this variable is
present in the pod.
@@ -392,7 +392,7 @@ The key is long-lived, carries whatever permissions its CAM
user has, and is not
### `adls-token`
-Gravitino requests an Azure [user delegation
SAS](https://learn.microsoft.com/en-us/rest/api/storageservices/create-user-delegation-sas)
and returns it to the client, scoped to the table path.
+Gravitino requests an Azure [user delegation
SAS](https://learn.microsoft.com/en-us/rest/api/storageservices/create-user-delegation-sas)
and returns it to the client. The SAS is scoped to the single requested root
directory that covers the table location and any requested write paths. This is
normally the table path, but a custom write path that is its parent results in
a broader SAS scope. Requests with independent root paths cannot be represented
by one directory-scoped SAS and are rejected.
Azure grants access through role assignments rather than policy documents. The
Microsoft Entra ID service principal identified by `azure-tenant-id`,
`azure-client-id`, and `azure-client-secret` needs two roles:
