Hi all, Following up on the earlier discussion around file-level access delegation in the REST Catalog spec, I put together a small POC PR against my personal Iceberg fork to explore the remote signing + caching direction we discussed:
https://github.com/williamhyun/iceberg/pull/198 The goal is to make the tradeoffs more concrete and evaluate whether this could be a practical approach for finer-grained delegated access. Feedback is very welcome. Best, William On Thu, Jul 2, 2026 at 5:29 PM William Hyun <[email protected]> wrote: > Hi all, > > Thank you again for the discussion during the 7/1 catalog community sync. > I wanted to summarize two alternative directions that came up to > pre-signed URLs: > 1. scoped credential vending > 2. remote signing with caching and bulk-signing > > I think scoped credential vending is worth considering in cases where the > authorization boundary maps cleanly to a small number of stable storage > prefixes. > Scoped credential vending seems less practical when access needs to be > granted for many specific files rather than a small number of prefixes. > > In AWS, session policies are passed as parameters during AssumeRole-style > issuance when the temporary session is created. [1] > This means that scoping down an already-issued credential cannot be done > locally by simply altering the existing token. > Instead, it requires minting another STS session whose policy encodes the > new scope. > This turns authorization into an STS issuance problem with limitations on > STS request frequency [2] and policy size. [3] > > By contrast, pre-signed URLs and remote signing can authorize exact object > requests using already-held credentials, without creating a new STS session > for each file set. > Based on that discussion, I am exploring a POC around bulk remote > signing + caching to evaluate whether it could be a practical alternative > to pre-signed URLs for these finer-grained access patterns. > > Best, > William > > [1] https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html > [2] > https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-quotas.html#reference_iam-quotas-sts-requests > [3] > https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html > > On Tue, Jun 30, 2026 at 7:43 PM William Hyun <[email protected]> wrote: > >> Hi everyone, >> >> I'd like to open a discussion on an extension to the REST Catalog spec to >> enable file-level access delegation during scan planning. >> Today, delegated access is table-scoped, which can force >> over-provisioning or table fragmentation when consumers should only see a >> subset of partitions or records. >> This proposal adds a path using pre-signed URLs to make partition-scoped >> sharing practical in the REST Catalog model. >> >> Here is my proposal: https://s.apache.org/n16st >> >> I look forward to hearing your thoughts. >> >> Relevant discussion: >> - https://lists.apache.org/thread/ko9kp0gvzhx85n7cvoxqnpw4vwnhmdg6 >> >> Best, >> William >> >>
