I want to push back on one idea in the thread: the sentiment that a v2 loadTable should "mandate that clients must fail if there are unsupported restrictions." A v2 endpoint will mostly be adopted for the unrelated cleanups — optional location, optional snapshots, creds out of properties. If using v2 also requires implementing read-restrictions, every client that just wants those cleanups is forced to handle restrictions too. That's coupling, not opt-in, and it'll drag on v2 adoption. So read-restrictions shouldn't be the leading motivation for v2 — I think it can just as well be introduced in v1.
On versioned endpoints as the path to maturity: versioning is the right tool for genuinely forward-breaking changes like the metadata cleanups, but if it becomes the default way to extend the spec we're looking at a version bump and a client migration for every additive field. And as Russell points out, the mandate doesn't actually bind an untrusted caller anyway — spec language can't. Enforcement of read-restrictions is an out-of-band trust decision the catalog makes about a principal/engine. A trusted client honors them whether they arrive over v1, a header, or v2; an untrusted one ignores them regardless. So the capability flag is advisory at best, and a v2 endpoint is redundant for this feature with the trust decision the catalog already has to make. Kurtis — I share your unease with the global header; a single signal that mixes "what I'm requesting" with "what I can handle" is exactly the conflation that makes it awkward. On versioned endpoints as the path to maturity, though, I'd be a bit careful: versioning is the right tool for genuinely forward-breaking changes like the metadata cleanups, but if it becomes the default way to extend the spec we're looking at a version bump and a client migration for every additive field. So I'd back Russell's recommendation: ship read-restrictions in v1 and pursue v2 in parallel for the genuinely forward-breaking metadata changes — that's where mandating client understanding actually buys something. Christian On Fri, 5 Jun 2026 at 03:05, Russell Spitzer <[email protected]> wrote: > I don't really have an issue with a v2 endpoint, there are some good > reasons on Ryan's list (optional location, optional snapshots, creds > out of properties) but I don't see how a v2 actually changes the read > restrictions story. > > From my perspective read restrictions aren't a client capability the > client gets to opt into in the first place. Either the admin has set > up the catalog to send restrictions for a particular principal/engine > or it hasn't, the client doesn't really get a vote. Both proposals > are trying to use the protocol to establish that the caller will > honor restrictions, the header does it by advertising support and v2 > does it by mandating it in the spec, but neither one actually does > that. > > A client that says "I support read-restrictions" can ignore > them, a client calling v2 can ignore them, spec language doesn't bind > an untrusted caller. The catalog still has to decide out of band > whether this principal is trusted to receive a restriction-bearing > response, and once that decision has been made the header and the > endpoint version don't really add anything. > > So the header is advisory at best (a "fail fast if misconfigured" > hint), and a v2 endpoint, for this particular feature, is redundant > with whatever admin configuration is already required. I don't think > we should be encoding a trust signal into the protocol that the > protocol can't actually enforce. > > I recommend we move ahead with read-restrictions without > the capability flag in the V1 interface and do V2 work in parallel. > > On Thu, Jun 4, 2026 at 5:56 PM <[email protected]> wrote: > >> Thank you for sending this to the dev list Dan. I was having connectivity >> issues during the previous meeting and didn’t quite wrap my head around the >> proposal at that time. >> >> +1 to creating a v2 loadTable api. >> >> I worry (maybe needlessly) about the pattern a generic global >> capabilities header might set and how future spec changes might try to >> leverage this header. It feels itchy that a client could make a request for >> something, but also inform the server that it doesn’t have the capability >> to handle the response. I know it’s a slippery slope argument, but it >> doesn’t seem like a reason not to be concerned about that potential use. >> >> I am of the opinion currently that now establishing the pattern of using >> versioned endpoints to extend Iceberg functionality is a greater step >> towards maturity of the project. >> >> Best Regards, >> Kurtis C. Wright >> >> On Jun 4, 2026, at 15:48, Ryan Blue <[email protected]> wrote: >> >> >> Ugh, I tried to send this on Saturday but ended up having it blocked >> because I used my personal gmail. Here's my original email. I think I'm >> saying much the same thing as Dan. >> >> I was talking to Dan about the REST capabilities header earlier this week >> and he convinced me that one problem with the generic header is the >> association with endpoints. If we say that the "read-restrictions" >> capability requires certain behavior of the load table endpoint, then what >> happens if we later realize that other endpoints should also participate? >> We would need additional headers and this is the key idea behind Dan's >> point that we would need to associate capabilities with specific endpoints >> -- and there's not a great way to document that. >> >> In that discussion, I had an idea that I think might be a good third >> option. Adding a capability is a way for clients to opt into new >> forward-breaking changes. But we already have a way to introduce >> forward-breaking changes: versioned endpoints. With a new endpoint, clients >> are required to understand payloads and behave certain ways. If we >> introduced a v2 loadTable endpoint, we could add read restrictions and >> mandate that clients must fail if there are unsupported restrictions. >> >> There are other reasons to introduce a v2 endpoint as well: we are making >> table location optional in v4, we have discussed making snapshots optional >> (an empty list means an empty table!), and we could default the snapshot >> loading behavior to refs. >> >> I think we should seriously consider adding a v2 endpoint since that's >> the way that we already have to introduce new required functionality. Then >> we don't have to worry about either too many custom headers or associating >> values from a generic header with endpoints and behavior. I think this is >> much cleaner than either head-based solution. >> >> On Thu, Jun 4, 2026 at 8:45 AM Daniel Weeks <[email protected]> wrote: >> >>> Thanks Prashant, >>> >>> I added some comments to the various PRs, but wanted to follow up here >>> as well. >>> >>> I have several concerns regarding the complexity this approach would >>> introduce. While this may seem simple initially, a number of artifacts >>> emerge when you start to dig into it: >>> - Capabilities are not just what the client supports, it's what the >>> engine supports and how the environment has been configured in some cases >>> (like read restrictions) >>> - A global header means that if associated behaviors apply only to a >>> subset of endpoints, it's not clear where the behavior is targeted >>> - Evolution is already being discussed through the addition of >>> versioning to the labels >>> - This can lead to a proliferation of capabilities that have behaviors >>> orthogonal (or at least nuanced variations) to what is defined in the >>> OpenAPI spec >>> >>> I feel like what we're actually doing is trying to work around >>> introducing new versioned endpoints (e.g. v2 loadTable). We already have >>> versioning in the path, but we have yet to justify introducing a new >>> version. Now we're taking extreme steps and introducing new features to >>> continue down that path. >>> >>> I think it's worth looking at what it would take to introduce a v2 >>> loadTable endpoint and correct for a number of issues with the original >>> endpoint (creds duplicated in properties, required metadata location) and >>> introduce new response objects so that it's clear the calling client >>> clearly understands the endpoint. >>> >>> I'm not fundamentally opposed to client capabilities at this point, but >>> it feels like we're creating complexity that isn't narrowly targeted. >>> >>> -Dan >>> >>> On Wed, May 27, 2026 at 11:50 PM Prashant Singh < >>> [email protected]> wrote: >>> >>>> Hi Dan, >>>> >>>> Wanted to follow up specifically on your concerns since we >>>> re-discussed this at the 05/26 sync (you weren't there for >>>> the final stretch). Recording: >>>> https://www.youtube.com/watch?v=-KEesN1udyY >>>> >>>> *On the per-endpoint vs. generic header pattern*: the room leaned >>>> generic, mostly on the argument that >>>> X-Iceberg-Access-Delegation is doing a different job - it's a >>>> per-request directive ("vend credentials for this >>>> call"), not a static declaration of what the client understands. >>>> Modeling capabilities the same way conflates two >>>> patterns: one is runtime preference, the other is a build-time fact >>>> about the client. >>>> >>>> *On the OpenAPI tooling concern (the "spec within a spec" point **from >>>> sync)*: the values are still a constrained enum in the >>>> OpenAPI schema - same shape as X-Iceberg-Access-Delegation. Generated >>>> clients validate values; servers get a closed >>>> list; adding a capability is a one-line enum edit. There's no >>>> parallel registry - it's a normal header whose schema >>>> happens to be enum-typed. >>>> >>>> * On bundling all capabilities per request*: Russell raised the same >>>> - could a client send only the relevant subset per >>>> endpoint? The sync's view was that since capabilities don't change >>>> between calls, per-endpoint subsetting adds client >>>> bookkeeping without much benefit. Open to revisiting if you have a >>>> use case in mind where conditional advertisement >>>> matters (e.g., a client that wants to suppress a capability it >>>> technically supports). >>>> >>>> Looking forward to hearing from you. >>>> >>>> Thanks, >>>> Prashant >>>> >>>> On Mon, May 25, 2026 at 12:47 PM Prashant Singh < >>>> [email protected]> wrote: >>>> >>>>> Thanks everyone for the feedback. Apologies for the delayed response >>>>> on the >>>>> list, I wasn't receiving mailing list emails and had only been >>>>> responding on >>>>> the PR so far. >>>>> >>>>> * On versioning (Sung):* >>>>> >>>>> I don't think we need versioning if we bake into the spec that >>>>> clients MUST >>>>> fail when they encounter something they don't understand within a >>>>> capability's >>>>> payload. The capability signal means "I understand this payload >>>>> exists and I >>>>> will do the right thing with it" — not "I understand every possible >>>>> value >>>>> within it forever." The fail-closed contract gives us forward >>>>> compatibility >>>>> without version suffixes. >>>>> >>>>> *On single generic header vs. per-feature headers (Dan, Russell):* >>>>> >>>>> Re-listening to the sync, I don't think we landed on a hard >>>>> conclusion either >>>>> way. I went with the generic approach based on how other protocols >>>>> handle >>>>> this. >>>>> >>>>> *Precedents*: >>>>> >>>>> - Anthropic's API : single *anthropic-beta* header with >>>>> comma-separated feature >>>>> tokens; different endpoints consume different tokens, client sends >>>>> the union. >>>>> - Delta Sharing : single *delta-sharing-capabilities* header mixing >>>>> different >>>>> feature dimensions, server acts only on what's relevant per endpoint. >>>>> >>>>> One distinction that helped me think about it: per-feature headers >>>>> like >>>>> X-Iceberg-Access-Delegation are directives for a specific request - >>>>> "vend >>>>> credentials for this call." A capabilities header is different - >>>>> it's a >>>>> passive, static advertisement of what the client understands. >>>>> Bundling these >>>>> into one header keeps them distinct from per-request directives. >>>>> >>>>> The practical benefit: adding a new capability is a one-line enum >>>>> addition. >>>>> >>>>> I don't have a strong preference between the two approaches and am >>>>> happy to >>>>> revise the PR based on community consensus. Russell, to your point >>>>> about >>>>> scoping it to relevant endpoints - Sung raised the same on the PR >>>>> and I'm open >>>>> to that. >>>>> >>>>> Happy to discuss more in the upcoming sync (05/26). >>>>> >>>>> Thanks, >>>>> Prashant >>>>> >>>>> >>>>> >>>>> On Fri, May 22, 2026 at 1:15 PM Russell Spitzer < >>>>> [email protected]> wrote: >>>>> >>>>>> This seems matched to what I thought we went over, I'm also not sure >>>>>> why the shape *capabilities : {foo, bar}* necessarily mean the >>>>>> client has to send all the capabilities with every request. Can the >>>>>> client >>>>>> just decide to send *capabilities:{foo}* to those endpoints that >>>>>> care about foo? >>>>>> >>>>>> On Tue, May 19, 2026 at 2:54 PM Daniel Weeks <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Thanks for following up on this, Prashant. >>>>>>> >>>>>>> My recollection of the conversation was not to add a single header >>>>>>> that allows an arbitrary number of capabilities set, but rather to >>>>>>> follow >>>>>>> the pattern established with the `x-iceberg-access-delegation' and use >>>>>>> specific headers applicable to endpoints where they would apply. >>>>>>> >>>>>>> I'm not convinced we want to have all the capabilities bundled up >>>>>>> and sent for every request (though this might be easier to implement in >>>>>>> some cases). There may be scenarios where you want to advertise >>>>>>> capabilities or not depending on the client configuration and would make >>>>>>> managing the set of properties more difficult. >>>>>>> >>>>>>> After reviewing the conversation, this doesn't quite correspond to >>>>>>> how we ended the discussion. >>>>>>> >>>>>>> -Dan >>>>>>> >>>>>>> On Tue, May 19, 2026 at 7:06 AM Sung Yun <[email protected]> wrote: >>>>>>> >>>>>>>> Thanks Prashant, I’m supportive of introducing a generic client >>>>>>>> capabilities header. I agree with the direction of treating this >>>>>>>> primarily >>>>>>>> as capability advertisement for compatibility and response shaping >>>>>>>> rather >>>>>>>> than as a trust mechanism. >>>>>>>> >>>>>>>> I left a couple comments on the PR. One thing I’m wondering is >>>>>>>> whether we should discuss and define the versioning and compatibility >>>>>>>> model >>>>>>>> for capability header values as part of this proposal. >>>>>>>> >>>>>>>> Capabilities like `read-restrictions` seem very likely to evolve >>>>>>>> over time in ways that older clients may not be able to safely >>>>>>>> consume. I’m >>>>>>>> curious whether the community thinks capability values should represent >>>>>>>> versioned compatibility contracts (like read-restrictions.v1) and >>>>>>>> whether >>>>>>>> we want to define any expectations around cross-version compatibility >>>>>>>> behavior now as we introduce the header. >>>>>>>> >>>>>>>> Sung >>>>>>>> >>>>>>>> On 2026/05/18 18:51:52 Prashant Singh wrote: >>>>>>>> > Hi all, >>>>>>>> > >>>>>>>> > I'd like to propose adding a new HTTP header, >>>>>>>> > X-Iceberg-Client-Capabilities, >>>>>>>> > to the REST catalog spec. This was brought up at the Read >>>>>>>> Restrictions >>>>>>>> > community sync on May 12, 2026; I'm bringing it to the broader >>>>>>>> list now >>>>>>>> > for >>>>>>>> > wider feedback. >>>>>>>> > >>>>>>>> > PR: https://github.com/apache/iceberg/pull/16394 >>>>>>>> > Recording: https://youtu.be/b9p6mI-k-0I >>>>>>>> > >>>>>>>> > *Context* >>>>>>>> > >>>>>>>> > Iceberg's REST protocol keeps evolving — server-side scan >>>>>>>> > planning, remote signing, and the in-flight ReadRestrictions >>>>>>>> proposal >>>>>>>> > (PR #13879) are all features that catalogs may return but older >>>>>>>> clients >>>>>>>> > can't handle. Today there's no standard way for a client to >>>>>>>> declare >>>>>>>> > "I understand X" to the server. One direction discussed at the >>>>>>>> community >>>>>>>> > sync was to introduce a single generic capability header rather >>>>>>>> than >>>>>>>> > per-feature negotiations; this thread is to gather broader >>>>>>>> input on that >>>>>>>> > proposal. >>>>>>>> > >>>>>>>> > *Proposal* >>>>>>>> > >>>>>>>> > Send X-Iceberg-Client-Capabilities on every catalog request: >>>>>>>> > >>>>>>>> > ex: X-Iceberg-Client-Capabilities: >>>>>>>> > vended-credentials,remote-signing,scan-planning >>>>>>>> > >>>>>>>> > The Java SDK adds it via HTTPClient.baseHeaders — the same path >>>>>>>> used for >>>>>>>> > X-Client-Version and X-Client-Git-Commit-Short today. Other >>>>>>>> client >>>>>>>> > implementations (PyIceberg, Rust, Go, etc.) can mirror the same >>>>>>>> enum. >>>>>>>> > >>>>>>>> > Initial capability set for Java SDK: >>>>>>>> > - vended-credentials >>>>>>>> > - remote-signing >>>>>>>> > - scan-planning >>>>>>>> > >>>>>>>> > read-restrictions will be added once PR #13879 lands. >>>>>>>> > >>>>>>>> > >>>>>>>> > * Design notes worth feedback* >>>>>>>> > 1. Forward-compat hint, not security. The header is >>>>>>>> informational — >>>>>>>> > clients >>>>>>>> > can trivially spoof its value, so servers MUST NOT base >>>>>>>> trust or >>>>>>>> > authorization decisions on it. Trust establishment (mTLS, >>>>>>>> OAuth, etc.) >>>>>>>> > is out of scope. The spec parameter description states this >>>>>>>> explicitly. >>>>>>>> > >>>>>>>> > 2. enum: constraint on the schema. Mirrors >>>>>>>> X-Iceberg-Access-Delegation. >>>>>>>> > Adding >>>>>>>> > a new capability is a one-line spec edit; generated clients >>>>>>>> can >>>>>>>> > validate >>>>>>>> > values; servers get a machine-readable closed list of >>>>>>>> recognized >>>>>>>> > values. >>>>>>>> > >>>>>>>> > 3. Skipped for /v1/oauth/tokens. OAuth token endpoints may be >>>>>>>> served by >>>>>>>> > external IdPs (Okta, Auth0, etc.) where Iceberg-specific >>>>>>>> headers are >>>>>>>> > noise. Mirrors how X-Iceberg-Access-Delegation is handled. >>>>>>>> > >>>>>>>> > *Links* >>>>>>>> > >>>>>>>> > - PR: https://github.com/apache/iceberg/pull/16394 >>>>>>>> > - May 12 sync recording: https://youtu.be/b9p6mI-k-0I >>>>>>>> > - Related: PR #13879 < >>>>>>>> https://github.com/apache/iceberg/pull/13879> >>>>>>>> > >>>>>>>> > Thanks, >>>>>>>> > Prashant >>>>>>>> > >>>>>>>> >>>>>>>
