I don't really have an issue with a v2 endpoint, there are some good reasons on Ryan's list (optional location, optional snapshots, creds out of properties) but I don't see how a v2 actually changes the read restrictions story.
>From my perspective read restrictions aren't a client capability the client gets to opt into in the first place. Either the admin has set up the catalog to send restrictions for a particular principal/engine or it hasn't, the client doesn't really get a vote. Both proposals are trying to use the protocol to establish that the caller will honor restrictions, the header does it by advertising support and v2 does it by mandating it in the spec, but neither one actually does that. A client that says "I support read-restrictions" can ignore them, a client calling v2 can ignore them, spec language doesn't bind an untrusted caller. The catalog still has to decide out of band whether this principal is trusted to receive a restriction-bearing response, and once that decision has been made the header and the endpoint version don't really add anything. So the header is advisory at best (a "fail fast if misconfigured" hint), and a v2 endpoint, for this particular feature, is redundant with whatever admin configuration is already required. I don't think we should be encoding a trust signal into the protocol that the protocol can't actually enforce. I recommend we move ahead with read-restrictions without the capability flag in the V1 interface and do V2 work in parallel. On Thu, Jun 4, 2026 at 5:56 PM <[email protected]> wrote: > Thank you for sending this to the dev list Dan. I was having connectivity > issues during the previous meeting and didn’t quite wrap my head around the > proposal at that time. > > +1 to creating a v2 loadTable api. > > I worry (maybe needlessly) about the pattern a generic global capabilities > header might set and how future spec changes might try to leverage this > header. It feels itchy that a client could make a request for something, > but also inform the server that it doesn’t have the capability to handle > the response. I know it’s a slippery slope argument, but it doesn’t seem > like a reason not to be concerned about that potential use. > > I am of the opinion currently that now establishing the pattern of using > versioned endpoints to extend Iceberg functionality is a greater step > towards maturity of the project. > > Best Regards, > Kurtis C. Wright > > On Jun 4, 2026, at 15:48, Ryan Blue <[email protected]> wrote: > > > Ugh, I tried to send this on Saturday but ended up having it blocked > because I used my personal gmail. Here's my original email. I think I'm > saying much the same thing as Dan. > > I was talking to Dan about the REST capabilities header earlier this week > and he convinced me that one problem with the generic header is the > association with endpoints. If we say that the "read-restrictions" > capability requires certain behavior of the load table endpoint, then what > happens if we later realize that other endpoints should also participate? > We would need additional headers and this is the key idea behind Dan's > point that we would need to associate capabilities with specific endpoints > -- and there's not a great way to document that. > > In that discussion, I had an idea that I think might be a good third > option. Adding a capability is a way for clients to opt into new > forward-breaking changes. But we already have a way to introduce > forward-breaking changes: versioned endpoints. With a new endpoint, clients > are required to understand payloads and behave certain ways. If we > introduced a v2 loadTable endpoint, we could add read restrictions and > mandate that clients must fail if there are unsupported restrictions. > > There are other reasons to introduce a v2 endpoint as well: we are making > table location optional in v4, we have discussed making snapshots optional > (an empty list means an empty table!), and we could default the snapshot > loading behavior to refs. > > I think we should seriously consider adding a v2 endpoint since that's the > way that we already have to introduce new required functionality. Then we > don't have to worry about either too many custom headers or associating > values from a generic header with endpoints and behavior. I think this is > much cleaner than either head-based solution. > > On Thu, Jun 4, 2026 at 8:45 AM Daniel Weeks <[email protected]> wrote: > >> Thanks Prashant, >> >> I added some comments to the various PRs, but wanted to follow up here as >> well. >> >> I have several concerns regarding the complexity this approach would >> introduce. While this may seem simple initially, a number of artifacts >> emerge when you start to dig into it: >> - Capabilities are not just what the client supports, it's what the >> engine supports and how the environment has been configured in some cases >> (like read restrictions) >> - A global header means that if associated behaviors apply only to a >> subset of endpoints, it's not clear where the behavior is targeted >> - Evolution is already being discussed through the addition of versioning >> to the labels >> - This can lead to a proliferation of capabilities that have behaviors >> orthogonal (or at least nuanced variations) to what is defined in the >> OpenAPI spec >> >> I feel like what we're actually doing is trying to work around >> introducing new versioned endpoints (e.g. v2 loadTable). We already have >> versioning in the path, but we have yet to justify introducing a new >> version. Now we're taking extreme steps and introducing new features to >> continue down that path. >> >> I think it's worth looking at what it would take to introduce a v2 >> loadTable endpoint and correct for a number of issues with the original >> endpoint (creds duplicated in properties, required metadata location) and >> introduce new response objects so that it's clear the calling client >> clearly understands the endpoint. >> >> I'm not fundamentally opposed to client capabilities at this point, but >> it feels like we're creating complexity that isn't narrowly targeted. >> >> -Dan >> >> On Wed, May 27, 2026 at 11:50 PM Prashant Singh <[email protected]> >> wrote: >> >>> Hi Dan, >>> >>> Wanted to follow up specifically on your concerns since we >>> re-discussed this at the 05/26 sync (you weren't there for >>> the final stretch). Recording: >>> https://www.youtube.com/watch?v=-KEesN1udyY >>> >>> *On the per-endpoint vs. generic header pattern*: the room leaned >>> generic, mostly on the argument that >>> X-Iceberg-Access-Delegation is doing a different job - it's a >>> per-request directive ("vend credentials for this >>> call"), not a static declaration of what the client understands. >>> Modeling capabilities the same way conflates two >>> patterns: one is runtime preference, the other is a build-time fact >>> about the client. >>> >>> *On the OpenAPI tooling concern (the "spec within a spec" point **from >>> sync)*: the values are still a constrained enum in the >>> OpenAPI schema - same shape as X-Iceberg-Access-Delegation. Generated >>> clients validate values; servers get a closed >>> list; adding a capability is a one-line enum edit. There's no parallel >>> registry - it's a normal header whose schema >>> happens to be enum-typed. >>> >>> * On bundling all capabilities per request*: Russell raised the same - >>> could a client send only the relevant subset per >>> endpoint? The sync's view was that since capabilities don't change >>> between calls, per-endpoint subsetting adds client >>> bookkeeping without much benefit. Open to revisiting if you have a use >>> case in mind where conditional advertisement >>> matters (e.g., a client that wants to suppress a capability it >>> technically supports). >>> >>> Looking forward to hearing from you. >>> >>> Thanks, >>> Prashant >>> >>> On Mon, May 25, 2026 at 12:47 PM Prashant Singh < >>> [email protected]> wrote: >>> >>>> Thanks everyone for the feedback. Apologies for the delayed response >>>> on the >>>> list, I wasn't receiving mailing list emails and had only been >>>> responding on >>>> the PR so far. >>>> >>>> * On versioning (Sung):* >>>> >>>> I don't think we need versioning if we bake into the spec that >>>> clients MUST >>>> fail when they encounter something they don't understand within a >>>> capability's >>>> payload. The capability signal means "I understand this payload >>>> exists and I >>>> will do the right thing with it" — not "I understand every possible >>>> value >>>> within it forever." The fail-closed contract gives us forward >>>> compatibility >>>> without version suffixes. >>>> >>>> *On single generic header vs. per-feature headers (Dan, Russell):* >>>> >>>> Re-listening to the sync, I don't think we landed on a hard >>>> conclusion either >>>> way. I went with the generic approach based on how other protocols >>>> handle >>>> this. >>>> >>>> *Precedents*: >>>> >>>> - Anthropic's API : single *anthropic-beta* header with >>>> comma-separated feature >>>> tokens; different endpoints consume different tokens, client sends >>>> the union. >>>> - Delta Sharing : single *delta-sharing-capabilities* header mixing >>>> different >>>> feature dimensions, server acts only on what's relevant per endpoint. >>>> >>>> One distinction that helped me think about it: per-feature headers >>>> like >>>> X-Iceberg-Access-Delegation are directives for a specific request - >>>> "vend >>>> credentials for this call." A capabilities header is different - it's >>>> a >>>> passive, static advertisement of what the client understands. >>>> Bundling these >>>> into one header keeps them distinct from per-request directives. >>>> >>>> The practical benefit: adding a new capability is a one-line enum >>>> addition. >>>> >>>> I don't have a strong preference between the two approaches and am >>>> happy to >>>> revise the PR based on community consensus. Russell, to your point >>>> about >>>> scoping it to relevant endpoints - Sung raised the same on the PR and >>>> I'm open >>>> to that. >>>> >>>> Happy to discuss more in the upcoming sync (05/26). >>>> >>>> Thanks, >>>> Prashant >>>> >>>> >>>> >>>> On Fri, May 22, 2026 at 1:15 PM Russell Spitzer < >>>> [email protected]> wrote: >>>> >>>>> This seems matched to what I thought we went over, I'm also not sure >>>>> why the shape *capabilities : {foo, bar}* necessarily mean the client >>>>> has to send all the capabilities with every request. Can the client just >>>>> decide to send *capabilities:{foo}* to those endpoints that care >>>>> about foo? >>>>> >>>>> On Tue, May 19, 2026 at 2:54 PM Daniel Weeks <[email protected]> >>>>> wrote: >>>>> >>>>>> Thanks for following up on this, Prashant. >>>>>> >>>>>> My recollection of the conversation was not to add a single header >>>>>> that allows an arbitrary number of capabilities set, but rather to follow >>>>>> the pattern established with the `x-iceberg-access-delegation' and use >>>>>> specific headers applicable to endpoints where they would apply. >>>>>> >>>>>> I'm not convinced we want to have all the capabilities bundled up and >>>>>> sent for every request (though this might be easier to implement in some >>>>>> cases). There may be scenarios where you want to advertise capabilities >>>>>> or >>>>>> not depending on the client configuration and would make managing the set >>>>>> of properties more difficult. >>>>>> >>>>>> After reviewing the conversation, this doesn't quite correspond to >>>>>> how we ended the discussion. >>>>>> >>>>>> -Dan >>>>>> >>>>>> On Tue, May 19, 2026 at 7:06 AM Sung Yun <[email protected]> wrote: >>>>>> >>>>>>> Thanks Prashant, I’m supportive of introducing a generic client >>>>>>> capabilities header. I agree with the direction of treating this >>>>>>> primarily >>>>>>> as capability advertisement for compatibility and response shaping >>>>>>> rather >>>>>>> than as a trust mechanism. >>>>>>> >>>>>>> I left a couple comments on the PR. One thing I’m wondering is >>>>>>> whether we should discuss and define the versioning and compatibility >>>>>>> model >>>>>>> for capability header values as part of this proposal. >>>>>>> >>>>>>> Capabilities like `read-restrictions` seem very likely to evolve >>>>>>> over time in ways that older clients may not be able to safely consume. >>>>>>> I’m >>>>>>> curious whether the community thinks capability values should represent >>>>>>> versioned compatibility contracts (like read-restrictions.v1) and >>>>>>> whether >>>>>>> we want to define any expectations around cross-version compatibility >>>>>>> behavior now as we introduce the header. >>>>>>> >>>>>>> Sung >>>>>>> >>>>>>> On 2026/05/18 18:51:52 Prashant Singh wrote: >>>>>>> > Hi all, >>>>>>> > >>>>>>> > I'd like to propose adding a new HTTP header, >>>>>>> > X-Iceberg-Client-Capabilities, >>>>>>> > to the REST catalog spec. This was brought up at the Read >>>>>>> Restrictions >>>>>>> > community sync on May 12, 2026; I'm bringing it to the broader >>>>>>> list now >>>>>>> > for >>>>>>> > wider feedback. >>>>>>> > >>>>>>> > PR: https://github.com/apache/iceberg/pull/16394 >>>>>>> > Recording: https://youtu.be/b9p6mI-k-0I >>>>>>> > >>>>>>> > *Context* >>>>>>> > >>>>>>> > Iceberg's REST protocol keeps evolving — server-side scan >>>>>>> > planning, remote signing, and the in-flight ReadRestrictions >>>>>>> proposal >>>>>>> > (PR #13879) are all features that catalogs may return but older >>>>>>> clients >>>>>>> > can't handle. Today there's no standard way for a client to declare >>>>>>> > "I understand X" to the server. One direction discussed at the >>>>>>> community >>>>>>> > sync was to introduce a single generic capability header rather >>>>>>> than >>>>>>> > per-feature negotiations; this thread is to gather broader input >>>>>>> on that >>>>>>> > proposal. >>>>>>> > >>>>>>> > *Proposal* >>>>>>> > >>>>>>> > Send X-Iceberg-Client-Capabilities on every catalog request: >>>>>>> > >>>>>>> > ex: X-Iceberg-Client-Capabilities: >>>>>>> > vended-credentials,remote-signing,scan-planning >>>>>>> > >>>>>>> > The Java SDK adds it via HTTPClient.baseHeaders — the same path >>>>>>> used for >>>>>>> > X-Client-Version and X-Client-Git-Commit-Short today. Other >>>>>>> client >>>>>>> > implementations (PyIceberg, Rust, Go, etc.) can mirror the same >>>>>>> enum. >>>>>>> > >>>>>>> > Initial capability set for Java SDK: >>>>>>> > - vended-credentials >>>>>>> > - remote-signing >>>>>>> > - scan-planning >>>>>>> > >>>>>>> > read-restrictions will be added once PR #13879 lands. >>>>>>> > >>>>>>> > >>>>>>> > * Design notes worth feedback* >>>>>>> > 1. Forward-compat hint, not security. The header is >>>>>>> informational — >>>>>>> > clients >>>>>>> > can trivially spoof its value, so servers MUST NOT base trust >>>>>>> or >>>>>>> > authorization decisions on it. Trust establishment (mTLS, >>>>>>> OAuth, etc.) >>>>>>> > is out of scope. The spec parameter description states this >>>>>>> explicitly. >>>>>>> > >>>>>>> > 2. enum: constraint on the schema. Mirrors >>>>>>> X-Iceberg-Access-Delegation. >>>>>>> > Adding >>>>>>> > a new capability is a one-line spec edit; generated clients >>>>>>> can >>>>>>> > validate >>>>>>> > values; servers get a machine-readable closed list of >>>>>>> recognized >>>>>>> > values. >>>>>>> > >>>>>>> > 3. Skipped for /v1/oauth/tokens. OAuth token endpoints may be >>>>>>> served by >>>>>>> > external IdPs (Okta, Auth0, etc.) where Iceberg-specific >>>>>>> headers are >>>>>>> > noise. Mirrors how X-Iceberg-Access-Delegation is handled. >>>>>>> > >>>>>>> > *Links* >>>>>>> > >>>>>>> > - PR: https://github.com/apache/iceberg/pull/16394 >>>>>>> > - May 12 sync recording: https://youtu.be/b9p6mI-k-0I >>>>>>> > - Related: PR #13879 < >>>>>>> https://github.com/apache/iceberg/pull/13879> >>>>>>> > >>>>>>> > Thanks, >>>>>>> > Prashant >>>>>>> > >>>>>>> >>>>>>
