Re: [DISCUSS] REST Endpoint discovery

2024-08-20 Thread Eduard Tudenhöfner
Thanks Yufei and everyone else that participated in the discussion. I'll open a vote shortly to get this moving. On Tue, Aug 20, 2024 at 8:04 PM Yufei Gu wrote: > I'm fine with the approach to present the endpoint ID as a plain string. > For any extension in the future, we can add a new field pa

Re: [DISCUSS] REST Endpoint discovery

2024-08-20 Thread Yufei Gu
I'm fine with the approach to present the endpoint ID as a plain string. For any extension in the future, we can add a new field parallel with the endpoint array. Thanks a lot for driving this, Edward. Yufei On Tue, Aug 20, 2024 at 8:05 AM Eduard Tudenhöfner wrote: > @Karuppayya: I've answere

Re: [DISCUSS] REST Endpoint discovery

2024-08-20 Thread Eduard Tudenhöfner
@Karuppayya: I've answered your question in the doc. @Yufei: Part of the reason why the proposed format is so simple is because there were different opinions about what level of granularity we should have in the context of REST capabilities, which then led to this version. It seems people are gene

Re: [DISCUSS] REST Endpoint discovery

2024-08-16 Thread Yufei Gu
I’m OK with using a plain string for the endpoint ID, as described in doc[1]. However, I’ve been thinking about how we can make this more flexible, especially since we’ve had quite a few discussions about granularity. For instance, if we expose a bit more of the API specification, we might not nee

Re: [DISCUSS] REST Endpoint discovery

2024-08-16 Thread karuppayya
Can we consider adding `OPTIONS` verb to the resource paths, as part of the spec? That way the endpoint discovery endpoint could return only the list of supported endpoints, without the verbs. `OPTIONS` on the resource path can return the list of all supported verbs, and also other information reg

Re: [DISCUSS] REST Endpoint discovery

2024-08-16 Thread Dmitri Bourlatchkov
Sorry for a bit of back-tracking, but I'd like to clarify the meaning of those endpoint strings. I initially assumed that the strings would need to be parsed into components (verb / path) for use in runtime. My suggestion for using a JSON representation was meant to make the parsing more standard

Re: [DISCUSS] REST Endpoint discovery

2024-08-16 Thread Eduard Tudenhöfner
If we really want to add more structure to the JSON representation, then I would probably prefer what Dmitri suggested in the doc as I think { "GET": { }, "POST": {} } looks a bit weird: "endpoints":[ {"verb": "GET","path": "/v1/{prefix}/namespaces/{namespace}"}, {"verb": "GET","path": "/v1/{p

Re: [DISCUSS] REST Endpoint discovery

2024-08-15 Thread Walaa Eldin Moustafa
Thank you Eduard for sharing this version of the proposal. Looks simple, functional, and extensible. On Thu, Aug 15, 2024 at 1:10 PM Ryan Blue wrote: > I think I'm fine either way. I lean toward the simplicity of the strings > in the proposal but would not complain if we went with Yufei's sugges

Re: [DISCUSS] REST Endpoint discovery

2024-08-15 Thread Ryan Blue
I think I'm fine either way. I lean toward the simplicity of the strings in the proposal but would not complain if we went with Yufei's suggestion. On Thu, Aug 15, 2024 at 12:12 PM Yufei Gu wrote: > The current proposal lists endpoints as plain strings, and I still believe > we could make things

Re: [DISCUSS] REST Endpoint discovery

2024-08-15 Thread Yufei Gu
The current proposal lists endpoints as plain strings, and I still believe we could make things a bit smoother by adding some structure to them. Here's the example if the previous one throws you off. *Before:* "GET /v1/{prefix}/namespaces","POST /v1/{prefix}/namespaces","GET /v1/{prefix}/namespac

Re: [DISCUSS] REST Endpoint discovery

2024-08-15 Thread Jack Ye
> But I propose to use a trimmed openAPI's format directly. Looking at the example, this feels quite complicated to me. > For example, it is easier if we want to include operationID I don't think we need to consider accommodating both, since operationId is an alternative to " ". > or adding featu

Re: [DISCUSS] REST Endpoint discovery

2024-08-15 Thread Yufei Gu
+1 for the proposal. In terms of the format, the current solution is simple enough. But I propose to use a trimmed openAPI's format directly. It won't add much cost as we can just take the minimum fields we want. But it opens a window to extend it in the future. For example, it is easier if we want

Re: [DISCUSS] REST Endpoint discovery

2024-08-15 Thread Ryan Blue
I think it's more straightforward to use the format from the existing proposal. That's unambiguous and seems easier to understand to me, rather than needing to refer to the spec to find out the details. The `UpdateTable` vs `CommitTable` discrepancy is a good example, and we could have names that a

Re: [DISCUSS] REST Endpoint discovery

2024-08-15 Thread Russell Spitzer
I'm on board for this proposal. I was in the off-mail chats and I think this is probably our simplest approach going forward. On Thu, Aug 15, 2024 at 10:39 AM Dmitri Bourlatchkov wrote: > OpenAPI tool will WARN a lot if Operation IDs overlap. Generated code/html > may also look odd in case of ov

Re: [DISCUSS] REST Endpoint discovery

2024-08-15 Thread Dmitri Bourlatchkov
OpenAPI tool will WARN a lot if Operation IDs overlap. Generated code/html may also look odd in case of overlaps. All-in-all, I think the best practice is to define unique Operation IDs up front. For Iceberg REST API, the yaml file is the API definition, so it should not be a problem to ensure th

Re: [DISCUSS] REST Endpoint discovery

2024-08-15 Thread Eduard Tudenhöfner
Hey Jack, thanks for the feedback. I replied in the doc but I can reiterate my answer here too: The *path* is unique and required so that feels more appropriate than requiring to have an optional *operationId* in the OpenAPI spec. Additionally, using the path is more straight-forward when we intro

Re: [DISCUSS] REST Endpoint discovery

2024-08-15 Thread Jack Ye
Hi Eduard, In general I agree with this proposal, thanks for putting this up! Just one question (which I also added in the design), what are the thoughts behind using " ", vs using the operationId defined in the OpenAPI? The operationId approach definitely looks much cleaner to me, but (1) in Ope

Re: [DISCUSS] REST Endpoint discovery

2024-08-14 Thread Eduard Tudenhöfner
Hey Dmitri, this proposal is the result of the community feedback from the Capabilities proposal. Ultimately the capabilities turned out to entail more complexity than necessary and so this proposal solves the core problem while keeping complexity and spec changes to an absolute minimum. Eduard

Re: [DISCUSS] REST Endpoint discovery

2024-08-14 Thread Dmitri Bourlatchkov
Would you mind adding a "related proposals" section to the doc to clarify this? Thanks, Dmitri. On Wed, Aug 14, 2024 at 11:15 AM Dmitri Bourlatchkov < dmitri.bourlatch...@dremio.com> wrote: > Hi Eduard, > > How is this proposal related to the Server Capabilities discussion? > > Thanks, > Dmitri.

Re: [DISCUSS] REST Endpoint discovery

2024-08-14 Thread Dmitri Bourlatchkov
Hi Eduard, How is this proposal related to the Server Capabilities discussion? Thanks, Dmitri. On Wed, Aug 14, 2024 at 5:14 AM Eduard Tudenhöfner wrote: > Hey everyone, > > I'd like to propose a way for REST servers to communicate to clients what > endpoints it supports via a new *endpoints* f

[DISCUSS] REST Endpoint discovery

2024-08-14 Thread Eduard Tudenhöfner
Hey everyone, I'd like to propose a way for REST servers to communicate to clients what endpoints it supports via a new *endpoints* field in the *CatalogConfig* of the *v1/config* endpoint. This enables clients to make better decisions and clearly signal that a particular endpoint isn’t supported