Re: [DISCUSSION] Addressing security questions in the Iceberg REST specification

Should we vote on this deprecation? As we said in the sync meeting, we do not want to make decisions in that meeting, all such proposal decisions should be made on devlist. -Jack On Tue, Jul 2, 2024 at 1:03 AM Robert Stupp wrote: > Summary of the REST-sync meeting yesterday, comments welcome: >

Re: [DISCUSSION] Addressing security questions in the Iceberg REST specification

Summary of the REST-sync meeting yesterday, comments welcome: There were no objections to the PR for M1 (deprecation of the endpoint in the spec). The only comment was to deprecate the oauth schema types as well. This has been integrated in the PR. A concern was raised that it's convenient

Re: [DISCUSSION] Addressing security questions in the Iceberg REST specification

It's been a while since the discussion started. There were no objections to the proposal, so I went ahead and started to implement "M1" of the document [1]. [1] PR for M1: https://github.com/apache/iceberg/pull/10603 Robert On 19.06.24 15:11, Robert Stupp wrote: Alright, I've just creat

[DISCUSSION] Addressing security questions in the Iceberg REST specification

Alright, I've just created the issue: https://github.com/apache/iceberg/issues/10537 The proposal document is here: https://docs.google.com/document/d/1Xi5MRk8WdBWFC3N_eSmVcrLhk3yu5nJ9x_wC0ec6kVQ/ Robert On 29.05.24 22:01, Robert Stupp wrote: Jack's proposal is pretty much along what we'v

Re: Addressing security questions in the Iceberg REST specification

> > As already shared on the mailing list, I'm working on a PR to have > interfaces with JAXRS/Swagger annotations to generate OpenAPI > JSON/YAML with the swagger-gradle-plugin. Shouldn't it be the other way around (having an OpenAPI doc and generating the interfaces) since we may want people to

Re: Addressing security questions in the Iceberg REST specification

Excellent suggestion Ajantha, I've created a milestone: https://github.com/apache/iceberg/milestone/46 I would also refrain from removing the endpoint without having an alternative. Also, we have to go through a deprecation process since this is all implemented in Java, Python, and Rust. As alrea

Re: Addressing security questions in the Iceberg REST specification

> In general, we find fewer, more focused proposals allow for better discussion and faster resolution. Let us have a milestone called "REST catalog v2 spec" (similar to https://github.com/apache/iceberg/milestone/42) and keep the multiple smaller proposals organized under that. - Ajantha On Thu,

Re: Addressing security questions in the Iceberg REST specification

Thanks JB, I do feel like the discussion around OAuth2, SigV4, etc. is a big enough topic that we wouldn't want to bundle it with other proposed changes. I think the discussion around both what is included in the spec and what the reference implementations will be for each of these protocols will

Re: Addressing security questions in the Iceberg REST specification

Hi Jack, Here's my comments: 1. I don't think we should remove the oauth2 endpoint directly like this. I would first deprecate the endpoint and plan the removal in the spec v2. 4. I agree, and it has to be pluggable. I updated the REST Spec v2 proposal including first steps on v1: https://docs.g

Re: Addressing security questions in the Iceberg REST specification

Glad we have reached a certain consensus here. Agreed with Dan, we need to follow the deprecating process to remove the token endpoint, which mitigates compatibility issues. For client-side change, other than deprecating the current default token endpoint and making the oauth2-server-uri mandato

Re: Addressing security questions in the Iceberg REST specification

> I do think we need a more complete discussion around client reference implementation and I would be opposed to making any changes to the spec until that is resolved. Agree with Dan's comment on discussion and proposal on client side story. Client side needs to combine both the regular REST and

Re: Addressing security questions in the Iceberg REST specification

I feel there's general openness to removing the specific OAuth2 portions of the REST Spec as they're largely duplicative of what's covered in the OAuth2 RFC. However, I do think we need a more complete discussion around client reference implementation and I would be opposed to making any changes t

Re: Addressing security questions in the Iceberg REST specification

Jack's proposal is pretty much along what we've been thinking of. We can help with OAuth 2 client implementations for Java that support bearer, client-credentials, password as well as authorization code and device code flows (see docs [1] and implementation [2]. All implementations are built v

Re: Addressing security questions in the Iceberg REST specification

Just to reiterate my points discussed in the community sync here: the more I think about it the more I agree the OAuth endpoint *should be removed from the REST spec*. Even though the endpoint is optional, and even if we do not care about the security concerns, it still provides users an impression

Re: Addressing security questions in the Iceberg REST specification

Wondering if the auth endpoints can be separated out to a separate OpenAPI spec file. Then we still have some reference for interactions with auth server and make it clear it is not required as part of the REST catalog server. In most enterprise environments, auth server is likely a separate server

Re: Addressing security questions in the Iceberg REST specification

Hi, > On point 4, isn't that possible today, Can't that be achieved with the > current token exchange approach, and the internal implementation of the > endpoint? Unfortunately, no. Token exchange is not widely adopted yet: for example, Keycloak has only partial support for it, and Authelia, or

Re: Addressing security questions in the Iceberg REST specification

I disagree with removing "/v1/oauth/tokens" and I think I also disagree with the premise that implementing that endpoint is required, but I can understand how that's not clear in the spec. I think we can address the required vs non-required discussion with the capabilities PR.

Re: Addressing security questions in the Iceberg REST specification

Not an expert on authentication, but reading from the context, I agree that it’s not a good practice to use a resource server as a token server. The resource server would need to securely handle and store credentials or tokens, increasing the risk of credential theft or leakage. Making the token en

Re: Addressing security questions in the Iceberg REST specification

Sounds like we should try to finalize a consensus around https://github.com/apache/iceberg/pull/9940, so that we make it very clear what APIs/features are optional. -Jack On Tue, May 28, 2024 at 7:25 AM Fokko Driesprong wrote: > Hey Robert, > > Sorry for the late reply as I was out last week. I

Re: Addressing security questions in the Iceberg REST specification

Hey Robert, Sorry for the late reply as I was out last week. I'm not an OAuth guru either, but some context from my end. * Credentials (for example username/password) must _never_ be sent to > the resource server, only to the authorization server. In an earlier discussion

Re: Addressing security questions in the Iceberg REST specification

Seems like this thread did not get much attention? > * (Naive) Iceberg REST servers may proxy requests received for > '/v1/oauth/tokens’ - and effectively become a “man-in-the-middle”, which is not fully compliant with the OAuth 2.0 specification. This seems like a concern to me, could any REST

Re: Addressing security questions in the Iceberg REST specification

I think Jack makes a good point with AWS SigV4 Authentication. I suppose, in REST Catalog implementations that support that auth method, the /v1/oauth/token Catalog REST endpoint is redundant. Cheers, Dmitri. On Thu, May 23, 2024 at 9:20 AM Jack Ye wrote: > I do not know enough details about OA

Re: Addressing security questions in the Iceberg REST specification

I do not know enough details about OAuth to make comments about this issue, but just regarding the statement "OAuth2 is the only mechanism supported by the Iceberg client", AWS Sigv4 auth is also supported, at least in the Java client implementation

Addressing security questions in the Iceberg REST specification

Hi all, Iceberg REST implementations, either accessible on the public internet or inside an organization, are usually being secured using appropriate authorization mechanisms. The Nessie team is looking at implementing the Iceberg REST specification and have some questions around the security