Re: TCK and ref impl in the "spec" repo. I believe the reference implementation there does not have to be complete in the sense of actually managing Iceberg metadata. It only needs to be complete enough to allow TCK tests to pass. TCKs will be the normative checks for "real" catalog implementations, not the reference "server" impl. We will have to be careful about making TCKs match and validate the intention behind the OpenAPI spec. Does that make sense?
For that matter I think runtime dependencies on the reference server implementation should not be an issue as they will not trickle down to any other implementation. Cheers, Dmitri. On Wed, Jun 19, 2024 at 11:43 AM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi folks, > > I wanted to provide an update on this proposal and also extend discussion. > > I started to rework on the REST Catalog proposal by splitting in > v1-improvements and v2-proposal (focusing on extensions), step by > step. > > I would like to emphasize a few proposals from the doc: > 1. In order to guarantee isolation of concerns and easier > contribution, I think it would be great to create iceberg-rest repo > and move openapi.yml in this repo. > 2. In the iceberg-rest repo, in order to verify the spec, we should > have a simple reference implementation. The purpose is NOT to have a > production grade implementation, but a simple ref implementation to > verify that all is working/integrate fine with engines and different > iceberg components. Weeks ago I started an implementation with Quarkus > but I stopped as we had a discussion saying it doesn't make sense to > have an "runtime" as part of Iceberg. I agree with that but I still > think the proposal of a simple ref implementation as part of > iceberg-rest makes sense. I think I was wrong to start with Quarkus > (too large), I will resume work this ref impl using pure Java (SPI > like, similar to what I did in Apache Karaf Minho > https://github.com/apache/karaf-minho). > 3. The iceberg-rest repo would also contain a TCK to verify REST > Catalog implementation compliance with the spec > > I propose to rework on the REST v2 proposal including these topics. > Thoughts ? > > Thanks > Regards > JB > > On Mon, Apr 1, 2024 at 4:36 PM Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > > > > Hi folks, > > > > With several people from the community, we started to craft a proposal > > to design new REST Catalog Spec. > > > > I used the "proposal process" to track this: > > > > - The proposal "issue" is here: > https://github.com/apache/iceberg/issues/10075 > > - The proposal document is here: > > > https://docs.google.com/document/d/1JUtFpdEoa6IAKt1EzJi_re0PUbh56XnfUtRe5WAfl0s/edit?usp=sharing > > > > I propose the following next steps about this proposal: > > 1. Please review and comment in the doc > > 2. After ~ week, I would like to schedule a meeting to discuss the > > proposal and refine the document > > 3. Once the proposal is good enough (I would say a couple of weeks), I > > will call for a formal vote (according to the proposal process). > > > > Thanks ! > > Regards > > JB >
