Re: [PROPOSAL] New REST Catalog Spec

2024-06-19 Thread Ajantha Bhat
+1 for TCK and a reference implementation. Once the reference implementation is ready, I suggest we also update the below quickstart to use it. https://iceberg.apache.org/spark-quickstart/?h=tabular#spark-and-iceberg-quickstart In my opinion, we should avoid directing open-source users to closed-s

Re: [PROPOSAL] New REST Catalog Spec

2024-06-19 Thread Jean-Baptiste Onofré
Hi Dmitri Yes, it makes sense to me. As discussed today during the community meeting, where the "simple ref impl"/TCK are located is not a big deal. But, I think it would be great to actually have sref impl/TCK (in a separate repo or not). Regards JB On Wed, Jun 19, 2024 at 7:08 PM Dmitri Bourl

Re: [PROPOSAL] New REST Catalog Spec

2024-06-19 Thread Dmitri Bourlatchkov
Re: TCK and ref impl in the "spec" repo. I believe the reference implementation there does not have to be complete in the sense of actually managing Iceberg metadata. It only needs to be complete enough to allow TCK tests to pass. TCKs will be the normative checks for "real" catalog implementations

Re: [PROPOSAL] New REST Catalog Spec

2024-06-19 Thread Jean-Baptiste Onofré
Hi folks, I wanted to provide an update on this proposal and also extend discussion. I started to rework on the REST Catalog proposal by splitting in v1-improvements and v2-proposal (focusing on extensions), step by step. I would like to emphasize a few proposals from the doc: 1. In order to gua

Re: [PROPOSAL] New REST Catalog Spec

2024-05-11 Thread Jean-Baptiste Onofré
Hi folks Quick update on the proposal. In order to illustrate the proposal in terms of versioning, TCK kind, new operations, I started a PR to generate the openapi yaml using interfaces. These interfaces are managed like public API (also covered by revapi). The purpose is also to have a concrete

Re: [PROPOSAL] New REST Catalog Spec

2024-04-12 Thread Jean-Baptiste Onofré
Hi, I updated the proposal document to have sections with target changes. I will work on more details (Jack will update the document as well I guess). I will start a simple branch to illustrate proposals from the document. Thanks, Regards JB On Thu, Apr 4, 2024 at 9:46 AM Jean-Baptiste Onofré

Re: [PROPOSAL] New REST Catalog Spec

2024-04-04 Thread Jean-Baptiste Onofré
Hi folks, Quick update: first of all, thanks a lot for your feedback in the doc ! It's great to see such interest in this proposal. According to your comments, I'm reworking the document, grouping in three main topics: - REST Catalog changes - Table/View Spec impacts - Related topics I started l

Re: [PROPOSAL] New REST Catalog Spec

2024-04-02 Thread Jean-Baptiste Onofré
Hi Ajantha Thanks for your comments in the doc ! I will address your comments. Regards JB On Tue, Apr 2, 2024 at 10:56 AM Ajantha Bhat wrote: > > Nice proposal. > Thanks for working on it. > Just finished going through the doc and added some inputs. > > I believe that through further discussion

Re: [PROPOSAL] New REST Catalog Spec

2024-04-02 Thread Ajantha Bhat
Nice proposal. Thanks for working on it. Just finished going through the doc and added some inputs. I believe that through further discussion, we can establish a robust standard for the catalog specification 👍 - Ajantha On Mon, Apr 1, 2024 at 8:07 PM Jean-Baptiste Onofré wrote: > Hi folks, > >