Re: [Proposal] REST Spec: Server-side Metadata Tables

2024-07-03 Thread Szehon Ho
Hi Piotr Thanks for the reply. It’s a good point, I was thinking it would be convenient in REST, and could avoid the hassle of spec change. But you are right that it probably belongs at a lower level if we support this feature generally (like an additional boolean on snapshot). Sorry to hija

Re: [Proposal] REST Spec: Server-side Metadata Tables

2024-07-03 Thread Ajantha Bhat
Thanks for the proposal. I would recommend creating the github issue with the "proposal" label for easy tracking of all the ongoing proposals as mentioned here . The first concern that comes to mind is whethe

Re: [Proposal] REST Spec: Server-side Metadata Tables

2024-07-03 Thread Piotr Findeisen
Hi Szehon, re listing 'removed' snapshots If I understand what you're saying is the following: Iceberg table format requires users to first delete metadata information about files and only then delete the files, and sometimes users want to order these events differently. We can solve this within

Re: [Discussion] Apache Iceberg Community Guideline - Initial Version

2024-07-03 Thread Justin Mclean
HI, For some historical context, it used to be that the board / Incubator asked projects to have a set of bylaws when they graduated. This fell out of favour more than ten years ago, and projects were expected to follow documented policy. Before this, things were not as well documented, so we a

Re: [Proposal] REST Spec: Server-side Metadata Tables

2024-07-03 Thread Szehon Ho
Yes, I was chatting with Yufei about this, in the first glance I agree this would be nice to have. I always thought that metadata tables are important enough to spec somewhere, and I think this is a nice place to do it. There seems to be some overlap with existing calls (ie, you can get snapshots

Re: [Proposal] REST Spec: Server-side Metadata Tables

2024-07-03 Thread Jack Ye
Hi Yufei, Interesting that we are thinking about similar things. I had this item as a part of the roadmap discussion items in the catalog sync meeting, and then I removed it before the meeting because I felt it's too early to discuss. My main concern for having server-side metadata tables is how

[Proposal] REST Spec: Server-side Metadata Tables

2024-07-03 Thread Yufei Gu
Hi folks, I'd like to discuss a new proposal to support server-side metadata tables. One of Iceberg's most advantageous features is the ability to inspect a table using metadata tables. For instance, we can query snapshots just like we query data rows using the following command: SELECT * FROM pr

Re: [Discussion] Apache Iceberg Community Guideline - Initial Version

2024-07-03 Thread Carl Steinbach
> 1. modified the name from "bylaws" to "community guidelines", following the latest ASF guideline I want to make sure everyone is aware that there is a substantive difference between the meaning of "bylaw" and "guideline." Here's how the two words are defined in the Cambridge Dictionary: - Bylaw