Re: [PROPOSAL] Apache Iceberg 1.4.3 release

2023-12-04 Thread Gabor Kaszab
Hey, We have recently found a regression in the expireSnapshot functionality. https://github.com/apache/iceberg/pull/9183 Would it make sense to include thisfix as well to the next patch release? Gabor On Sun, Dec 3, 2023 at 6:04 AM Jean-Baptiste Onofré wrote: > Hi Fokko, > > I know we have so

Re: [PROPOSAL] Apache Iceberg 1.4.3 release

2023-12-04 Thread Jean-Baptiste Onofré
Hi Gabor It sounds reasonable to me. Let me review #9183. Thanks ! Regards JB On Mon, Dec 4, 2023 at 9:51 AM Gabor Kaszab wrote: > > Hey, > > We have recently found a regression in the expireSnapshot functionality. > https://github.com/apache/iceberg/pull/9183 Would it make sense to include

Re: Is there a way to distcp iceberg table from hadoop?

2023-12-04 Thread Russell Spitzer
Delta now exposes this functionality as a command, and some groups (like ours) have some internal functionality for doing this. I think it's worth reconsidering this as a first class procedure in the Iceberg-Spark module since we get a lot of requests about it and now position deletes are a bit mor

Re: Is there a way to distcp iceberg table from hadoop?

2023-12-04 Thread Yufei Gu
This rewrite tool(https://github.com/apache/iceberg/pull/4705) works for COW. And it is not hard to extend it to support MOR. It needs blessing from the community if we want to merge it. Yufei On Mon, Dec 4, 2023 at 8:05 AM Russell Spitzer wrote: > Delta now exposes this functionality as a com

Invitation to contribute to OneTable

2023-12-04 Thread Jesus Camacho Rodriguez
Hi All, We are reaching out regarding a new project in the table formats space - OneTable. OneTable[1] is an omni-directional converter for table formats that facilitates interoperability across data processing systems and query engines. Currently, OneTable supports widely adopted open-source tab

Re: Invitation to contribute to OneTable

2023-12-04 Thread Walaa Eldin Moustafa
Thanks Jesus for sharing OneTable. Looks like it touches upon some of the topics we discussed in the Rise of Table Formats panel at VLDB back in September. I was browsing through the source code, and I ran into the OneField

Re: Invitation to contribute to OneTable

2023-12-04 Thread Gaurav Agarwal
HI Thanks for this mail , I would like to know is there any group discussion also happened or any call to discuss the issues. thanks On Tue, Dec 5, 2023 at 9:29 AM Walaa Eldin Moustafa wrote: > Thanks Jesus for sharing OneTable. Looks like it touches upon some of the > topics we discussed in t