Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

Hey everyone, The suggestion of re-using that endpoint was just thinking out loud. My underlying concern is what Yufei mentioned; having a proliferation of endpoints that address specific needs. I like the approach suggested by Taeyun, and would love to know what others think. I'd like to know if

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

Hi, I like the idea and it makes sense. As soon as it's clearly stated in the spec (using If-Modified-Since header and 304 status code), it looks good to me. Thanks ! Regards JB On Fri, Nov 15, 2024 at 1:58 AM Taeyun Kim wrote: > > Hi, > > (Apologies if this email is a duplicate. This is my thi

Re: [DISCUSS] Iceberg 1.7.1 release

Hi Aihua I don't think we should include such changes in a micro release: micro release is for bug fixes, not adding things like data type. Just my $0.01 :) Regards JB On Fri, Nov 15, 2024 at 6:22 AM Aihua Xu wrote: > > Hi Bryan, > > I would like to include the following in 1.7.1 if possible.

Re: [DISCUSS] Iceberg 1.7.1 release

I agree, we should leave this one for 1.8. -Bryan > On Nov 14, 2024, at 9:22 PM, Aihua Xu wrote: > > Hi Bryan, > > I would like to include the following in 1.7.1 if possible. I keep the change > minimum and I will try to get this merged soon. > > Add Variant data type to API: > https://githu

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

How does HTTP caching handle desynchronized clocks between clients and the server? - At *t0*, the client gets the latest table version. - At *t1*, the server makes a new commit. - At *t2*, the client sends a request with a timestamp *t2*, but due to desynchronization, it refers to *t0*

Re: [PROPOSAL] Create Iceberg DockerHub repository

+1 to Iceberg REST TCK docker image. Thanks, JB for driving this and Ajantha for setting up the docker image. We already found a bug in PyIceberg [1] from integrating with the TCK docker image. It would be great to have a nightly build, perhaps we can set up a Github Action to automate the docker i

Re: [DISCUSS] Iceberg 1.7.1 release

That makes sense. Originally I thought wasb scheme change was a feature change. I will wait for 1.8.0. then. On Fri, Nov 15, 2024 at 7:18 AM Bryan Keller wrote: > I agree, we should leave this one for 1.8. > -Bryan > > On Nov 14, 2024, at 9:22 PM, Aihua Xu > wrote: > > Hi Bryan, > > I would lik

Re: [PROPOSAL] Create Iceberg DockerHub repository

+1 — excited to see this happen! For the TCK, I think we can release this with the Java together, and have a nightly build (tag the container with nightly Dockerhub). This way we can already test out (and start implementing) the new features in the related projects. Thoughts on that? Regarding th

Re: Re: [Proposal] Replicating version-hint onto the file system

Hey Ashvin, Thanks for taking the time to write up the proposal. I have one big question that we need to clarify first. Many implementations out there today expect that the location is unique to the table, but this isn't called out in the spec exp

Re: [VOTE] Release Apache PyIceberg 0.8.0rc2

Hi Kevin, Thank you again for running this release! I've verified the License headers, checksums and signatures. Downloaded the RC from SVN and ran the tests. Downloaded the package from pypi and ran sanity checks. +1 (non-binding) Sung On 2024/11/14 20:56:44 Kevin Liu wrote: > Hi Everyone,

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

Hi All, First of all it's great to see that there are others who could benefit from giving a solution to this problem. I appreciate all the comments and feedback so far. There were a number of different opinions, so let me start with summarizing the different topics that came up: *New endpoint vs

Re: [VOTE][Go] Release Apache Iceberg Go v0.1.0 RC2

+1 (binding) Verified checksums, signatures, and tests with the validation script using the Apache dist (thanks Kevin!). Did some checks on the dependencies and looks good. Kind regards, Fokko Op vr 15 nov 2024 om 06:48 schreef Eduard Tudenhöfner < etudenhoef...@apache.org>: > +1 (binding) >

Re: [VOTE] Release Apache PyIceberg 0.8.0rc2

+1 binding Thanks for running this release! Checked the signatures, checksums, and licenses. Kind regards, Fokko Op vr 15 nov 2024 om 14:52 schreef Sung Yun : > Hi Kevin, > > Thank you again for running this release! > > I've verified the License headers, checksums and signatures. > > Downloade

Re: Changing default delete file granularity for Spark writes from partition to file scoped

Following up on this thread, > I don't think this is a bad idea from a theoretical perspective. Do we have any actual numbers to back up the change? There are no numbers yet, changing the default is largely driven by the fact that the previous downside of file scoped deletes leading to many files

Re: [DISCUSS] Removal of last-column-id of public API

Thanks for bringing this up Fokko. It makes sense to hide `last-column-id` from the public API, as it is an implementation detail. As mentioned in the PR, I checked references to `last-column-id`

Re: [DISCUSS] Add a implementation status page for iceberg

Thanks, Renjie! Happy to review and help fill out the matrix! :) Best, Kevin Liu On Wed, Nov 13, 2024 at 10:51 PM Renjie Liu wrote: > Hi: > > Thanks for everyone's comments. I think we reached agreement on the > design, and I'll send a pr for it. > > On Tue, Nov 12, 2024 at 1:13 AM Yufei Gu wr

Re: [VOTE] Release Apache PyIceberg 0.8.0rc2

+1 (binding) Thanks for running the release! - Verified signatures/checksum/license - Ran tests "make test-coverage" in python 3.11 Best regards, Honah On Fri, Nov 15, 2024 at 7:46 AM Fokko Driesprong wrote: > +1 binding > > Thanks for running this release! Checked the signatures, checksums,