Re: [DISCUSS] Duplicate KEYS files

Hi Fokko As we discussed about that together on Slack, I'm fine merging and removing the dev located KEYS file. Regards JB On Mon, Nov 11, 2024 at 4:13 PM Fokko Driesprong wrote: > > Hi everyone, > > While looking at the release steps for iceberg-go, I noticed that we have two > KEYS files: >

Re: [DISCUSS] Duplicate KEYS files

Oh yeah, thank you! On Mon, Nov 11, 2024 at 6:20 PM Xuanwo wrote: > Hi, Kevin > > https://downloads.apache.org/iceberg points to > https://dist.apache.org/repos/dist/release/iceberg > so we don't > need to edit it by hand. > > On Tue, Nov

Re: [DISCUSS] Duplicate KEYS files

Hi, Kevin https://downloads.apache.org/iceberg points to https://dist.apache.org/repos/dist/release/iceberg so we don't need to edit it by hand. On Tue, Nov 12, 2024, at 10:16, Kevin Liu wrote: > Does anyone know how to edit the KEYS fi

Re: [DISCUSS] Duplicate KEYS files

Does anyone know how to edit the KEYS file at https://downloads.apache.org/iceberg/KEYS? The previous instruction [1] uses `svn` which doesn't work with the above URL. ``` svn co https://dist.apache.org/repos/dist/dev/iceberg icebergsvn # works svn co https://downloads.apache.org/iceberg icebergsvn

Re: [DISCUSS] Duplicate KEYS files

+1 (binding) for merging. On Tue, Nov 12, 2024 at 1:56 AM Kevin Liu wrote: > +1 (non-binding) > Here are some places in the code base we would need to update. > https://grep.app/search?q=dist.apache.org/repos/dist/.%2A/iceberg/KEYS®exp=true > I also double-checked against the "/KEYS" search, see

Re: Greater Seattle Iceberg Meetup

Bumping this thread one last time. Looking forward to seeing everyone this Wednesday! If you have not done so, RSVP at https://lu.ma/kxi04g2m We have 2 awesome presentations: 1. Jeremy Song (Principal Engineer, AWS Glue) will discuss Glue's Iceberg table optimizations—covering topics like compacti

Re: [VOTE][Go] Release Apache Iceberg Go v0.1.0 RC0

BTW for folks verifying this RC, these are the changes I made locally for the `dev/release/verify_rc.sh` script to work. https://github.com/apache/iceberg-go/pull/199/files Best, Kevin Liu On Mon, Nov 11, 2024 at 3:03 PM Kevin Liu wrote: > Hi Matt, > > Thanks for the release candidate! +1 (non-

Re: [VOTE] Release Apache PyIceberg 0.8.0rc1

+1 (binding) Verified signatures/checksums/RAT checks. Ran install and tests Thanks, Amogh Jahagirdar On Sun, Nov 10, 2024 at 5:01 PM André Luis Anastácio wrote: > +1 (non-binding) > > - verified signature and checksum > - verified license check > - ran install and some manual tests in python

Re: [VOTE][Go] Release Apache Iceberg Go v0.1.0 RC0

Hi Matt, Thanks for the release candidate! +1 (non-binding). I was able to download, verify checksums and signatures, and run the unit tests successfully after making a few changes locally. I tried to follow the verification steps outlined in https://github.com/apache/iceberg-go/blob/main/dev/re

Re: [ANNOUNCE] Apache Iceberg release 1.7.0

Great news on the 1.7 release! Thanks to everyone who contributed and thanks Russ for driving the release itself over the finish line! On Mon, Nov 11, 2024 at 8:22 AM Bryan Keller wrote: > A user discovered an issue with the Kafka Connect distribution as a result > of an Azure dependency update

[VOTE][Go] Release Apache Iceberg Go v0.1.0 RC0

Hi, I would like to propose the following release candidate (RC0) of Apache Iceberg Go version v0.1.0. This release candidate is based on commit: adc8193de3299b04c9763c2fba529a7b94d080ce [1] The source release rc0 is hosted at [2]. Please download, verify checksums and signatures, run the unit

[Proposal] Replicating version-hint onto the file system

Hello Community, We would like to share a proposal to standardize a file system based method to identify Iceberg tables’ current snapshot. Proposal doc: Adding a File System based Consistent Method to Identify Iceberg Tables’ Current Snapshot

Re: Changing default delete file granularity for Spark writes from partition to file scoped

I don't think this is a bad idea from a theoretical perspective. Do we have any actual numbers to back up the change? I would think for most folks we would recommend just going to V3 rather than changing granularity for their new tables. It would just affect new tables though so I'm not opposed to

Changing default delete file granularity for Spark writes from partition to file scoped

Hi all, I wanted to discuss changing the default position delete file granularity for Spark from partition to file level for any newly created V2 tables. See this PR [1] Context on delete file granularity: - Partition granularity: Writers group delete files for multiple data files from the

Re: [DISCUSS] Duplicate KEYS files

+1 (non-binding) Here are some places in the code base we would need to update. https://grep.app/search?q=dist.apache.org/repos/dist/.%2A/iceberg/KEYS®exp=true I also double-checked against the "/KEYS" search, seems like we've captured all the necessary changes above https://grep.app/search?q=/KEYS

Re: [DISCUSS] Duplicate KEYS files

+1 (non-binding) for merging, I can update the docs on the iceberg Go release README after it's done! On Mon, Nov 11, 2024, 12:20 PM Yufei Gu wrote: > +1 merging sounds good. It should still work for previous releases. > > Yufei > > > On Mon, Nov 11, 2024 at 7:46 AM Xuanwo wrote: > >> Hi >> >>

Dynamic Flink Iceberg Sink

Hi Team, With Max Michels, we started to work on enhancing the current Iceberg Sink to allow inserting evolving records into a changing table. See: https://docs.google.com/document/d/1R3NZmi65S4lwnmNjH4gLCuXZbgvZV5GNrQKJ5NYdO9s Created the project to follow the lifecycle of the proposal: https://g

Re: [DISCUSS] Duplicate KEYS files

+1 merging sounds good. It should still work for previous releases. Yufei On Mon, Nov 11, 2024 at 7:46 AM Xuanwo wrote: > Hi > > Thank you, Fokko, for proposing this. Here is my +1, non-binding. > > I'd also like to mention that as part of the ASF release policy, we must > refer to "https://do

Re: [DISCUSS] Add a implementation status page for iceberg

LGTM. Thanks Renjie! Yufei On Mon, Nov 11, 2024 at 5:38 AM Renjie Liu wrote: > Hi: > > > One minor suggestion: adding a table spec version label along with the > feature in the support matrix. That doesn't apply to REST spec though. > > Updated the doc, please take a look. > > > My only comment

Re: [ANNOUNCE] Apache Iceberg release 1.7.0

A user discovered an issue with the Kafka Connect distribution as a result of an Azure dependency update. I have a PR up to fix that. It would be nice if we could get this in a follow-up 1.7.1 patch release, as the connector won't run on MSK without this. Issue: https://github.com/apache/iceber

Re: [DISCUSS] Duplicate KEYS files

Hi Thank you, Fokko, for proposing this. Here is my +1, non-binding. I'd also like to mention that as part of the ASF release policy, we must refer to "https://downloads.apache.org/iceberg/KEYS"; for KEYS; other links are not allowed. Ref: https://infra.apache.org/release-download-pages.html#l

Re: [DISCUSS] Duplicate KEYS files

Sounds good to me, although I guess it's really just up to the Rust and GO maintainers to converge On Mon, Nov 11, 2024 at 9:13 AM Fokko Driesprong wrote: > Hi everyone, > > While looking at the release steps for iceberg-go > , I notice

[DISCUSS] Duplicate KEYS files

Hi everyone, While looking at the release steps for iceberg-go , I noticed that we have two KEYS files: - https://dist.apache.org/repos/dist/dev/iceberg/KEYS - https://dist.apache.org/repos/dist/release/iceberg/KEYS (Also availa

Re: [DISCUSS] Add a implementation status page for iceberg

Hi: > One minor suggestion: adding a table spec version label along with the feature in the support matrix. That doesn't apply to REST spec though. Updated the doc, please take a look. > My only comment is probably to use versions instead of check marks, but all good :) In current approach we w

Duplicate data issue PR

Hi, I posted this PR https://github.com/apache/iceberg/pull/10962 a couple of months ago. The issue it solves can lead to duplicate data when rewrites are performed. I don't think there is any objection to the fix but it seems to have fallen through the cracks. While the issue won't happen very o