Re: [DISCUSS] Add a implementation status page for iceberg

Hi: Thanks for everyone's comments. I think we reached agreement on the design, and I'll send a pr for it. On Tue, Nov 12, 2024 at 1:13 AM Yufei Gu wrote: > LGTM. Thanks Renjie! > Yufei > > > On Mon, Nov 11, 2024 at 5:38 AM Renjie Liu > wrote: > >> Hi: >> >> > One minor suggestion: adding a ta

Re: [VOTE] Release Apache PyIceberg 0.8.0rc1

Hi folks, While testing out the Rest Catalog Adapter docker image that Ajantha has been working working on, I ran into an issue when parsing the TableResponse of a staged table. While the metadata-location is an optional field according to the Iceberg Rest Catalog Spec, the field is being hand

Re: [VOTE][Go] Release Apache Iceberg Go v0.1.0 RC2

Ugh, Gmail messed up the link. [2] should be https://github.com/apache/iceberg-go/releases/v0.1.0-rc2 Sorry. On Wed, Nov 13, 2024, 6:06 PM Matt Topol wrote: > Hi, > > I would like to propose the following release candidate (RC2) of Apache > Iceberg Go version v0.1.0. > > This release candidate

[VOTE][Go] Release Apache Iceberg Go v0.1.0 RC2

Hi, I would like to propose the following release candidate (RC2) of Apache Iceberg Go version v0.1.0. This release candidate is based on commit: 0921b84b53e3184a1867481bf1e1a22f5a059b5c [1] The source release rc2 is hosted at [2]. Please download, verify checksums and signatures, run the unit

Re: [VOTE][Go] Release Apache Iceberg Go v0.1.0 RC1

Issue is fixed and will be sending out a vote for RC2. Sorry everyone! On Wed, Nov 13, 2024 at 2:50 PM Kevin Liu wrote: > Hey Matt, > > -1 (non binding) We need a new RC due to the issues below. > > I found a few issues while verifying the RC. > > The first one is minor. The "[2]:" text mentions

Re: [VOTE][Go] Release Apache Iceberg Go v0.1.0 RC1

Hey Matt, -1 (non binding) We need a new RC due to the issues below. I found a few issues while verifying the RC. The first one is minor. The "[2]:" text mentions rc1 but links to rc0. The second is blocking. While running the verification script, I found that the sha256 file contains the wrong

Re: [Proposal] Replicating version-hint onto the file system

Thanks for the proposal Ashvin! I see value in adding this to support the use case of allowing read only access from Snowflake. Currently we push updates with an ALTER TABLE command to synchronize our internally-hosted

Re: [DISCUSS] Spark 3.3 support?

+1 to deprecating and removing it On Wed, Nov 13, 2024 at 5:03 PM Anton Okolnychyi wrote: > What do folks think about our Spark 3.3 support? Spark 3.3.0 was released > in June, 2022. Given the 18 month maintenance period in Spark, it is no > longer maintained. The last release was in December, 2

Re: [DISCUSS] Spark 3.3 support?

+1 to deprecating and removing it Kind regards, Fokko Op wo 13 nov 2024 om 18:02 schreef Jean-Baptiste Onofré : > +1 to deprecating and removing. > > Users can still use previous Iceberg versions if they need Spark 3.3.0 > support. > > Regards > JB > > On Wed, Nov 13, 2024 at 5:02 PM Anton Okoln

[VOTE][Go] Release Apache Iceberg Go v0.1.0 RC1

Hi, I would like to propose the following release candidate (RC1) of Apache Iceberg Go version v0.1.0. This release candidate is based on commit: 8ca0bb7bdf7868bac4aca83f50db047d9bf0277c [1] The source release rc0 is hosted at [2]. Please download, verify checksums and signatures, run the unit

Re: [DISCUSS] Spark 3.3 support?

+1 to deprecating and removing. Users can still use previous Iceberg versions if they need Spark 3.3.0 support. Regards JB On Wed, Nov 13, 2024 at 5:02 PM Anton Okolnychyi wrote: > > What do folks think about our Spark 3.3 support? Spark 3.3.0 was released in > June, 2022. Given the 18 month m

Re: [VOTE][Go] Release Apache Iceberg Go v0.1.0 RC1

Slight fix to the previous email: the text should say "The source release *rc1* is hosted at [2]" The link is correct, it's only the text that needed to be updated. On Wed, Nov 13, 2024 at 11:43 AM Matt Topol wrote: > Hi, > > I would like to propose the following release candidate (RC1) of Apac

Re: [DISCUSS] Spark 3.3 support?

+1 to deprecating and removing it Thanks, Steve Zhang > On Nov 13, 2024, at 11:23 AM, Eduard Tudenhöfner > wrote: > > +1 to deprecating and removing it

Re: [VOTE][Go] Release Apache Iceberg Go v0.1.0 RC0

The updates and fixes have been merged, with help from @Fokko a new RC has been put up. I'll send out the new vote email shortly! Thanks all! On Tue, Nov 12, 2024 at 1:38 PM Eduard Tudenhöfner wrote: > @Matt yes the licensing is a blocker unfortunately, so we should cancel > the RC and start an

[DISCUSS] Spark 3.3 support?

What do folks think about our Spark 3.3 support? Spark 3.3.0 was released in June, 2022. Given the 18 month maintenance period in Spark, it is no longer maintained. The last release was in December, 2023. There is also no active development in the Iceberg repo for that version, except some occasion

Re: [VOTE] Release Apache PyIceberg 0.8.0rc1

+1 (non binding) I checked: - Signature and hash are OK - ASF header present - LICENSE and NOTICE look good Thanks ! Regards JB On Thu, Nov 7, 2024 at 10:57 PM Kevin Liu wrote: > > Hi Everyone, > > I propose that we release the following RC as the official PyIceberg 0.8.0 > release. > > The co

Re: Optionally disable SSL verification for RESTCatalog

Hi Vladimir, Personally, even testing "local" REST catalogs, I'm setting up SSL certificates with a local CA, etc. It's not very painful. That said, I got your point, and I think we can update https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/HTTPClient.java t

Re: Dynamic Flink Iceberg Sink

Thanks for the proposal! I will take a look asap. Regards JB On Mon, Nov 11, 2024 at 6:32 PM Péter Váry wrote: > > Hi Team, > > With Max Michels, we started to work on enhancing the current Iceberg Sink to > allow inserting evolving records into a changing table. > See: > https://docs.google.c

Re: REST catalog removes void transform

Hi Rian, This is not a valid use case per-se, but I noted this when tested different catalog implementations with Trino. In Trino, there hundreds of unit tests verifying various non-trivial use cases. What I noticed, is that some catalogs would retain "void" transforms, while the REST catalog expl

Optionally disable SSL verification for RESTCatalog

Hi, Currently, RESTCatalog can work either over HTTP or HTTPS. In the latter case, Iceberg always performs a fully-fledged check of server certificate. While it is expected in production environments, this is not necessarily ok for nob-prod on-premises REST catalog deployments (DEV, QA, etc). Con

Re: [DISCUSS] Duplicate KEYS files

Hi For context, in multi-subprojects project (like we have in iceberg with Java, Python, rust, go, …) usually we have a single KEYS file gathering the keys from all release managers for the different subprojects. Regards JB Le mar. 12 nov. 2024 à 20:30, Russell Spitzer a écrit : > Yep sorry I

Re: Dynamic Flink Iceberg Sink

Thanks for taking the lead on this, Peter! The Dynamic Iceberg Sink is designed to address several challenges with the current Flink Iceberg sink. It offers three main benefits: 1. *Flexibility in Table Writing*: It allows writing to multiple tables, eliminating the 1:1 sink-to-topic restri