Re: [DISCUSS] Introducing Iceberg Features ?

2025-04-15 Thread Jean-Baptiste Onofré
Hi Xuanwo Thanks for the feedback. Fair enough. Regards JB Le mer. 16 avr. 2025 à 05:44, Xuanwo a écrit : > Hi, JB > > Thank you for starting this discussion. Based on my experience with > Parquet, when a specification allows readers and writers to freely choose > which features to use, it oft

Re: [DISCUSS] FileFormat API proposal

2025-04-15 Thread Péter Váry
Hi Renjie, The first one for the proposed new API is here: https://github.com/apache/iceberg/pull/12774 Thanks, Peter On Wed, Apr 16, 2025, 05:40 Renjie Liu wrote: > Hi, Peter: > > Thanks for the effort. I totally agree with splitting them into smaller > prs to move forward. > > I'm quite intere

Re: [DISCUSS] Introducing Iceberg Features ?

2025-04-15 Thread Xuanwo
Hi, JB Thank you for starting this discussion. Based on my experience with Parquet, when a specification allows readers and writers to freely choose which features to use, it often leads to the entire ecosystem relying on only the minimal feature set. As a result, many valuable features are ove

Re: [DISCUSS] FileFormat API proposal

2025-04-15 Thread Renjie Liu
Hi, Peter: Thanks for the effort. I totally agree with splitting them into smaller prs to move forward. I'm quite interested in this topic, and please ping me in those splitted prs and I'll help to review. On Mon, Apr 14, 2025 at 11:22 PM Jean-Baptiste Onofré wrote: > Hi Peter > > Awesome ! Th

Re: [DISCUSS] Fix CVE-2025-30065 on 1.8.x / 1.7.x / 1.6.x?

2025-04-15 Thread Manu Zhang
Hi Eduard, I’d love to be the release manager of 1.8.2. Thanks, Manu Eduard Tudenhöfner 于2025年4月16日 周三02:04写道: > So it sounds like we may want to do 1.8.2 with the updated Parquet version > as that should be very low risk. > Does anyone want to volunteer and be the release manager for 1.8.2? >

Re: [DISCUSS] Introducing Iceberg Features ?

2025-04-15 Thread Jean-Baptiste Onofré
Thanks for your feedback. I got your points. My question was more about the features that an engine (reader/writer) should support: for v3 it means that an engine will have to implement/support all features from v3 (required features). They can stay on v2 or fully update to v3. That makes sense to

Re: [VOTE] Release Apache Iceberg 1.9.0 RC0

2025-04-15 Thread Jean-Baptiste Onofré
Ah yes agree: I don’t want to block 1.9.0 release. I was surprising to not seeing the distributions. We can include it for next release. Le mar. 15 avr. 2025 à 19:40, Russell Spitzer a écrit : > I agree we should be distributing them, but I wouldn't block 1.9.0 for it > > On Tue, Apr 15, 2025 a

Re: [DISCUSS] Introducing Iceberg Features ?

2025-04-15 Thread Russell Spitzer
I'm not a big fan of this, I am currently a strong supporter of the V3 is V3 approach. This is one of the reasons we decided to make row-lineage mandatory, we want to avoid encouraging engines from selectively adopting requirements. On Tue, Apr 15, 2025 at 1:42 PM Fokko Driesprong wrote: > Hey J

Re: [DISCUSS] Introducing Iceberg Features ?

2025-04-15 Thread Fokko Driesprong
Hey JB, Thanks for raising this. This would be another way of indicating (next to the format version) what's supported. At first glance, I'm reluctant to add this. For two reasons: 1. Because of the added complexity, both from a technical perspective, and because it also might confuse downs

Re: [DISCUSS] Fix CVE-2025-30065 on 1.8.x / 1.7.x / 1.6.x?

2025-04-15 Thread Eduard Tudenhöfner
So it sounds like we may want to do 1.8.2 with the updated Parquet version as that should be very low risk. Does anyone want to volunteer and be the release manager for 1.8.2? On Mon, Apr 14, 2025 at 5:29 PM Ryan Blue wrote: > I agree with Fokko. It's a good idea to get a release out soon that h

Re: [VOTE] Release Apache Iceberg 1.9.0 RC0

2025-04-15 Thread Russell Spitzer
I agree we should be distributing them, but I wouldn't block 1.9.0 for it On Tue, Apr 15, 2025 at 12:12 PM Jean-Baptiste Onofré wrote: > Thanks Russell, I remember this issue. > > My question is: > 1. As we build the distributions (main and hive), do we want to > publish as it is (and let the us

[DISCUSS] Table Identifiers in Iceberg View Spec

2025-04-15 Thread Walaa Eldin Moustafa
Hi Everyone, Starting this thread to resume our discussion on how to reference table identifiers from Iceberg metadata, a key aspect of the view specification, particularly in relation to the MV (materialized view) extensions. I had the chance to speak offline with a few community members to bett

Re: [VOTE] Release Apache Iceberg 1.9.0 RC0

2025-04-15 Thread Jean-Baptiste Onofré
Thanks Russell, I remember this issue. My question is: 1. As we build the distributions (main and hive), do we want to publish as it is (and let the users extend) 2. If 1 is not good enough, should we still build the distributions ? Just wondering ;) Personally, I think we should distribute the d

Re: [VOTE] Release Apache Iceberg 1.9.0 RC0

2025-04-15 Thread Russell Spitzer
We had a ticket about improving kafka connect distribution, https://github.com/apache/iceberg/issues/12507 because the current docs require you to build your own kafka-connect zip. On Mon, Apr 14, 2025 at 8:15 AM Jean-Baptiste Onofré wrote: > Another point that I saw: the kafka-connect runtime d

Re: [VOTE] Release Apache Iceberg 1.9.0 RC0

2025-04-15 Thread Péter Váry
+1 checked the signatures compiled the code run some tests Aihua Xu ezt írta (időpont: 2025. ápr. 14., H, 0:54): > +1 (non-binding). > > Verified against Snowflake build. > > On Sun, Apr 13, 2025 at 12:20 AM Yuya Ebihara < > yuya.ebih...@starburstdata.com> wrote: > >> +1 (non-binding) >> >> Tr