Re: [DISCUSS] Apache Iceberg 1.10.0 release

Could we at least get the new File Format API in. Not the actual implementation, but the API itself? I would love to move forward with it, but I still need reviews from the fellow community members. Thanks, Peter On Tue, May 27, 2025, 20:26 Russell Spitzer wrote: > Thanks Steven! I know we are

Re: [DISCUSS] Enabling more Meetups

Hi Ryan I just wanted to remind what should be done for meetups (a kind of context), especially: "The VP, Brand Management, or the VP of the relevant Apache project, must approve the use of Apache marks in any events run by third parties. This includes conferences, meetups, get-togethers, and any

Re: [VOTE] [REST SPEC] Add row lineage fields.

Thanks everyone who participated in the vote for [REST SPEC] Add row lineage fields The vote result is: +1: 5 (binding), 2 (non-binding). {Steven, Eduard, Peter, Yufei, Daniel} | {JB, Huaxin} +0: 0 (binding), 0 (non-binding) -1: 0 (binding), 0 (non-binding) Therefore, the vote is passed. Would

Re: [VOTE] Release Apache Iceberg Rust 0.5.1 RC1

+1 (non-binding) [x] Download links are valid. [x] Checksums and signatures. [x] LICENSE/NOTICE files exist [x] No unexpected binary files [x] All source files have ASF headers [x] Can compile from source On Tue, May 27, 2025 at 9:17 AM NOTME ZE wrote: > +1 non-binding > > [x] Download links a

[DISCUSS] Proposal for Relative Path Support In Table Spec

Hi, Iceberg Community, As mentioned at the last sync, Dan and I have been working on a proposal to add support for relative paths, which has been a long requested feature. There have been a number of discussions/proposals over the years, but we'd like to scope down and refocus effort to make some

[RESULT][VOTE] Release Apache Iceberg 1.9.1 RC1

Thanks everyone who participated in the vote for Release Apache Iceberg 1.9.1 RC1. The vote result is: +1: 3 (binding), 5 (non-binding). {Russell, Steven Wu, Fokko} | {JB, Karuppayya, Kevin, Huaxin, Aihua} +0: 0 (binding), 0 (non-binding) -1: 0 (binding), 0 (non-binding) Therefore, the release

Re: [VOTE] Release Apache Iceberg 1.9.1 RC1

I'm +1 (Binding) as well so I'll close out this vote On Tue, May 27, 2025 at 1:28 PM Aihua Xu wrote: > +1 (non-binding). > > Verified against Snowflake engine. > > On Tue, May 27, 2025 at 9:28 AM huaxin gao wrote: > >> +1 (non-binding) >> Verified signature, checksum, license and ran some tests

Re: [DISCUSS] Apache Iceberg 1.10.0 release

Thanks Steven! I know we are going to have a busy June but I think pretty much everything (except Geo Types) is close to being ready in the reference Spark implementation. On Tue, May 27, 2025 at 1:08 PM Steven Wu wrote: > > As discussed in the community sync, we are planning for the next 1.10.0

Re: [VOTE] Release Apache Iceberg 1.9.1 RC1

+1 (non-binding). Verified against Snowflake engine. On Tue, May 27, 2025 at 9:28 AM huaxin gao wrote: > +1 (non-binding) > Verified signature, checksum, license and ran some tests. > > On Tue, May 27, 2025 at 9:06 AM Russell Spitzer > wrote: > >> For all those who haven't seen this before, GP

[DISCUSS] Apache Iceberg 1.10.0 release

As discussed in the community sync, we are planning for the next 1.10.0 release. I will serve as the release manager after chatting with Russel (the original RM volunteer). The adoption of V3 spec changes

Re: Wide tables in V4

I feel like we have two different issues we're talking about here that aren't necessarily tied (though solutions may address both): 1) wide tables, 2) adding columns Wide tables are definitely a problem where parquet has limitations. I'm optimistic about the ongoing work to help improve parquet fo

Re: Wide tables in V4

Yes having to rewrite the whole file is not ideal but I believe most of the cost of rewriting a file comes from decompression, encoding, stats calculations etc. If you are adding new values for some columns but are keeping the rest of the columns the same in the file, then a bunch of rewrite cost c

Re: Wide tables in V4

Point definitely taken. We really should probably POC some of these ideas and see what we are actually dealing with. (He said without volunteering to do the work :P) On Tue, May 27, 2025 at 11:55 AM Selcuk Aya wrote: > Yes having to rewrite the whole file is not ideal but I believe most of > the

Re: Wide tables in V4

I think that "after the fact" modification is one of the requirements here, IE: Updating a single column without rewriting the whole file. If we have to write new metadata for the file aren't we in the same boat as having to rewrite the whole file? On Tue, May 27, 2025 at 11:27 AM Selcuk Aya wrot

Re: [VOTE] [REST SPEC] Add row lineage fields.

+1 (binding) On Fri, May 23, 2025 at 9:49 AM huaxin gao wrote: > +1 (non-binding) > > On Fri, May 23, 2025 at 9:20 AM Yufei Gu wrote: > >> +1 (binding) >> Yufei >> >> >> On Fri, May 23, 2025 at 9:18 AM Jean-Baptiste Onofré >> wrote: >> >>> +1 (non binding) >>> >>> Regards >>> JB >>> >>> Le ven

Re: [VOTE] Release Apache Iceberg 1.9.1 RC1

+1 (non-binding) Verified signature, checksum, license and ran some tests. On Tue, May 27, 2025 at 9:06 AM Russell Spitzer wrote: > For all those who haven't seen this before, GPG key signing is a very > "early hacker" sort of thing. The idea is the only way to trust a signature > is to > have i

Re: Wide tables in V4

If files represent column projections of a table rather than the whole columns in the table, then any read that reads across these files needs to identify what constitutes a row. Lance DB for example has vertical partitioning across columns but also horizontal partitioning across rows such that in

Re: Wide tables in V4

There's a `file_path` field in the parquet ColumnChunk structure, https://github.com/apache/parquet-format/blob/apache-parquet-format-2.11.0/src/main/thrift/parquet.thrift#L959-L962 I'm not sure what tooling actually supports this though. Could be interesting to see what the history of this is. ht

Re: [VOTE] Release Apache Iceberg Rust 0.5.1 RC1

+1 non-binding [x] Download links are valid. [x] Checksums and signatures. [x] LICENSE/NOTICE files exist [x] No unexpected binary files [x] All source files have ASF headers [x] Can compile from source Christian Thiel 于2025年5月27日周二 16:10写道： > +1 non-binding > > [x] Download links are valid. >

Re: [VOTE] Release Apache Iceberg 1.9.1 RC1

For all those who haven't seen this before, GPG key signing is a very "early hacker" sort of thing. The idea is the only way to trust a signature is to have it signed by someone that you also trust. This builds a network of trust so you could essentially do something like say I trust that key X is

Re: Wide tables in V4

I have to agree that while there can be some fixes in Parquet, we fundamentally need a way to split a "row group" or something like that between separate files. If that's something we can do in the parquet project that would be great but it feels like we need to start exploring more drastic options

Re: [DISCUSS] Enabling more Meetups

JB, can you give us a bit more context about why you're recommending those pages? Do they have policies that already do what is being suggested? Do they impose limits that mean we could not do this? Without that information, I'm not sure what I'm looking for in those docs. On Sat, May 24, 2025 at

Re: [VOTE] Release Apache Iceberg 1.9.1 RC1

+1 (non-binding) - Verified signature, checksum, license. * Build + test passed using Java 17 on M1 * Ran a few examples on Spark * Ran pyiceberg integration tests Best, Kevin Liu On Tue, May 27, 2025 at 7:59 AM karuppayya wrote: > When verifying >

Re: [VOTE] Release Apache Iceberg 1.9.1 RC1

When verifying signatures. I got a warning. Am I missing something with the gpg configuration? gpg: assuming signed data in 'apache-iceberg-1.9.1.tar.gz' gpg: Signature made Wed May 21 15:19:17 2025 PDT gpg:using RSA

Re: [VOTE][Go] Release Apache Iceberg Go v0.3.0 RC0

+1 (non binding) I checked: * signature and checksum are good * no binary found in the source distribution * ASF header in all expected files * LICENSE and NOTICE look good (nit: NOTICE copyright should be updated I guess and also not sure if the "This product includes software developed at The Ap

Re: [VOTE] Release Apache Iceberg 1.9.1 RC1

+1 (non binding) I checked: * source distribution ** checksum and signature are good ** LICENSE and NOTICE look good ** No binary file found in the source distribution ** Header looks good in files ** Build works from the source distribution ** Tested with Spark and Polaris * in the bundled jar fi

Re: [VOTE][Go] Release Apache Iceberg Go v0.3.0 RC0

+1 (binding) Thanks for running this Matt! Kind regards, Fokko Op ma 26 mei 2025 om 12:22 schreef Eduard Tudenhöfner < etudenhoef...@apache.org>: > +1 (binding) > > Thanks everyone! > > On Thu, May 22, 2025 at 12:19 AM Leon Lin wrote: > >> +1 (non-binding) >> >> Thank you Matt for running the

Re: [VOTE] Release Apache Iceberg 1.9.1 RC1

+1 (binding) Ran signatures/checksums/tests/license Kind regards, Fokko Op di 27 mei 2025 om 14:48 schreef Péter Váry : > +1 (binding) > tested signatures, built, run some tests > > Ajantha Bhat ezt írta (időpont: 2025. máj. 27., > K, 14:40): > >> +1 (non-binding) >> >> * validated checksum an

Re: [VOTE] Release Apache Iceberg 1.9.1 RC1

+1 (binding) tested signatures, built, run some tests Ajantha Bhat ezt írta (időpont: 2025. máj. 27., K, 14:40): > +1 (non-binding) > > * validated checksum and signature > * checked license docs & ran RAT checks > * ran build and tests > > - Ajantha > > > On Tue, May 27, 2025 at 7:19 AM Yuya Eb

Re: [VOTE] Release Apache Iceberg 1.9.1 RC1

+1 (non-binding) * validated checksum and signature * checked license docs & ran RAT checks * ran build and tests - Ajantha On Tue, May 27, 2025 at 7:19 AM Yuya Ebihara wrote: > +1 (non-binding) > > Confirmed that Trino and Starburst CI are green. > It runs tests against several catalogs, inc

Re: Discuss proposal - IRC APIs for Multi-Statement Multi-Table Transactions

Forgot to attach a link to the update proposal . On Tue, May 27, 2025 at 1:06 AM Maninderjit Singh < parmar.maninder...@gmail.com> wrote: > Hi community, > > I have updated

Re: [VOTE] Release Apache Iceberg Rust 0.5.1 RC1

+1 non-binding [x] Download links are valid. [x] Checksums and signatures. [x] LICENSE/NOTICE files exist [x] No unexpected binary files [x] All source files have ASF headers [x] Can compile from source Tested icebrg-core in our Pipelines On Tue, 27 May 2025 at 09:18, Xuanwo wrote: > +1 non-bi

Re: [DISCUSS] Events Endpoint for IRC

Dear all, I think we have reached mostly consensus here. There is one more change since our last discussion: We removed the recursive "assumed-by" field of actors in favor or an "actor-chain" list. If there is any more need for discussion please voice it here on the Mailing List or in the Catalog

Re: Discuss proposal - IRC APIs for Multi-Statement Multi-Table Transactions

Hi community, I have updated the proposal with both the options (overwriting existing timestamps-ms vs introducing a new sequence/timestamp field) as we have initial consensus on using catalog authored sequence/timestamp. Jagdeep, please review to ensure that the options are correctly captured. I

Re: [VOTE] Release Apache Iceberg Rust 0.5.1 RC1

+1 non-binding [x] Download links are valid. [x] Checksums and signatures. [x] LICENSE/NOTICE files exist [x] No unexpected binary files [x] All source files have ASF headers [x] Can compile from source Tested icebrg-core and iceberg-rest. On Tue, May 27, 2025, at 13:54, Renjie Liu wrote: > +1 (