[DISCUSS] Table name in table metadata

Hi all, >From time to time, users ask me about the status of their Iceberg tables by sending me a *path*, which they've received in a file system alert email. Usually I look for the corresponding *table name *and query metadata tables through Spark SQL. However, it's not easy to find the table nam

Re: Proposal: Parquet footer size in Iceberg metadata

Do we want to create data file format specific maps for these metadata to better separate for the differentformats? I don't think having Parquet footer size is relevant for Avro or ORC. On Mon, Feb 10, 2025, 06:52 Xuanwo wrote: > +1 non-binding from me. > > I love this idea. Even though S3 suppo

Re: [VOTE] Simplify multi-arg table metadata

+1 On Mon, Feb 10, 2025, 03:44 Manu Zhang wrote: > +1 (non-binding) > > On Mon, Feb 10, 2025 at 10:25 AM roryqi wrote: > >> +1 >> >> xianjin 于2025年2月10日周一 10:02写道： >> >>> +1 (non-binding) >>> >>> On Mon, Feb 10, 2025 at 2:03 AM Hussein Awala wrote: >>> +1 (non-binding) On Sun,

[VOTE] Release Apache Iceberg 1.8.0 RC0

Hi Everyone, I propose that we release the following RC as the official Apache Iceberg 1.8.0 release. The commit ID is c277c2014a1b37fe755cfe37f173b6465bb8cb73 * This corresponds to the tag: apache-iceberg-1.8.0-rc0 * https://github.com/apache/iceberg/commits/apache-iceberg-1.8.0-rc0 * https://gi

Re: Proposal: Parquet footer size in Iceberg metadata

+1 non-binding from me. I love this idea. Even though S3 supports reading from the tail, this value can still be useful in cases where the size is incorrectly hinted, requiring an additional read for the Parquet footer size. On Mon, Feb 10, 2025, at 09:21, Anton Okolnychyi wrote: > +1 from me,

Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release

There's still https://github.com/apache/iceberg/pull/11216 under 1.8.0 milestone. Do we want to include it? On Sun, Feb 9, 2025 at 3:01 PM Jean-Baptiste Onofré wrote: > Thanks Amogh > > I updated the PR with some cleanups. > > Regards > JB > > On Sun, Feb 9, 2025 at 4:04 AM Amogh Jahagirdar <2am

Re: [VOTE] Simplify multi-arg table metadata

+1 (non-binding) On Mon, Feb 10, 2025 at 10:25 AM roryqi wrote: > +1 > > xianjin 于2025年2月10日周一 10:02写道： > >> +1 (non-binding) >> >> On Mon, Feb 10, 2025 at 2:03 AM Hussein Awala wrote: >> >>> +1 (non-binding) >>> >>> On Sun, Feb 9, 2025 at 6:15 PM Matt Topol >>> wrote: >>> +1 (non-bindin

Re: [VOTE] Simplify multi-arg table metadata

+1 xianjin 于2025年2月10日周一 10:02写道： > +1 (non-binding) > > On Mon, Feb 10, 2025 at 2:03 AM Hussein Awala wrote: > >> +1 (non-binding) >> >> On Sun, Feb 9, 2025 at 6:15 PM Matt Topol wrote: >> >>> +1 (non-binding) >>> >>> Will definitely make it easier for iceberg-go to support v3 :) >>> >>> On S

Re: [VOTE] Simplify multi-arg table metadata

+1 (non-binding) On Mon, Feb 10, 2025 at 2:03 AM Hussein Awala wrote: > +1 (non-binding) > > On Sun, Feb 9, 2025 at 6:15 PM Matt Topol wrote: > >> +1 (non-binding) >> >> Will definitely make it easier for iceberg-go to support v3 :) >> >> On Sun, Feb 9, 2025, 12:14 PM Szehon Ho wrote: >> >>> +

Re: Proposal: Parquet footer size in Iceberg metadata

+1 from me, I'd love to see this implemented (maybe even in V3 if anyone is willing to pick it up?). Eduard and I were discussing DV file compaction where we need to know the ratio of live vs orphan DVs in a particular DV file. Manifests contain sizes of individual DV blobs as well as the total DV

Re: Proposal: Parquet footer size in Iceberg metadata

Hey Sreeram, Sounds like there's a fair amount of interest/support for this. Anton also mentioned that having this information would help estimate orphaned DVs, so there's multiple cases where this would be beneficial. We might want to tie this change to a format version release (even if just an

Re: [VOTE] Simplify multi-arg table metadata

+1 (non-binding) On Sun, Feb 9, 2025 at 6:15 PM Matt Topol wrote: > +1 (non-binding) > > Will definitely make it easier for iceberg-go to support v3 :) > > On Sun, Feb 9, 2025, 12:14 PM Szehon Ho wrote: > >> +1 (binding) >> >> Thanks Fokko! >> Szehon >> >> > On Feb 9, 2025, at 8:14 AM, Jean-Bap

Re: [VOTE] Simplify multi-arg table metadata

+1 (non-binding) Will definitely make it easier for iceberg-go to support v3 :) On Sun, Feb 9, 2025, 12:14 PM Szehon Ho wrote: > +1 (binding) > > Thanks Fokko! > Szehon > > > On Feb 9, 2025, at 8:14 AM, Jean-Baptiste Onofré > wrote: > > > > +1 (non binding) > > > > Thanks to the cat :) > > >

Re: [VOTE] Simplify multi-arg table metadata

+1 (binding) Thanks Fokko! Szehon > On Feb 9, 2025, at 8:14 AM, Jean-Baptiste Onofré wrote: > > +1 (non binding) > > Thanks to the cat :) > > Regards > JB > >> On Sun, Feb 9, 2025 at 10:01 AM Fokko Driesprong wrote: >> >> (Second attempt, the cat ran over the keyboard) >> >> Hey everyone

Re: [VOTE] Simplify multi-arg table metadata

+1 (non binding) Thanks to the cat :) Regards JB On Sun, Feb 9, 2025 at 10:01 AM Fokko Driesprong wrote: > > (Second attempt, the cat ran over the keyboard) > > Hey everyone, > > After the positive responses on the devlist, I would like to raise a vote to > simplify the multi-argument transfor

Re: [VOTE] Simplify multi-arg table metadata

+1 (non-binding) (she says hi to your cat!) Best, Gang On Sun, Feb 9, 2025 at 5:02 PM Fokko Driesprong wrote: > (Second attempt, the cat ran over the keyboard) > > Hey everyone, > > After the positiv

Re: [VOTE] Simplify multi-arg table metadata

(Second attempt, the cat ran over the keyboard) Hey everyone, After the positive responses on the devlist , I would like to raise a vote to simplify the multi-argument transforms metadata and make it exclu

[VOTE] Simplify multi-arg table metadata

Hey everyone, After the positive responses on the devlist , I would like to raise a vote to simplify the multi-argument transforms metadata, and make it exclusve A vote to simplify the