Re: [ANNOUNCE] New committer: Honah J.

[roryqi]

Congrats!

Jun H.  于2024年1月14日周日 01:25写道：

> Congratulations!
>
> On Jan 12, 2024, at 10:12 PM, Péter Váry 
> wrote:
>
> 
> Congratulations!
>
> On Sat, Jan 13, 2024, 06:26 Jean-Baptiste Onofré  wrote:
>
>> Congrats !
>>
>> Regards
>> JB
>>
>> Le ven. 12 janv. 2024 à 22:11, Fokko Driesprong  a
>> écrit :
>>
>>> On behalf of the Iceberg PMC, I'm happy to announce that Honah has
>>> accepted an invitation to become a committer on Apache (Py)Iceberg.
>>> Welcome, and thank you for your contributions!
>>>
>>> Kind regards,
>>> Fokko
>>>
>>

Re: [Vote] Deprecate oauth tokens endpoint

[roryqi]

+1.

Driesprong, Fokko  于2024年7月10日周三 16:29写道：

> +1 (binding)
>
> Op wo 10 jul 2024 om 10:14 schreef Jean-Baptiste Onofré :
>
>> +1 (non binding)
>>
>> NB: a few comments in the PR should be "addressed", but it's OK.
>>
>> Regards
>> JB
>>
>> On Mon, Jul 8, 2024 at 6:15 PM Robert Stupp  wrote:
>> >
>> > Hi Everyone,
>> >
>> > I propose that we merge PR to "Deprecate oauth/tokens endpoint".
>> >
>> > The background and overall plan is discussed on this mailing list [2]
>> > and this google doc [3].
>> >
>> > Please vote in the next 72 hours.
>> >
>> > Robert
>> >
>> >
>> >
>> > [1] https://github.com/apache/iceberg/pull/10603
>> >
>> > [2] https://lists.apache.org/thread/twk84xx7v0xy5q5tfd9x5torgr82vv50
>> and
>> > https://lists.apache.org/thread/wcm9ylm0nbwfrx65n8b1tpjrdhgvcx24 and
>> > https://lists.apache.org/thread/qksh9j9d8h6nt6qrfl47bj76jthddb0p
>> >
>> > [3]
>> >
>> https://docs.google.com/document/d/1Xi5MRk8WdBWFC3N_eSmVcrLhk3yu5nJ9x_wC0ec6kVQ
>> >
>> > --
>> > Robert Stupp
>> > @snazy
>> >
>>
>

Re: [ANNOUNCE] Welcoming new committers and PMC members

[roryqi]

Congrats!

Ye Xianjin  于2024年7月25日周四 04:50写道：

> Congrats all, well done !
>
> Sent from my iPhone
>
> On Jul 24, 2024, at 11:33 PM, Péter Váry 
> wrote:
>
> 
> Congratulations all!
>
> Bryan Keller  ezt írta (időpont: 2024. júl. 24., Sze,
> 16:21):
>
>> Congrats all!
>>
>> On Jul 24, 2024, at 3:14 AM, Eduard Tudenhöfner 
>> wrote:
>>
>> Congrats everyone, it's amazing to see such great people contributing and
>> improving the Iceberg community.
>>
>>
>>
>> On Wed, Jul 24, 2024 at 8:04 AM Honah J.  wrote:
>>
>>> Thank you all! Congratulations to Kevin, Piotr, Sung, Xuanwo, and Renjie!
>>>
>>> It’s truly an honor to contribute to the project and witness the
>>> incredible growth of our community. I’m excited to continue working
>>> together and help drive the project to new heights!
>>>
>>> On Tue, Jul 23, 2024 at 7:24 PM Renjie Liu 
>>> wrote:
>>>
 Thanks everyone! And congratulations to  Pitor, Kevin, Sung, Xuanwo,
 Honah!

 It's awesome to work within and see the growth of this community!

 On Wed, Jul 24, 2024 at 9:41 AM Manu Zhang 
 wrote:

> Congrats everyone! Well done!
>
> Regards,
> Manu
>
> On Wed, Jul 24, 2024 at 4:48 AM Piotr Findeisen <
> piotr.findei...@gmail.com> wrote:
>
>> Dear all,
>> thank you for your trust. Very much appreciated
>>
>> Kevin, Sung, Xuanwo, Honah, Renjie -- congratulations! it's awesome
>> that your efforts were noticed and the value you bring to the table --
>> recognized.
>>
>> Best,
>> Piotr
>>
>>
>>
>> On Tue, 23 Jul 2024 at 18:56, Steve Zhang
>>  wrote:
>>
>>> Congrats everyone!
>>>
>>> Thanks,
>>> Steve Zhang
>>>
>>>
>>>
>>> On Jul 23, 2024, at 9:20 AM, Anton Okolnychyi 
>>> wrote:
>>>
>>> Congrats everyone!
>>>
>>>
>>>
>>

Re: Welcome Péter, Amogh and Eduard to the Apache Iceberg PMC

[roryqi]

Congrats!

Steven Wu  于2024年8月14日周三 11:12写道：

> Congratulations!
>
> On Tue, Aug 13, 2024 at 7:18 PM Kevin Liu  wrote:
>
>> Congratulations all!
>>
>> On Wed, Aug 14, 2024 at 9:38 AM Jun H.  wrote:
>>
>>> Congratulations!
>>>
>>>
>>> On Tue, Aug 13, 2024 at 4:28 PM Rodrigo Meneses 
>>> wrote:
>>>
 This is amazing! Congratulations!!!

 On Tue, Aug 13, 2024 at 3:07 PM karuppayya 
 wrote:

> Congratulations, everyone!
>
> On Tue, Aug 13, 2024 at 2:54 PM Sung Yun  wrote:
>
>> Congratulations Péter, Amogh and Eduard!
>>
>> On 2024/08/13 20:25:17 Russell Spitzer wrote:
>> > Hi Y'all,
>> >
>> > It is my pleasure to let everyone know that the Iceberg PMC has
>> voted to
>> > have several talented individuals join us.
>> >
>> > So without further ado, please welcome Péter Váry, Amogh Jahagirdar
>> and
>> > Eduard Tudenhoefner to the Apache Iceberg PMC.
>> >
>> > As usual I am excited about the future of this community and
>> thankful for
>> > the hard work and stewardship of its members.
>> >
>> > Thank you for your time,
>> > Russell Spitzer
>> >
>>
>

Re: [VOTE] Merge REST Spec change to add RemovePartitionSpecsUpdate update type

[roryqi]

+1

Manu Zhang  于2024年8月27日周二 11:44写道：

> +1 (non-binding)
>
> On Tue, Aug 27, 2024 at 11:00 AM xianjin  wrote:
>
>> +1 (non-binding)
>> Sent from my iPhone
>>
>> On Aug 27, 2024, at 4:22 AM, Fokko Driesprong  wrote:
>>
>> 
>> +1
>>
>> Op ma 26 aug 2024 om 22:00 schreef Yufei Gu :
>>
>>> +1
>>> Yufei
>>>
>>>
>>> On Mon, Aug 26, 2024 at 11:06 AM Ryan Blue 
>>> wrote:
>>>
 +1

 On Mon, Aug 26, 2024 at 11:04 AM Amogh Jahagirdar <2am...@gmail.com>
 wrote:

> I've opened a PR [1] to add a RemovePartitionSpecsUpdate update type
> so that removing partition specs update operation against REST catalogs 
> can
> be supported. See the discuss thread [2] for more context.
>
> Please vote on merging this change.  The vote will remain open for at
> least 72 hours.
>
> [] +1
> [] +0
> [] -1, do not merge because ...
>
> [1] https://github.com/apache/iceberg/pull/10846
> [2] https://lists.apache.org/thread/2lo59wkhn8cm29wmokfmg2f18934qnln
>
> Thanks,
>
> Amogh Jahagirdar
>


 --
 Ryan Blue
 Databricks

>>>

Re: [DISCUSS] Remove iceberg-pig module ?

[roryqi]

+1.

Péter Váry  于2024年10月18日周五 13:44写道：

> +1
>
> On Fri, Oct 18, 2024, 04:50 Manu Zhang  wrote:
>
>> +1
>>
>> On Fri, Oct 18, 2024 at 8:50 AM Rodrigo Meneses 
>> wrote:
>>
>>> +1
>>>
>>> On Thu, Oct 17, 2024 at 4:38 PM Bryan Keller  wrote:
>>>
 +1


 On Oct 17, 2024, at 1:51 PM, Anton Okolnychyi 
 wrote:

 +1

 чт, 17 жовт. 2024 р. о 13:42 Steven Wu  пише:

> +1
>
> On Thu, Oct 17, 2024 at 10:44 AM John Zhuge  wrote:
>
>> +1 (non-binding)
>>
>> On Thu, Oct 17, 2024 at 10:21 AM Yufei Gu 
>> wrote:
>>
>>> +1 for deprecating it in 1.7
>>> Yufei
>>>
>>>
>>> On Thu, Oct 17, 2024 at 9:51 AM Ajantha Bhat 
>>> wrote:
>>>
 +1 for dropping it.

 On Thu, Oct 17, 2024 at 8:55 PM Daniel Weeks 
 wrote:

> +1 for deprecating and dropping
>
> On Thu, Oct 17, 2024 at 7:46 AM Eduard Tudenhöfner <
> etudenhoef...@apache.org> wrote:
>
>> +1 for marking the project deprecated (in 1.7.0) and dropping it
>> in the next release (1.8.0)
>>
>> On Thu, Oct 17, 2024 at 4:36 PM Russell Spitzer <
>> russell.spit...@gmail.com> wrote:
>>
>>> +1 (oink)
>>>
>>> If anyone really cares please chime in but seriously we should
>>> drop it
>>>
>>> On Thu, Oct 17, 2024 at 8:07 AM Jean-Baptiste Onofré <
>>> j...@nanthrax.net> wrote:
>>>
 Hi folks,

 Even if it seems the project is pretty close to 0.18 release,
 Apache
 Pig is a "dormant" project.

 I would like to discuss here if it would not make sense to
 remove the
 iceberg-pig module.

 Thoughts ?

 Regards
 JB

>>>
>>
>> --
>> John Zhuge
>>
>

Re: [DISCUSS] Spark 3.3 support?

[roryqi]

+1 to deprecate it and remove it.

Yufei Gu  于2024年11月19日周二 15:32写道：
>
> +1 to deprecate it and remove it.
>
> Yufei
>
>
> On Wed, Nov 13, 2024 at 9:17 AM Fokko Driesprong  wrote:
>>
>> +1 to deprecating and removing it
>>
>> Kind regards,
>> Fokko
>>
>> Op wo 13 nov 2024 om 18:02 schreef Jean-Baptiste Onofré :
>>>
>>> +1 to deprecating and removing.
>>>
>>> Users can still use previous Iceberg versions if they need Spark 3.3.0 
>>> support.
>>>
>>> Regards
>>> JB
>>>
>>> On Wed, Nov 13, 2024 at 5:02 PM Anton Okolnychyi  
>>> wrote:
>>> >
>>> > What do folks think about our Spark 3.3 support? Spark 3.3.0 was released 
>>> > in June, 2022. Given the 18 month maintenance period in Spark, it is no 
>>> > longer maintained. The last release was in December, 2023. There is also 
>>> > no active development in the Iceberg repo for that version, except some 
>>> > occasional cherry-picks. Shall we deprecate in 1.8 and remove in 1.9?
>>> >
>>> > - Anton

Re: [VOTE] Simplify multi-arg table metadata

[roryqi]

+1

xianjin  于2025年2月10日周一 10:02写道：

> +1 (non-binding)
>
> On Mon, Feb 10, 2025 at 2:03 AM Hussein Awala  wrote:
>
>> +1 (non-binding)
>>
>> On Sun, Feb 9, 2025 at 6:15 PM Matt Topol  wrote:
>>
>>> +1 (non-binding)
>>>
>>> Will definitely make it easier for iceberg-go to support v3 :)
>>>
>>> On Sun, Feb 9, 2025, 12:14 PM Szehon Ho  wrote:
>>>
 +1 (binding)

 Thanks Fokko!
 Szehon

 > On Feb 9, 2025, at 8:14 AM, Jean-Baptiste Onofré 
 wrote:
 >
 > +1 (non binding)
 >
 > Thanks to the cat :)
 >
 > Regards
 > JB
 >
 >> On Sun, Feb 9, 2025 at 10:01 AM Fokko Driesprong 
 wrote:
 >>
 >> (Second attempt, the cat ran over the keyboard)
 >>
 >> Hey everyone,
 >>
 >> After the positive responses on the devlist, I would like to raise a
 vote to simplify the multi-argument transforms metadata and make it
 exclusive for V3+ tables. The corresponding PR can be found here.
 >>
 >> This vote will be open for at least 72 hours.
 >>
 >> [ ] +1 Update the metadata to remove multi-arg transforms for V1 and
 V2 tables
 >> [ ] +0
 >> [ ] -1 I have questions and/or concerns
 >>
 >> Kind regards,
 >> Fokko
 >>
 >> Op zo 9 feb 2025 om 09:57 schreef Driesprong, Fokko
 :
 >>>
 >>> Hey everyone,
 >>>
 >>> After the positive responses on the devlist, I would like to raise
 a vote to simplify the multi-argument transforms metadata, and make it
 exclusve
 >>>
 >>> A vote to simplify the

>>>

Re: Welcome Huaxin Gao as a committer!

[roryqi]

Congrats!

Anton Okolnychyi  于2025年2月7日周五 09:22写道：

> Thanks for the contributions, Huaxin! Congrats!
>
> чт, 6 лют. 2025 р. о 13:55 Daniel Weeks  пише:
>
>> Congrats Huaxin!
>>
>> On Thu, Feb 6, 2025 at 11:52 AM Anurag Mantripragada
>>  wrote:
>>
>>> Congratulations, Huaxin! Great work!
>>>
>>> ~ Anurag Mantripragada
>>>
>>>
>>>
>>> On Feb 6, 2025, at 10:36 AM, Honah J.  wrote:
>>>
>>> Congratulations Huaxin!
>>>
>>> On Thu, Feb 6, 2025 at 10:32 AM John Zhuge  wrote:
>>>
 Congratulations Huaxin!

 On Thu, Feb 6, 2025 at 9:52 AM Huang-Hsiang Cheng
  wrote:

> Congratulations Huaxin!
>
> Huang-Hsiang
>
> On Feb 6, 2025, at 9:32 AM, Wing Yew Poon 
> wrote:
>
> Congratulations Huaxin! Awesome!
>
>
> On Thu, Feb 6, 2025 at 9:27 AM Yufei Gu  wrote:
>
>> Congrats Huaxin!
>>
>> Yufei
>>
>>
>> On Thu, Feb 6, 2025 at 9:09 AM Steve Zhang
>>  wrote:
>>
>>> Congratulations Huaxin, well deserved!
>>>
>>> Thanks,
>>> Steve Zhang
>>>
>>>
>>>
>>> On Feb 6, 2025, at 8:16 AM, Xingyuan Lin 
>>> wrote:
>>>
>>> Congrats Huaxin!
>>>
>>> On Thu, Feb 6, 2025 at 11:11 AM Denny Lee 
>>> wrote:
>>>
 Congratulations Huaxin!!!

 On Thu, Feb 6, 2025 at 7:47 AM Amogh Jahagirdar <2am...@gmail.com>
 wrote:

> Congratulations Huaxin!
>
> On Thu, Feb 6, 2025 at 8:41 AM Kevin Liu 
> wrote:
>
>> Congratulations Huaxin!! Looking forward to working together 🎉
>> 
>>
>> Best,
>> Kevin Liu
>>
>> On Thu, Feb 6, 2025 at 7:30 AM Prashant Singh <
>> prashant010...@gmail.com> wrote:
>>
>>> Congratulations Huaxin !
>>>
>>> Best,
>>> Prashant Singh
>>>
>>>
>>> On Thu, Feb 6, 2025 at 7:25 AM himadri pal 
>>> wrote:
>>>
 Congratulations Huaxin.

 On Thu, Feb 6, 2025 at 6:45 AM Sung Yun 
 wrote:

> That's fantastic news Huaxin. Congratulations!
>
> On 2025/02/06 13:40:09 Rodrigo Meneses wrote:
> > Congrats and best wishes !!!
> >
> > On Thu, Feb 6, 2025 at 5:04 AM Gidon Gershinsky <
> gg5...@gmail.com> wrote:
> >
> > > Congrats Huaxin!
> > >
> > > Cheers, Gidon
> > >
> > >
> > > On Thu, Feb 6, 2025 at 2:46 PM Tushar Choudhary <
> > > tushar.choudhary...@gmail.com> wrote:
> > >
> > >> Congratulations Husain!
> > >>
> > >> Cheers,
> > >> Tushar Choudhary
> > >>
> > >>
> > >> On Thu, 6 Feb 2025 at 6:15 PM, xianjin <
> xian...@apache.org> wrote:
> > >>
> > >>> Congrats huaxin!
> > >>> Sent from my iPhone
> > >>>
> > >>> On Feb 6, 2025, at 7:35 PM, Fokko Driesprong <
> fo...@apache.org> wrote:
> > >>>
> > >>> 
> > >>>
> > >>> Congratulations Huaxin!
> > >>>
> > >>> Op do 6 feb 2025 om 12:21 schreef Russell Spitzer <
> > >>> russell.spit...@gmail.com>:
> > >>>
> >  Congratulations!
> > 
> >  On Thu, Feb 6, 2025 at 11:35 AM Péter Váry <
> peter.vary.apa...@gmail.com>
> >  wrote:
> > 
> > > Congratulations!
> > >
> > > Matt Topol  ezt írta
> (időpont: 2025. febr.
> > > 6., Cs, 10:40):
> > >
> > >> Congrats! Welcome!
> > >>
> > >> On Thu, Feb 6, 2025, 10:19 AM Raúl Cumplido <
> rau...@apache.org>
> > >> wrote:
> > >>
> > >>> Congrats Huaxin!
> > >>>
> > >>> El jue, 6 feb 2025 a las 10:16, Gang Wu (<
> ust...@gmail.com>)
> > >>> escribió:
> > >>>
> >  Congrats Huaxin!
> > 
> >  Best,
> >  Gang
> > 
> >  On Thu, Feb 6, 2025 at 5:10 PM Szehon Ho <
> szehon.apa...@gmail.com>
> >  wrote:
> > 
> > > Hi everyone,
> > >
> > > The Project Management Committee (PMC) for Apache
> Iceberg has
> > > invited Huaxin Gao to become a committer, and I am
> happy to
> > > announce that she has accepted.  Huaxin h

Re: [VOTE] Update partition stats spec for V3

[roryqi]

+1

Amogh Jahagirdar <2am...@gmail.com> 于2025年2月2日周日 10:16写道：

> +1
>
> On Sat, Feb 1, 2025 at 11:05 AM huaxin gao  wrote:
>
>> +1 (non-binding)
>>
>> On Sat, Feb 1, 2025 at 8:50 AM Manish Malhotra <
>> manish.malhotra.w...@gmail.com> wrote:
>>
>>> +1(nonbinding)
>>>
>>> On Sat, Feb 1, 2025 at 2:49 AM Russell Spitzer <
>>> russell.spit...@gmail.com> wrote:
>>>
 +1

 On Sat, Feb 1, 2025 at 3:01 AM Anton Okolnychyi 
 wrote:

> Hi all,
>
> I propose the following updates to our partition stats spec in V3:
>
> - Modify `position_delete_record_count` to include a sum of position
> deletes across position delete files and DVs
> - Keep `position_delete_file_count` to represent the number of
> position delete files (ignoring DVs)
> - Add `dv_count` to represent the number of DVs
> - Make delete counts required to avoid ambiguity w.r.t NULL vs unknown.
>
> Here is the PR with the spec update:
> https://github.com/apache/iceberg/pull/12098
>
> - Anton
>

Re: New committer: Scott Donnelly

[roryqi]

Congrats!

Fenil Jain  于2024年12月11日周三 14:26写道：

> Congratulations Scott!
>
> On Wed, Dec 11, 2024 at 8:56 AM Renjie Liu 
> wrote:
> >
> > Hey everyone,
> >
> > The Project Management Committee (PMC) for Apache Iceberg has invited
> Scott Donnelly to become a committer. Scott did a lot of impressive work in
> iceberg-rust, and we are pleased to announce that he has accepted.
> >
> > Please join us in welcoming Scott to hist new role and responsibility in
> our project community.
>

Re: [DISCUSS] Standardizing Error Handling in the Iceberg Spark Module

[roryqi]

Great work!

Rodrigo Meneses  于2024年12月20日周五 10:38写道：

> Thanks a lots it’s a great improvement. I’ll check what’s the current
> behavior that we have on flink with respect to error handling !
>
> On Thu, Dec 19, 2024 at 3:46 PM Anurag Mantripragada
>  wrote:
>
>> This is great. +1 from me. Thanks Huaxin.
>>
>>
>> Anurag Mantripragada
>>
>>
>>
>>
>>
>>
>> On Dec 19, 2024, at 12:39 PM, Prashant Singh 
>> wrote:
>>
>> +1, Thanks, Huaxin !
>>
>> On Thu, Dec 19, 2024 at 10:41 AM John Zhuge  wrote:
>>
>>> +1 Thanks
>>>
>>> John Zhuge
>>>
>>>
>>> On Thu, Dec 19, 2024 at 9:47 AM Yufei Gu  wrote:
>>>
 +1 on the direction. It's great that Spark has standardized the error
 code so that Iceberg didn't have to rely on error messages.

 Yufei


 On Thu, Dec 19, 2024 at 8:47 AM rdb...@gmail.com 
 wrote:

> This looks like a good improvement to me. Thanks, Huaxin!
>
> On Wed, Dec 18, 2024 at 11:37 PM huaxin gao 
> wrote:
>
>> Hi everyone,
>>
>> While working on integrating Spark 4.0 with Iceberg, I noticed that
>> error conditions in the Spark module are primarily validated through the
>> content of error messages. I need to revise some of the validation 
>> because
>> the error messages have changed in Spark 4.0. Spark has standardized 
>> error
>> handling by introducing error classes and SQLSTATE codes since 3.1. I 
>> would
>> like to align the error handling in the Iceberg Spark module with Spark's
>> standard error handling framework, specifically by shifting from 
>> validating
>> error message content to validating error classes and SQLSTATE codes.  I
>> have prepared a quick write-up
>> 
>> for background information and an example. Please let me know what you
>> think. If there are no objections to this proposal, I will begin updating
>> the error handling in the Spark module.
>>
>> Thanks,
>> Huaxin
>>
>>
>
>>

Re: Table metadata swap not work for REST Catalog (#12134)

[roryqi]

Agree. Gravitino‘s Iceberg REST service provides similar mechanism to
notify the changes. You can see
https://gravitino.apache.org/docs/0.8.0-incubating/iceberg-rest-service#event-listener

Yufei Gu  于2025年2月11日周二 10:01写道：

> The push-based mirroring highlighted by Ryan is a popular use case.
> Polaris has already implemented notification APIs to address this, and
> there have been several Iceberg community discussions surrounding it. If I
> recall correctly, we generally agreed that the notification API is
> beneficial per last discussion. Polaris supports various notification types(
> https://github.com/apache/polaris/blob/be343f1b484d2d79877978ff55f3ad36031dc4a5/spec/polaris-catalog-apis/notifications-api.yaml#L108).
> The "UPDATE" type provides the same functionality as "register table
> force". I believe using notification APIs, or a similar concept is the
> right approach for achieving push-based mirroring, as "register table
> force" on its own isn't sufficient. Here is a list of notification types
> used in Polaris for reference:
> - CREATE
> - UPDATE
> - DROP
> - VALIDATE
>
> Yufei
>
>
> On Mon, Feb 10, 2025 at 4:49 PM Steve Zhang
>  wrote:
>
>> Thank you Russell and Ryan.
>>
>>   Let me start to work on a new API to support force table registration
>> in catalog.
>>
>> Thanks,
>> Steve Zhang
>>
>>
>>
>> On Feb 10, 2025, at 4:29 PM, rdb...@gmail.com wrote:
>>
>> Yeah, it sounds like a "register table force" is the right concept here.
>> I think we want to make sure that table updates remain change-based as
>> the best practice in the REST API. But there are some irregular use cases
>> that justify having some mechanism to completely replace the state (like
>> push-based mirroring). I think it makes sense to revisit mirroring and this
>> use case and come up with a path forward.
>>
>> On Mon, Feb 10, 2025 at 3:12 PM Russell Spitzer <
>> russell.spit...@gmail.com> wrote:
>>
>>> I still would like a "register table" force" option
>>>
>>> On Mon, Feb 10, 2025 at 5:06 PM Steve Zhang
>>>  wrote:
>>>
 Thank you Dan for your detailed reply. Based on your explanation, do
 you think it would be worthwhile to support non-linear or complete metadata
 replacements in the REST implementation? I am happy to contribute but might
 need some guidance from the community on the best approach.

 For additional context, we explored into the workaround of using a
 combination of dropping table and re-registering the table with concerns of
 reading in between. There’s also an attempt to add a force option to the
 register-table API (https://github.com/apache/iceberg/pull/5327),
 which would allow for metadata swap on an existing table. However, it was
 suggested that use TableOperations.commit(base, new) is preferred to
 achieve atomicity.

 Thanks,
 Steve Zhang



 On Feb 10, 2025, at 1:49 PM, Daniel Weeks  wrote:

 Hey Steve,

 I think the issue here is that you're using the commit api in table
 operations to perform a non-incremental/linear change to the metadata.  The
 REST implementation is a little more strict in that it builds a set of
 updates based on the mutations made to the metadata and the commit process
 applies those changes.  In this scenario, no changes have been made and the
 call is attempting a complete replacement.

 The other implementations are just blindly swapping the location, so
 while that operation does achieve the effect you're looking for, it's not
 the right semantics for the commit.

 You might want to consider using the "register table" operation
 instead, which takes the table identifier and location to perform this type
 of swap.

 -Dan

 On Fri, Feb 7, 2025 at 10:17 AM Steve Zhang
  wrote:

> Hey Iceberg Experts:
>
>   I am seeking assistance and insights regarding an issue we’ve
> encountered with RESTTableOperations and its inability to support 
> on-demand
> table metadata swaps. We are currently adopting the REST-based catalog 
> from
> Hive and have noticed a potential gap in the TableOperations.commit()
> API. Typically, we use the commit API to revert a table to a previously
> known state, as demonstrated below:
>
> String deisredMetadataPath =
> "/var/newdb/table/metadata/3-579b23d1-4ca5-4acf-85ec-081e1699cb83.metadata.json""
> ops.commit(ops.current(), TableMetadataParser.read(ops.io(),
> dedeisredMetadataPath));
>
>   However, this approach is no longer working with the REST-based
> catalog. I suspect that the issue may be related to how the update type is
> modeled in RESTTableOperations.  I have shared a unit test that reproduces
> the problem on https://github.com/apache/iceberg/issues/12134, where
> it works on JDBC and in-memory catalogs, but not with RESTCatalog.
>
> Best Regard