Hey Steve, I think the issue here is that you're using the commit api in table operations to perform a non-incremental/linear change to the metadata. The REST implementation is a little more strict in that it builds a set of updates based on the mutations made to the metadata and the commit process applies those changes. In this scenario, no changes have been made and the call is attempting a complete replacement.
The other implementations are just blindly swapping the location, so while that operation does achieve the effect you're looking for, it's not the right semantics for the commit. You might want to consider using the "register table" operation instead, which takes the table identifier and location to perform this type of swap. -Dan On Fri, Feb 7, 2025 at 10:17 AM Steve Zhang <hongyue_zh...@apple.com.invalid> wrote: > Hey Iceberg Experts: > > I am seeking assistance and insights regarding an issue we’ve > encountered with RESTTableOperations and its inability to support on-demand > table metadata swaps. We are currently adopting the REST-based catalog from > Hive and have noticed a potential gap in the TableOperations.commit() > API. Typically, we use the commit API to revert a table to a previously > known state, as demonstrated below: > > String deisredMetadataPath = > "/var/newdb/table/metadata/00003-579b23d1-4ca5-4acf-85ec-081e1699cb83.metadata.json"" > ops.commit(ops.current(), TableMetadataParser.read(ops.io(), > dedeisredMetadataPath)); > > However, this approach is no longer working with the REST-based catalog. > I suspect that the issue may be related to how the update type is modeled > in RESTTableOperations. I have shared a unit test that reproduces the > problem on https://github.com/apache/iceberg/issues/12134, where it works > on JDBC and in-memory catalogs, but not with RESTCatalog. > > Best Regards, > Steve Zhang > > > >
