Hello Ha, This might have been an unintended oversight that we can improve and fix. Could you please open an issue with any further details that you have?
Eduard On Wed, Nov 22, 2023 at 7:37 PM Ha Cao <ha....@twosigma.com> wrote: > Hello, > > > > I am using the CatalogUtil::dropTableData > <https://sourcegraph.com/github.com/apache/iceberg@e8cf33db7d3fc637504a51a801c055dce54474b7/-/blob/core/src/main/java/org/apache/iceberg/CatalogUtil.java?L86> > to delete data and metadata files for my tables after I drop them from the > catalog. I notice that if the program terminates unexpectedly (e.g. VM > shutdown, connection exception, etc.) midway through dropTableData, if it > successfully deletes the manifest files (.avro files), but fails somewhere > after that (it hasn’t got to delete the metadata.json file), on a retry, > CatalogUtil::dropTableData will fail at line 94 – > Iterables.addAll(manifestsToDelete, snapshot.allManifests(io)) > <https://sourcegraph.com/github.com/apache/iceberg@e8cf33db7d3fc637504a51a801c055dce54474b7/-/blob/core/src/main/java/org/apache/iceberg/CatalogUtil.java?L94> > (specifically because of snapshot.allManifests) with awssdk > NoSuchKeyException because the manifest file (.avro file) is no longer > there. I wonder why in the current implementation, we prefer to let it > throw an exception at this line, instead of catching it, adding a WARN log > or something like that, and letting it continue to delete other files > afterwards. > > > > Thank you very much! > > Happy Thanksgiving, > > Ha >
