Noemi Pap-Takacs has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23512 )
Change subject: IMPALA-13725: Add Iceberg table repair functionalities ...................................................................... IMPALA-13725: Add Iceberg table repair functionalities In some cases users delete files directly from storage without going through the Iceberg API, e.g. they remove old partitions. This corrupts the table, and makes queries that try to read the missing files fail. This change introduces a repair statement that deletes the dangling references of missing files from the metadata. Note that the table cannot be repaired if there are missing delete files because Iceberg's DeleteFiles API which is used to execute the operation allows removing only data files. Testing: - E2E - HDFS - S3, Ozone - analysis Change-Id: I514403acaa3b8c0a7b2581d676b82474d846d38e Reviewed-on: http://gerrit.cloudera.org:8080/23512 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M common/thrift/JniCatalog.thrift A fe/src/main/java/org/apache/impala/analysis/AlterTableExecuteRepairMetadataStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableExecuteStmt.java A fe/src/main/java/org/apache/impala/catalog/iceberg/ImpalaRepairIcebergTable.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M tests/query_test/test_iceberg.py 8 files changed, 283 insertions(+), 10 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/23512 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I514403acaa3b8c0a7b2581d676b82474d846d38e Gerrit-Change-Number: 23512 Gerrit-PatchSet: 8 Gerrit-Owner: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Daniel Vanko <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
