[
https://issues.apache.org/jira/browse/IMPALA-13654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932057#comment-17932057
]
ASF subversion and git services commented on IMPALA-13654:
----------------------------------------------------------
Commit d928815d1a3bbab25eea7bfc40baa8913a9465f2 in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d928815d1 ]
IMPALA-13654: Tolerate missing data files of Iceberg tables
Before this patch we got a TableLoadingException for missing data files.
This means the IcebergTable will be in an incomplete state in Impala's
memory, therefore we won't be able to do any operation on it.
We should continue table loading in such cases, and only throw exception
for queries that are about to read the missing data files.
This way ROLLBACK / DROP PARTITION, and some SELECT statements should
still work.
If Impala is running in strict mode via CatalogD flag
--iceberg_allow_datafiles_in_table_location_only, and an Iceberg table
has data files outside of table location, we still raise an exception
and leave the table in an unloaded state. To retain this behavior, the
IOException we threw is substituted to TableLoadingException which fits
better to logic errors anyway.
Testing
* added e2e tests
Change-Id: If753619d8ee1b30f018e90157ff7bdbe5d7f1525
Reviewed-on: http://gerrit.cloudera.org:8080/22367
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Tolerate missing data files of Iceberg tables
> ---------------------------------------------
>
> Key: IMPALA-13654
> URL: https://issues.apache.org/jira/browse/IMPALA-13654
> Project: IMPALA
> Issue Type: Bug
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
>
> Currently we get a TableLoadingException for missing data files. This means
> the IcebergTable will be in an incomplete state in Impala's memory, therefore
> we won't be able to do any operation on it.
> We should continue table loading in such cases, and only throw exception for
> queries that are about to read the missing data files.
> This way ROLLBACK / DROP PARTITION, and some SELECT statements should still
> work.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
