[
https://issues.apache.org/jira/browse/IMPALA-14336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18015912#comment-18015912
]
ASF subversion and git services commented on IMPALA-14336:
----------------------------------------------------------
Commit 2fb56afb5e645a0c4e2b9bc3ab55e52cbee9fac0 in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=2fb56afb5 ]
IMPALA-14336: Avoid loading tables during table listing in the
IcebergMetaProvider
IcebergMetaProvider unnecessarily loads Iceberg tables in
loadTableList(). Table loading is a slow operation which can make
simple table listings painfully slow. This behavior is also contrast to
CatalogdMetaProvider which lists tables without loading them.
In our tests there were unloadable Iceberg tables which was never
intended, some test tables were just wrongly created under
iceberg_test/hadoop_catalog/, but they didn't use the HadoopCatalog.
Normally we can assume that the tables returned by an Iceberg REST
Catalog are loadable. Even if they are not, it shouldn't be too
problematic to get an exception a bit later. Also, the new behavior
is aligned with CatalogdMetaProvider, i.e. the tables are listed
without fully loading them, and we only get an error when we want
to use an unloadable table.
This patch moves the Iceberg tables out from
iceberg_test/hadoop_catalog/ that do not conform to HadoopCatalog.
Testing
* existing tests updated with the new paths
Change-Id: I9ff75a751be5ad4b5159a1294eaaa304049c454a
Reviewed-on: http://gerrit.cloudera.org:8080/23326
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Avoid loading tables during table listing in the IcebergMetaProvider
> --------------------------------------------------------------------
>
> Key: IMPALA-14336
> URL: https://issues.apache.org/jira/browse/IMPALA-14336
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
>
> Currently we redundantly load the tables when we just need a list of them. We
> should avoid the table loading because it makes tables listings very slow.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
