Peter Rozsa has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/22878 )
Change subject: IMPALA-14016: Add multi-catalog support for local catalog mode ...................................................................... IMPALA-14016: Add multi-catalog support for local catalog mode This patch adds a new MetaProvider called MultiMetaProvider, which is capable of handling multiple MetaProviders at once, prioritizing one primary provider over multiple secondary providers. The primary provider handles some methods exclusively for deterministic behavior. In database listings, if one database name occurs multiple times, the first occurrence shadows the further occurrences. This change also modifies the local catalog implementation's initialization. If catalogd is deployed, then it instantiates the CatalogdMetaProvider and checks if the catalog configuration directory is set as a backend flag. If it's set, then it tries to load every configuration from the folder, and tries to instantiate the IcebergMetaProvider from those configs. If the instantiation fails, an error is reported to the logs, but the startup is not interrupted. Tests: - E2E tests for multi-catalog behavior - Unit test for ConfigLoader Change-Id: Ifbdd0f7085345e7954d9f6f264202699182dd1e1 --- M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/NoopAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java A fe/src/main/java/org/apache/impala/catalog/local/MultiMetaProvider.java D fe/src/main/java/org/apache/impala/service/FeCatalogManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java A fe/src/main/java/org/apache/impala/service/catalogmanager/CatalogdImpl.java A fe/src/main/java/org/apache/impala/service/catalogmanager/ConfigLoader.java A fe/src/main/java/org/apache/impala/service/catalogmanager/FeCatalogManager.java A fe/src/main/java/org/apache/impala/service/catalogmanager/LocalImpl.java A fe/src/main/java/org/apache/impala/service/catalogmanager/TestImpl.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java A fe/src/test/java/org/apache/impala/service/catalogmanager/ConfigLoaderTest.java A testdata/workloads/functional-query/queries/QueryTest/iceberg-rest-catalog-multicatalog.test M tests/custom_cluster/test_iceberg_rest_catalog.py 16 files changed, 929 insertions(+), 283 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/22878/2 -- To view, visit http://gerrit.cloudera.org:8080/22878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifbdd0f7085345e7954d9f6f264202699182dd1e1 Gerrit-Change-Number: 22878 Gerrit-PatchSet: 2 Gerrit-Owner: Peter Rozsa <pro...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
