[
https://issues.apache.org/jira/browse/IMPALA-14416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019337#comment-18019337
]
ASF subversion and git services commented on IMPALA-14416:
----------------------------------------------------------
Commit a41c5cbfddcf0bd84df37f4f3b9940b609abbf29 in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=a41c5cbfd ]
IMPALA-14416: JniFrontend.getDbs() should handle
InconsistentMetadataFetchException
JniFrontend.getDbs() returns the thrift representation of all the dbs.
This might trigger multiple getPartialCatalogObject requests to catalogd
and could fail in InconsistentMetadataFetchException, e.g. if a db is
removed after coordinator fetching the db name list and before
coordinator fetching the msDb of that db.
This patch fixes the issue by retrying the above steps when hitting
InconsistentMetadataFetchException, similar to what other methods in
Frontend do. Adds getThriftDbs() in Frontend to directly return the
thrift db list so JniFrontend can use it directly and the retry can be
added inside Frontend.java.
TestAuthorization.test_local_catalog_show_dbs_with_transient_db is an
existing test to verify a similar problem. Running this test with
authorization disabled can reproduce the current bug. So this patch
extracts the test code into
TestLocalCatalogRetries._run_show_dbs_with_transient_db() and share it
in both authz enabled and disabled tests.
Tests
- Ran TestLocalCatalogRetries.test_show_dbs_retry 60 times. Without the
fix, it fails in about a dozen times.
Change-Id: Ib337f88a2ac0f35142417f6cee51d30497f12845
Reviewed-on: http://gerrit.cloudera.org:8080/23402
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> JniFrontend.getDbs() could fail in InconsistentMetadataFetchException
> ---------------------------------------------------------------------
>
> Key: IMPALA-14416
> URL: https://issues.apache.org/jira/browse/IMPALA-14416
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Attachments: catalogd-logs.txt, query_log.txt
>
>
> Saw a SHOW DATABASES command failed by InconsistentMetadataFetchException in
> local catalog mode:
> https://jenkins.impala.io/job/ubuntu-20.04-dockerised-tests/3986
> {noformat}
> I20250908 20:51:17.859867 60748 Frontend.java:2405]
> f74a2de0c6f403f2:b983af6600000000] Analyzing query: show databases db: default
> ...
> I20250908 20:51:17.876580 60748 CatalogdMetaProvider.java:1752]
> f74a2de0c6f403f2:b983af6600000000] Invalidated objects in cache: [list of
> database names, HMS_METADATA for DB test_drop_db_6d7dba7f_db]
> I20250908 20:51:17.876839 60748 jni-util.cc:321]
> f74a2de0c6f403f2:b983af6600000000]
> org.apache.impala.catalog.local.InconsistentMetadataFetchException: Fetching
> DATABASE failed: DB_NOT_FOUND for TCatalogObject(type:DATABASE,
> catalog_version:0, db:TDatabase(db_name:test_drop_db_6d7dba7f_db))
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider.sendRequest(CatalogdMetaProvider.java:515)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider.access$100(CatalogdMetaProvider.java:209)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider$2.call(CatalogdMetaProvider.java:742)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider$2.call(CatalogdMetaProvider.java:737)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider.loadWithCaching(CatalogdMetaProvider.java:612)
> at
> org.apache.impala.catalog.local.CatalogdMetaProvider.loadDb(CatalogdMetaProvider.java:734)
> at
> org.apache.impala.catalog.local.LocalDb.getMetaStoreDb(LocalDb.java:101)
> at org.apache.impala.catalog.local.LocalDb.toThrift(LocalDb.java:318)
> at
> org.apache.impala.service.JniFrontend.getDbs(JniFrontend.java:395){noformat}
> After getting the list of FeDb in JniFrontend.getDbs(), getting thrift
> objects for all of them triggers new GetPartialCatalogObject requests to
> catalogd which could fail if the db is removed concurrently in catalogd.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
