[
https://issues.apache.org/jira/browse/IMPALA-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith resolved IMPALA-13989.
------------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
> ALTER TABLE RENAME can fail with concurrent INVALIDATE METADATA
> ---------------------------------------------------------------
>
> Key: IMPALA-13989
> URL: https://issues.apache.org/jira/browse/IMPALA-13989
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 5.0.0
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Minor
> Fix For: Impala 5.0.0
>
>
> IMPALA-13631 removes holding the catalog's versionLock_ writeLock during the
> whole operation (including HMS RPC). That introduces a possible failure mode
> where {{ALTER TABLE RENAME}} fails with
> {quote}Table/view rename succeeded in the Hive Metastore, but failed in
> Impala's Catalog Server.{quote}
> when {{INVALIDATE METADATA}} is run concurrently. This shows up in the new
> statements added to test_concurrent_ddls.py.
> I can reproduce this error by adding a delay after HMS {{alter_table}} RPC
> completes (and before we {{getNextMetastoreEventsForTableIfEnabled}}) and
> running {{INVALIDATE METADATA}} from another session. I think that suggests
> the scenario as:
> # {{alter_table}} RPC completes
> # Impala {{invalidate metadata}} executes and processes {{alter_table}} event
> # {{alterTableOrViewRename}} runs {{catalog_.alterTable}}, but old table has
> already been removed from the catalog so it fails
> This should be pretty rare. Running a global {{invalidate metadata}} is a bad
> idea in a production environment as it's akin to restarting catalogd. However
> I think we can address this with better error handling in
> {{alterTableOrViewRename}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
