Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/22743
Change subject: IMPALA-13936: REFRESH should wait for ALTER ownership events ...................................................................... IMPALA-13936: REFRESH should wait for ALTER ownership events Coordinator uses collectTableRefs() to collect table names used by a statement. For ResetMetadataStmt used by REFRESH and INVALIDATE METADATA commands, it's intended to not return the table name in collectTableRefs() to avoid triggering unneccessary table metadata loading. However, when this method is used for the HMS event sync feature, we do want to know what the table it is. So catalogd can return the latest metadata of it after waiting for HMS events are synced. This bug leads to REFRESH/INVALIDATE not waiting for HMS ALTER ownership events to be synced. REFRESH/INVALIDATE statements might unexpectedly fail or succeed due to stale ownership info in coordinators. To avoid changing the existing logic of collectTableRefs(), this patch uses getTableName() directly for REFRESH statements since we know it's a single-table statement. There are other kinds of such single-table statements like DROP TABLE. To be generic, introduces a new interface, SingleTableStmt, for all such statements that have a single table name. If a statement is a SingleTableStmt, we use getTableName() directly instead of collectTableRefs() in collectRequiredObjects(). This improves coordinator in collecting table names for single-table statements. E.g. "DROP TABLE mydb.foo" previously has two candidate table names - "mydb.foo" and "default.mydb" (assuming the session db is "default"). Now it just collects "mydb.foo". Catalogd can return less metadata in the response. Tests: - Added FE tests for collectRequiredObjects() where coordinators collect db/table names. - Added authorization tests on altering the ownership in Hive and running queries in Impala. Change-Id: I813007e9ec42392d0f6d3996331987c138cc4fb8 --- M fe/src/main/java/org/apache/impala/analysis/AlterTableStmt.java M fe/src/main/java/org/apache/impala/analysis/CommentOnDbStmt.java M fe/src/main/java/org/apache/impala/analysis/CommentOnTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/ConvertTableToIcebergStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/DescribeHistoryStmt.java M fe/src/main/java/org/apache/impala/analysis/DropStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java M fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowCreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowFilesStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java A fe/src/main/java/org/apache/impala/analysis/SingleTableStmt.java M fe/src/main/java/org/apache/impala/analysis/TruncateStmt.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/service/FrontendTest.java M fe/src/test/java/org/apache/impala/service/JdbcTestBase.java M tests/authorization/test_ranger.py M tests/common/impala_test_suite.py 22 files changed, 387 insertions(+), 35 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/22743/1 -- To view, visit http://gerrit.cloudera.org:8080/22743 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I813007e9ec42392d0f6d3996331987c138cc4fb8 Gerrit-Change-Number: 22743 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
