Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22743 )
Change subject: IMPALA-13936: REFRESH should wait for ALTER ownership events ...................................................................... IMPALA-13936: REFRESH should wait for ALTER ownership events Coordinator uses collectTableRefs() to collect table names used by a statement. For ResetMetadataStmt used by REFRESH and INVALIDATE METADATA commands, it's intended to not return the table name in collectTableRefs() to avoid triggering unneccessary table metadata loading. However, when this method is used for the HMS event sync feature, we do want to know what the table is. Thus, catalogd can return the latest metadata of it after waiting for HMS events are synced. This bug leads to REFRESH/INVALIDATE not waiting for HMS ALTER ownership events to be synced. REFRESH/INVALIDATE statements might unexpectedly fail or succeed due to stale ownership info in coordinators. To avoid changing the existing logic of collectTableRefs(), this patch uses getTableName() directly for REFRESH statements since we know it's a single-table statement. There are other kinds of such single-table statements like DROP TABLE. To be generic, introduces a new interface, SingleTableStmt, for all such statements that have a single table name. If a statement is a SingleTableStmt, we use getTableName() directly instead of collectTableRefs() in collectRequiredObjects(). This improves coordinator in collecting table names for single-table statements. E.g. "DROP TABLE mydb.foo" previously has two candidate table names - "mydb.foo" and "default.mydb" (assuming the session db is "default"). Now it just collects "mydb.foo". Catalogd can return less metadata in the response. Tests: - Added FE tests for collectRequiredObjects() where coordinators collect db/table names. - Added authorization tests on altering the ownership in Hive and running queries in Impala. Change-Id: I813007e9ec42392d0f6d3996331987c138cc4fb8 Reviewed-on: http://gerrit.cloudera.org:8080/22743 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M fe/src/main/java/org/apache/impala/analysis/AlterTableStmt.java M fe/src/main/java/org/apache/impala/analysis/CommentOnDbStmt.java M fe/src/main/java/org/apache/impala/analysis/CommentOnTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/ConvertTableToIcebergStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/DescribeHistoryStmt.java M fe/src/main/java/org/apache/impala/analysis/DropStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java M fe/src/main/java/org/apache/impala/analysis/ResetMetadataStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowCreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowFilesStmt.java M fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java A fe/src/main/java/org/apache/impala/analysis/SingleTableStmt.java M fe/src/main/java/org/apache/impala/analysis/TruncateStmt.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/test/java/org/apache/impala/service/FrontendTest.java M fe/src/test/java/org/apache/impala/service/JdbcTestBase.java M tests/authorization/test_ranger.py M tests/common/impala_test_suite.py 22 files changed, 395 insertions(+), 39 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/22743 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I813007e9ec42392d0f6d3996331987c138cc4fb8 Gerrit-Change-Number: 22743 Gerrit-PatchSet: 8 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Anonymous Coward <k.venureddy2...@gmail.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Sai Hemanth Gantasala <saihema...@cloudera.com>
