Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/23487 )
Change subject: IMPALA-14131: Add flag to configure the default value of 'impala.disableHmsSync' ...................................................................... Patch Set 4: (4 comments) http://gerrit.cloudera.org:8080/#/c/23487/3/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/23487/3/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@4832 PS3, Line 4832: To be clear, let's add the flag name. http://gerrit.cloudera.org:8080/#/c/23487/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/23487/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1265 PS3, Line 1265: "is not set. Global level flag is set to {} ", msTbl_.getTableName(), Let's mention the tabel/db level key name and the flag name here as well. http://gerrit.cloudera.org:8080/#/c/23487/4/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/23487/4/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1239 PS4, Line 1239: // check if global hmssync flag is disabled nit: this comment is unrelated to the following code. Pls move this to L1262 or remove it. http://gerrit.cloudera.org:8080/#/c/23487/4/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/23487/4/tests/custom_cluster/test_events_custom_configs.py@1861 PS4, Line 1861: self.run_stmt_in_hive( : """alter table {} SET TBLPROPERTIES ('impala.disableHmsSync'='false')""" : .format(tbl1)) I ran the test and found that processing ALTER_TABLE event from this invalidates the table. Then the following ADD_PARTITION event from the insert statement is skipped since the table is unloaded. I20251114 10:57:47.279266 615853 MetastoreEvents.java:936] EventId: 387761 EventType: ALTER_TABLE Target: test_disable_hms_sync_globally_fc9939ef.test_disable_hms_sync_1. Before flag value null after flag value false changed for table test_disable_hms_sync_globally_fc9939ef.test_disable_hms_sync_1 I20251114 10:57:47.279541 615853 CatalogServiceCatalog.java:3357] Invalidating table metadata: test_disable_hms_sync_globally_fc9939ef.test_disable_hms_sync_1 I20251114 10:57:56.283649 615853 MetastoreEvents.java:936] EventId: 387764 EventType: ADD_PARTITION Target: test_disable_hms_sync_globally_fc9939ef.test_disable_hms_sync_1. Found table level flag impala.disableHmsSync is set to false for table test_disable_hms_sync_globally_fc9939ef.test_disable_hms_sync_1 I20251114 10:57:56.283746 615853 CatalogOpExecutor.java:4986] EventId: 387764 Table test_disable_hms_sync_globally_fc9939ef.test_disable_hms_sync_1 is not loaded. Skipping add partitions I20251114 10:57:56.283828 615853 MetastoreEvents.java:947] EventId: 387764 EventType: ADD_PARTITION Target: test_disable_hms_sync_globally_fc9939ef.test_disable_hms_sync_1. Incremented skipped metric to 7 since no partitions were added. Let's run this alter_table statement in Impala so the table can still be in loaded state. -- To view, visit http://gerrit.cloudera.org:8080/23487 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ee617aed48575502d9cf5cf2cbea6ec897d6839 Gerrit-Change-Number: 23487 Gerrit-PatchSet: 4 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Fri, 14 Nov 2025 03:10:12 +0000 Gerrit-HasComments: Yes
