Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/23487 )
Change subject: IMPALA-14131: Add flag to configure the default value of 'impala.disableHmsSync' ...................................................................... Patch Set 5: (5 comments) http://gerrit.cloudera.org:8080/#/c/23487/5/be/src/catalog/catalog-server.cc File be/src/catalog/catalog-server.cc: http://gerrit.cloudera.org:8080/#/c/23487/5/be/src/catalog/catalog-server.cc@323 PS5, Line 323: all objects > CREATE/DROP/ALTER_DATABASE events are still processed, right? Otherwise how We always process database events. https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java#L1664 http://gerrit.cloudera.org:8080/#/c/23487/5/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/23487/5/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1263 PS5, Line 1263: disable hms sync > nit: to be consistent with other logs like L1254, let's use "flag {}" with Ack http://gerrit.cloudera.org:8080/#/c/23487/5/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1264 PS5, Line 1264: disable_hms_sync_by_default > nit: "global flag disable_hms_sync_by_default" Ack http://gerrit.cloudera.org:8080/#/c/23487/5/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/23487/5/tests/custom_cluster/test_events_custom_configs.py@1848 PS5, Line 1848: self.client.execute("invalidate metadata {}".format(tbl1)) > This is not a new behavior, but what happens with loaded tables in a databa I think you meant 'impala.disableHmsSync' is set to 'true' for the database, in that case loaded tables will still be in cache. But they would be stale as they might have missed metadata sync. So user would have invalidate them manually. http://gerrit.cloudera.org:8080/#/c/23487/5/tests/custom_cluster/test_events_custom_configs.py@1849 PS5, Line 1849: self.client.execute("invalidate metadata {}".format(tbl2)) : self.run_stmt_in_hive( : """insert into {tb1} partition(year=2025) values(1),(2); : insert into {tb2} values(1),(2),(3);""" : .format(tb1=tbl1, tb2=tbl2)) : EventProcessorUtils.wait_for_event_processing(self) : tb1_data = self.client.execute("select * from {}".format(tbl1)) : assert len(tb1_data.data) == 2 : # disable hms sync at database leve > This doesn't seem to validate that Impala actually detects the events, as t I had to invalidate the tables because they are not loaded in the cache. I have addressed your concern in the latest patchset where we load the table first then modify the flag and then verify that events are not skipped so that select statement fetches from cache instead of loading data directly. -- To view, visit http://gerrit.cloudera.org:8080/23487 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4ee617aed48575502d9cf5cf2cbea6ec897d6839 Gerrit-Change-Number: 23487 Gerrit-PatchSet: 5 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Thu, 20 Nov 2025 19:51:41 +0000 Gerrit-HasComments: Yes
