This is an automated email from the ASF dual-hosted git repository. michaelsmith pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/impala.git
commit bff814f0790eedc2ab8040c8ff2b9ad0218c9c69 Author: Sai Hemanth Gantasala <[email protected]> AuthorDate: Tue Nov 18 17:02:36 2025 -0800 IMPALA-14562: Enable Hierarchical event processing by default IMPALA-12709 Added support for hierarchical metastore event processing. This commit enables hierarchical event processing by default. hms_event_polling_interval_s can now be set to decimal value (eg: 0.5) to support millisecond precision interval. Along with that others configs can be fine tuned, such as: num_db_event_executors: To set the number of database level event executors. num_table_event_executors_per_db_event_executor: To set the number of table level event executors within a database event executor. min_event_processor_idle_ms: To set the minimum time to retain idle db processors and table processors. max_outstanding_events_on_executors: To set the limit of maximum outstanding events to process on event executors. Testing: - All the testing required to enable this flag is done in IMPALA-12709 and IMPALA-13801. Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b Reviewed-on: http://gerrit.cloudera.org:8080/23687 Reviewed-by: <[email protected]> Reviewed-by: Quanlong Huang <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- be/src/catalog/catalog-server.cc | 2 +- be/src/common/global-flags.cc | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/be/src/catalog/catalog-server.cc b/be/src/catalog/catalog-server.cc index 2208bca77..4f0c0e94a 100644 --- a/be/src/catalog/catalog-server.cc +++ b/be/src/catalog/catalog-server.cc @@ -258,7 +258,7 @@ DEFINE_string(common_hms_event_types, "ADD_PARTITION,ALTER_PARTITION,DROP_PARTIT "ALTER_SCHEMA_VERSION, DROP_SCHEMA_VERSION, CREATE_CATALOG, ALTER_CATALOG," "DROP_CATALOG, CREATE_DATACONNECTOR, ALTER_DATACONNECTOR, DROP_DATACONNECTOR."); -DEFINE_bool(enable_hierarchical_event_processing, false, +DEFINE_bool(enable_hierarchical_event_processing, true, "This configuration is used to enable hierarchical event processing. The default " "value is false. When enabled, events are fetched, dispatched and processed in " "different threads."); diff --git a/be/src/common/global-flags.cc b/be/src/common/global-flags.cc index f2272b24c..31c0599d8 100644 --- a/be/src/common/global-flags.cc +++ b/be/src/common/global-flags.cc @@ -288,7 +288,7 @@ DEFINE_bool(invalidate_tables_on_memory_pressure, false, "Configure catalogd to "invalidate_table_timeout_s. To enable this feature, a true flag must be applied to " "both catalogd and impalad."); -DEFINE_double(hms_event_polling_interval_s, 1, +DEFINE_double(hms_event_polling_interval_s, 0.5, "Configure catalogd to refresh cached table metadata based on metastore events. " "These metastore events could be generated by external systems like Apache Hive or " "a different Impala cluster using the same Hive metastore server as this one. "
