Venugopal Reddy K created IMPALA-13801:
------------------------------------------
Summary: Enhance Lag mechanism in hierarchical metastore event
processing
Key: IMPALA-13801
URL: https://issues.apache.org/jira/browse/IMPALA-13801
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Venugopal Reddy K
1. Need to redefine the lag in the hierarchical processing mode. Hierarchical
mode currently have a mechanism to check the total number of outstanding events
at the moment so that memory usage pressure on catalogd can be controlled when
event processing becomes slow i.e., iff outstanding event count exceeds
max_outstanding_events_on_executors configured value.
2. Currently, with enable_hierarchical_event_processing as true,
lastSyncedEventId_ and lastSyncedEventTimeSecs_ are updated upon event dispatch
to EventExecutorService for processing on respective DbEventExecutor and/or
TableEventExecutor. So lastSyncedEventId_ and lastSyncedEventTimeSecs_ doesn't
actually mean events are processed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
