Riza Suminto has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23760 )
Change subject: IMPALA-14612: Add global metrics for admission state map size ...................................................................... IMPALA-14612: Add global metrics for admission state map size We need better observability for the admission state map to warn about potential memory leaks. The admission state map tracks queries currently being processed or queued. An entry is added when a query is submitted for admission. The entry is removed when the query finishes execution, is rejected by admission control, times out while queuing, or is cancelled. If the removal logic is missed due to bugs, the map size grows indefinitely, causing a memory leak. We have observed cases where admission state entries were not released, causing memory leaks in admissiond. Adds the metric admission-control-service.num-queries and its high water mark to track the number of active entries. This patch updates GenericShardedQueryMap to support an optional AtomicHighWaterMarkGauge. When set, the map automatically increments or decrements the gauge during Add and Delete operations. This ensures the metric accurately reflects the map size without requiring manual updates at every call site. Tests: Updated and passed test_admission_state_map_mem_leak to verify the metrics. Change-Id: Ie803aabf8d91b6381c5d0d7534cd9c9fc2166a73 Reviewed-on: http://gerrit.cloudera.org:8080/23760 Reviewed-by: Riza Suminto <[email protected]> Reviewed-by: Jason Fehr <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/scheduling/admission-control-service.cc M be/src/util/sharded-query-map-util.cc M be/src/util/sharded-query-map-util.h M common/thrift/metrics.json M tests/custom_cluster/test_admission_controller.py 5 files changed, 54 insertions(+), 8 deletions(-) Approvals: Riza Suminto: Looks good to me, but someone else must approve Jason Fehr: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/23760 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie803aabf8d91b6381c5d0d7534cd9c9fc2166a73 Gerrit-Change-Number: 23760 Gerrit-PatchSet: 8 Gerrit-Owner: Yida Wu <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]>
