[
https://issues.apache.org/jira/browse/IMPALA-13487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17943792#comment-17943792
]
Quanlong Huang commented on IMPALA-13487:
-----------------------------------------
Uploaded an off-cpu flamegraph captured for a tcmalloc contention scenario.
[^mt_4_con_4-off-cpu.svg]
The query is a simple scan with selective predicates but materializing large
arrays. Ran with MT_DOP=4 and 4 concurrent queries are running. On a machine
with a 16 cores CPU. All threads of impalad are tracked here. Looking into the
fragment instance threads (impala::QueryState::ExecFInstance), most of the
off-cpu time is spent in waiting for locks of TCMalloc:
!image-2025-04-12-18-57-46-354.png|width=871,height=461!
The above patch is able to reveal time spent in these (stuck in tcmalloc).
> Capture metrics for memory allocation in execution profile
> ----------------------------------------------------------
>
> Key: IMPALA-13487
> URL: https://issues.apache.org/jira/browse/IMPALA-13487
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Manish Maheshwari
> Assignee: Quanlong Huang
> Priority: Critical
> Attachments: image-2025-04-12-18-57-46-354.png, mt_4_con_4-off-cpu.svg
>
>
> Capture metrics for memory allocation in execution profile to identify
> queries that are bound on memory allocation and allocation wait times esp
> during running many concurrent queries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
