[
https://issues.apache.org/jira/browse/IMPALA-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17950628#comment-17950628
]
Quanlong Huang commented on IMPALA-14064:
-----------------------------------------
I have a WIP patch for this: https://gerrit.cloudera.org/c/22123/
> Support MemBool backed by BufferPool in parquet scanner
> -------------------------------------------------------
>
> Key: IMPALA-14064
> URL: https://issues.apache.org/jira/browse/IMPALA-14064
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
>
> TCMalloc is not suitable for allocating/freeing large memory (>256KB)
> frequently in multiple threads. The allocation will go into CentralFreeList
> that is protected by a global lock. This serializes the threads in large
> allocation and free operations.
> MemPool is widely used in Backend and currently is backed by TCMalloc.
> Operators like scanner could hit this issue so query performance degrades in
> high concurrency.
> This aims to support MemPool backed by BufferPool which maintains a per-core
> arena so allocations will be recycled, alleviating the pressure on the
> TCMalloc thread cache. To limit the scope as a first step, only key MemPools
> used in parquet scanner will be changed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
