[
https://issues.apache.org/jira/browse/IMPALA-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-14064:
------------------------------------
Priority: Critical (was: Major)
> Support MemBool backed by BufferPool in parquet scanner
> -------------------------------------------------------
>
> Key: IMPALA-14064
> URL: https://issues.apache.org/jira/browse/IMPALA-14064
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
>
> TCMalloc is not suitable for allocating/freeing large memory (>256KB)
> frequently in multiple threads. The allocation will go into CentralFreeList
> that is protected by a global lock. This serializes the threads in large
> allocation and free operations.
> MemPool is widely used in Backend and currently is backed by TCMalloc.
> Operators like scanner could hit this issue so query performance degrades in
> high concurrency.
> This aims to support MemPool backed by BufferPool which maintains a per-core
> arena so allocations will be recycled, alleviating the pressure on the
> TCMalloc thread cache. To limit the scope as a first step, only key MemPools
> used in parquet scanner will be changed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
