xinyiZzz commented on code in PR #39908:
URL: https://github.com/apache/doris/pull/39908#discussion_r1731247878
##########
be/src/vec/common/allocator.cpp:
##########
@@ -211,14 +211,44 @@ void Allocator<clear_memory_, mmap_populate, use_mmap,
MemoryAllocator>::memory_
template <bool clear_memory_, bool mmap_populate, bool use_mmap, typename
MemoryAllocator>
void Allocator<clear_memory_, mmap_populate, use_mmap,
MemoryAllocator>::consume_memory(
- size_t size) const {
+ size_t size) {
+ // save tracker of the first alloc, all subsequent allocs must have the
same tracker in thread context.
+ // note that the tracker in thread context when Allocator is constructed
may be different from the first alloc.
+ if (mem_tracker == nullptr) {
+ mem_tracker =
doris::thread_context()->thread_mem_tracker_mgr->limiter_mem_tracker();
+ } else {
+ DCHECK(doris::thread_context()->thread_mem_tracker()->label() ==
mem_tracker->label())
+ << ", thread mem tracker label: "
+ << doris::thread_context()->thread_mem_tracker()->label()
+ << ", allocator tracker label: " << mem_tracker->label();
+ }
CONSUME_THREAD_MEM_TRACKER(size);
}
template <bool clear_memory_, bool mmap_populate, bool use_mmap, typename
MemoryAllocator>
void Allocator<clear_memory_, mmap_populate, use_mmap,
MemoryAllocator>::release_memory(
size_t size) const {
- RELEASE_THREAD_MEM_TRACKER(size);
+ doris::ThreadContext* thread_context = doris::thread_context(true);
+ if (thread_context && mem_tracker &&
+ thread_context->thread_mem_tracker()->label() == mem_tracker->label())
{
+ // If thread_context not exist and the label of thread_mem_tracker is
equal to
+ // Allocator mem_tracker label, this means that in the scope of
SCOPED_ATTACH_TASK,
+ // so thread_mem_tracker should be used to release memory.
+ RELEASE_THREAD_MEM_TRACKER(size);
+ } else if (mem_tracker) {
+ // if thread_context does not exist or the label of thread_mem_tracker
is not equal to
+ // Allocator tracker label, it usually happens during object
destruction. This means that
+ // the scope of SCOPED_ATTACH_TASK has been left, so release memory
using Allocator tracker.
+ SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER(mem_tracker);
+ RELEASE_THREAD_MEM_TRACKER(size);
+ } else {
+ // there is a scenario where an object that inherits Allocator has
never called alloc, but frees memory.
+ // in phmap, the memory alloced by an object may be transferred to
another object and then free.
+ // in this case, thread context must attach a memory tracker other
than Orphan,
+ // otherwise memory tracking will be wrong.
+ DCHECK(thread_context && thread_context->thread_mem_tracker()->label()
!= "Orphan");
Review Comment:
有可能，所以会走到第二条路径，使用 allocator 里保存的 tracker
第三条路径的意思是，如果 allocator tracker == nullptr，那 DCHECK thread context 一定不等于
nullptr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
