Csaba Ringhofer created IMPALA-13610:
----------------------------------------
Summary: Merging KrpcDataStreamRecvr mem estimate is inconsistent
with BE
Key: IMPALA-13610
URL: https://issues.apache.org/jira/browse/IMPALA-13610
Project: IMPALA
Issue Type: Bug
Components: Backend, Frontend
Reporter: Csaba Ringhofer
The biggest difference is that the planner assumes that in the merging case mem
usage scales with the number of senders, while BE uses a shared limit
(KrpcDataStreamRecvr::total_buffer_limit_) based on
exchg_node_buffer_size_bytes (10 MB by default).
https://github.com/apache/impala/blob/e6b8e88f658651097657e4ccbe3f2b75508b89dc/fe/src/main/java/org/apache/impala/planner/ExchangeNode.java#L343
example:
set mt_dop=8; select * from tpch_text_gzip.lineitem order by l_orderkey;
02:MERGING-EXCHANGE peak/estimated peak: 20.71 MB / 122.75 MB
Note that if there are a lot of senders the bottleneck is likely to be the
single receiver and queue mem limit is probably fully utilized.
Some other issues:
- the planner ignores the extra row batch per sender used by SortedRunMerger
- the BE uses AtomicInt32 num_buffered_bytes_ to store the actually used bytes,
which could overflow if the limit would be above 2GB
- total_buffer_limit_ is not enforced strictly (which seems to be more or less
by design) and can be exceed by heavily num_buffered_bytes_ in case a few
queues occupies memory up to the limit and new row batches arrive to the other
empty queues
I am not sure about the correct solution - in case values are distributed
randomly and the sender queues will be drained evenly, 10MB per sender seems
unnecessary if there are a lot of senders. On the other hand fixed 10MB seems
to little and is likely to be often exceeded due to non strict enforcement of
the limit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
