songwdfu opened a new pull request, #16152:
URL: https://github.com/apache/pinot/pull/16152
Previously we copy the left and right input of join before evaluating the
non-equi join condition that touches both sides.
This PR introduces a List that serves as a "view" over the left and right
input, allowing join condition evaluation before arraycopy. Then array copy
only happens if join condition evaluates to true.
Testing with a simple query on TPC-H
```
SET maxRowsInJoin=2147483647;
SET useMultistageEngine=true;
SET timeoutMs=100000;
SELECT c_custkey, o_orderkey, c_acctbal, o_totalprice
FROM customer c
JOIN orders o
ON o_custkey = c_custkey AND o_totalprice <= c_acctbal * {};
-- param = 0.0, 0.2, 0.4, 0.6, 0.8, 1.0
```
execution time result (ms, 10-time avg):
| query | before | after |
| test1 | 601.2ms | 559.6ms |
| test2 | 655.4ms | 547.4ms |
| test3 | 651.8ms | 534.6ms |
| test4 | 644.1ms | 548.3ms |
| test5 | 619.5ms | 528.9ms |
| test6 | 636.7ms | 538.8ms |
Alloc profiling for test1 (with high-selectivity join condition), join alloc
memory percentage drops from 41% to 25%, that is 39% reduction.:
Before:
After:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
