songwdfu commented on PR #16308:
URL: https://github.com/apache/pinot/pull/16308#issuecomment-3129994079
Another benchmark run comparing single-threaded sequential merge and
multi-threaded pair-wise merge of sort aggregate
```
Benchmark
(_limit) (_numRecordsPerSegment) (_numSegments) (_numThreads)
(_query) Mode Cnt Score
Error Units
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 1000 20 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 357.333 ±
119.728 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 1000 20 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 359.308 ±
114.897 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 1000 20 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 361.735 ±
136.665 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 1000 50 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 862.478 ±
191.188 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 1000 50 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 900.720 ±
1307.384 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 1000 50 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 877.098 ±
178.293 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 10000 20 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 623.214 ±
593.546 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 10000 20 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 606.617 ±
68.112 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 10000 20 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 605.825 ±
174.194 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 10000 50 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1145.891 ±
244.456 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 10000 50 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1118.887 ±
233.065 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
500 10000 50 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1153.058 ±
183.399 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 1000 20 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 734.181 ±
103.436 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 1000 20 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 735.315 ±
88.299 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 1000 20 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 727.283 ±
85.610 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 1000 50 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1781.519 ±
259.990 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 1000 50 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1801.488 ±
359.455 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 1000 50 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1797.671 ±
54.163 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 10000 20 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1222.646 ±
854.341 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 10000 20 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1243.499 ±
589.980 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 10000 20 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1235.796 ±
827.973 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 10000 50 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 2325.667 ±
1487.379 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 10000 50 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 2257.402 ±
657.506 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sequentialCombineGroupBy
1000 10000 50 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 2250.112 ±
1189.091 us/op
```
new approach
```
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 1000 20 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 355.668 ±
206.736 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 1000 20 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 319.416 ±
138.880 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 1000 20 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 325.901 ±
89.033 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 1000 50 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 732.589 ±
99.759 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 1000 50 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 622.708 ±
83.707 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 1000 50 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 682.032 ±
901.032 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 10000 20 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 551.203 ±
1053.912 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 10000 20 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 527.575 ±
1155.315 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 10000 20 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 519.549 ±
534.012 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 10000 50 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 965.699 ±
2549.696 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 10000 50 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 897.387 ±
2678.923 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
500 10000 50 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 851.057 ±
3228.230 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 1000 20 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 553.756 ±
108.116 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 1000 20 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 546.014 ±
691.173 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 1000 20 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 553.075 ±
374.007 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 1000 50 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1079.726 ±
963.387 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 1000 50 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1060.148 ±
199.760 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 1000 50 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1057.522 ±
253.411 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 10000 20 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 851.051 ±
783.987 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 10000 20 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 835.396 ±
835.311 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 10000 20 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 815.366 ±
126.425 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 10000 50 4 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1471.049 ±
2465.292 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 10000 50 8 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1422.474 ±
3653.252 us/op
BenchmarkPairwiseCombineOrderByGroupBy.sortedPairwiseCombineGroupBy
1000 10000 50 16 SELECT sum(m1),
max(m2) FROM testTable GROUP BY d1, d2 ORDER BY d1, d2 avgt 3 1321.530 ±
3312.729 us/op
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
