kazuyukitanimura commented on issue #670:
URL:
https://github.com/apache/datafusion-comet/issues/670#issuecomment-2259153166
After #741 there are still some issues with `if` and `case`
```
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
add_many_decimals 11255 11385
184 25.6 39.1 1.0X
add_many_decimals: Comet (Scan) 14372 14545
245 20.0 49.9 0.8X
add_many_decimals: Comet (Scan, Exec) 9846 9933
123 29.3 34.2 1.1X
Running benchmark: TPCDS Micro Benchmarks
Running case: add_many_integers
Stopped after 2 iterations, 7870 ms
Running case: add_many_integers: Comet (Scan)
Stopped after 2 iterations, 6910 ms
Running case: add_many_integers: Comet (Scan, Exec)
Stopped after 2 iterations, 7138 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
add_many_integers 3866 3935
98 74.5 13.4 1.0X
add_many_integers: Comet (Scan) 3450 3455
7 83.5 12.0 1.1X
add_many_integers: Comet (Scan, Exec) 3548 3569
30 81.2 12.3 1.1X
Running benchmark: TPCDS Micro Benchmarks
Running case: agg_high_cardinality
Stopped after 2 iterations, 3620 ms
Running case: agg_high_cardinality: Comet (Scan)
Stopped after 2 iterations, 5411 ms
Running case: agg_high_cardinality: Comet (Scan, Exec)
Stopped after 2 iterations, 2126 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
agg_high_cardinality 1769 1810
59 40.7 24.6 1.0X
agg_high_cardinality: Comet (Scan) 2642 2706
90 27.3 36.7 0.7X
agg_high_cardinality: Comet (Scan, Exec) 1060 1063
4 67.9 14.7 1.7X
Running benchmark: TPCDS Micro Benchmarks
Running case: agg_low_cardinality
Stopped after 5 iterations, 2032 ms
Running case: agg_low_cardinality: Comet (Scan)
Stopped after 3 iterations, 2179 ms
Running case: agg_low_cardinality: Comet (Scan, Exec)
Stopped after 7 iterations, 2089 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
agg_low_cardinality 371 406
34 194.2 5.1 1.0X
agg_low_cardinality: Comet (Scan) 717 727
13 100.4 10.0 0.5X
agg_low_cardinality: Comet (Scan, Exec) 278 298
19 258.7 3.9 1.3X
Running benchmark: TPCDS Micro Benchmarks
Running case: agg_sum_decimals_no_grouping
Stopped after 2 iterations, 14633 ms
Running case: agg_sum_decimals_no_grouping: Comet (Scan)
Stopped after 2 iterations, 78280 ms
Running case: agg_sum_decimals_no_grouping: Comet (Scan, Exec)
Stopped after 2 iterations, 16892 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------
agg_sum_decimals_no_grouping 7251
7317 93 39.7 25.2 1.0X
agg_sum_decimals_no_grouping: Comet (Scan) 38946
39140 274 7.4 135.2 0.2X
agg_sum_decimals_no_grouping: Comet (Scan, Exec) 8407
8446 55 34.3 29.2 0.9X
Running benchmark: TPCDS Micro Benchmarks
Running case: agg_sum_integers_no_grouping
Stopped after 2 iterations, 8131 ms
Running case: agg_sum_integers_no_grouping: Comet (Scan)
Stopped after 2 iterations, 8375 ms
Running case: agg_sum_integers_no_grouping: Comet (Scan, Exec)
Stopped after 2 iterations, 9179 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------
agg_sum_integers_no_grouping 3914
4066 215 73.6 13.6 1.0X
agg_sum_integers_no_grouping: Comet (Scan) 4079
4188 154 70.6 14.2 1.0X
agg_sum_integers_no_grouping: Comet (Scan, Exec) 4557
4590 47 63.2 15.8 0.9X
Running benchmark: TPCDS Micro Benchmarks
Running case: case_when_column_or_null
Stopped after 2 iterations, 3114 ms
Running case: case_when_column_or_null: Comet (Scan)
Stopped after 2 iterations, 5693 ms
Running case: case_when_column_or_null: Comet (Scan, Exec)
Stopped after 2 iterations, 3415 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
case_when_column_or_null 1440 1557
166 200.1 5.0 1.0X
case_when_column_or_null: Comet (Scan) 2832 2847
20 101.7 9.8 0.5X
case_when_column_or_null: Comet (Scan, Exec) 1691 1708
23 170.3 5.9 0.9X
Running benchmark: TPCDS Micro Benchmarks
Running case: case_when_scalar
Stopped after 9 iterations, 2090 ms
Running case: case_when_scalar: Comet (Scan)
Stopped after 2 iterations, 2312 ms
Running case: case_when_scalar: Comet (Scan, Exec)
Stopped after 5 iterations, 2048 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
case_when_scalar 196 232
51 367.0 2.7 1.0X
case_when_scalar: Comet (Scan) 1133 1156
33 63.5 15.7 0.2X
case_when_scalar: Comet (Scan, Exec) 376 410
38 191.5 5.2 0.5X
Running benchmark: TPCDS Micro Benchmarks
Running case: filter_highly_selective
Stopped after 11 iterations, 2132 ms
Running case: filter_highly_selective: Comet (Scan)
Stopped after 3 iterations, 2233 ms
Running case: filter_highly_selective: Comet (Scan, Exec)
Stopped after 10 iterations, 2054 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
filter_highly_selective 154 194
85 466.1 2.1 1.0X
filter_highly_selective: Comet (Scan) 734 745
9 98.0 10.2 0.2X
filter_highly_selective: Comet (Scan, Exec) 156 205
108 462.3 2.2 1.0X
Running benchmark: TPCDS Micro Benchmarks
Running case: filter_less_selective
Stopped after 7 iterations, 2091 ms
Running case: filter_less_selective: Comet (Scan)
Stopped after 3 iterations, 2093 ms
Running case: filter_less_selective: Comet (Scan, Exec)
Stopped after 10 iterations, 2272 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
filter_less_selective 165 299
165 435.4 2.3 1.0X
filter_less_selective: Comet (Scan) 691 698
11 104.3 9.6 0.2X
filter_less_selective: Comet (Scan, Exec) 193 227
39 373.7 2.7 0.9X
Running benchmark: TPCDS Micro Benchmarks
Running case: if_column_or_null
Stopped after 2 iterations, 2630 ms
Running case: if_column_or_null: Comet (Scan)
Stopped after 2 iterations, 3102 ms
Running case: if_column_or_null: Comet (Scan, Exec)
Stopped after 2 iterations, 5368 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
if_column_or_null 1315 1315
0 219.1 4.6 1.0X
if_column_or_null: Comet (Scan) 1518 1551
46 189.8 5.3 0.9X
if_column_or_null: Comet (Scan, Exec) 2606 2684
111 110.5 9.0 0.5X
Running benchmark: TPCDS Micro Benchmarks
Running case: join_anti
Stopped after 2 iterations, 12404 ms
Running case: join_anti: Comet (Scan)
Stopped after 2 iterations, 12023 ms
Running case: join_anti: Comet (Scan, Exec)
[528.105s][warning][gc,alloc] Executor task launch worker for task 2.0 in
stage 802.0 (TID 14773): Retried waiting for GCLocker too often allocating
134217730 words
Stopped after 2 iterations, 11999 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
join_anti 6128 6202
104 11.7 85.1 1.0X
join_anti: Comet (Scan) 5774 6012
335 12.5 80.2 1.1X
join_anti: Comet (Scan, Exec) 5971 6000
41 12.1 82.9 1.0X
Running benchmark: TPCDS Micro Benchmarks
Running case: join_condition
Stopped after 2 iterations, 4255 ms
Running case: join_condition: Comet (Scan)
Stopped after 2 iterations, 3245 ms
Running case: join_condition: Comet (Scan, Exec)
Stopped after 2 iterations, 3721 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
join_condition 2055 2128
103 264.4 3.8 1.0X
join_condition: Comet (Scan) 1534 1623
125 354.2 2.8 1.3X
join_condition: Comet (Scan, Exec) 1808 1861
75 300.6 3.3 1.1X
Running benchmark: TPCDS Micro Benchmarks
Running case: join_exploding_output
Stopped after 2 iterations, 3425 ms
Running case: join_exploding_output: Comet (Scan)
Stopped after 2 iterations, 3004 ms
Running case: join_exploding_output: Comet (Scan, Exec)
Stopped after 2 iterations, 3495 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
join_exploding_output 1603 1713
155 339.0 2.9 1.0X
join_exploding_output: Comet (Scan) 1413 1502
126 384.5 2.6 1.1X
join_exploding_output: Comet (Scan, Exec) 1722 1748
37 315.6 3.2 0.9X
Running benchmark: TPCDS Micro Benchmarks
Running case: join_inner
Stopped after 4 iterations, 2194 ms
Running case: join_inner: Comet (Scan)
Stopped after 5 iterations, 2324 ms
Running case: join_inner: Comet (Scan, Exec)
Stopped after 3 iterations, 2110 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
join_inner 512 549
36 562.2 1.8 1.0X
join_inner: Comet (Scan) 462 465
3 623.9 1.6 1.1X
join_inner: Comet (Scan, Exec) 694 703
10 415.1 2.4 0.7X
Running benchmark: TPCDS Micro Benchmarks
Running case: join_left_outer
Stopped after 2 iterations, 192095 ms
Running case: join_left_outer: Comet (Scan)
Stopped after 2 iterations, 192225 ms
Running case: join_left_outer: Comet (Scan, Exec)
Stopped after 2 iterations, 191166 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
join_left_outer 95996 96048
73 3.3 303.0 1.0X
join_left_outer: Comet (Scan) 95378 96113
1038 3.3 301.1 1.0X
join_left_outer: Comet (Scan, Exec) 95377 95583
292 3.3 301.1 1.0X
Running benchmark: TPCDS Micro Benchmarks
Running case: join_semi
[1448.983s][warning][gc,alloc] Executor task launch worker for task 0.0 in
stage 1153.0 (TID 20715): Retried waiting for GCLocker too often allocating
134217730 words
[1463.584s][warning][gc,alloc] Executor task launch worker for task 2.0 in
stage 1159.0 (TID 20772): Retried waiting for GCLocker too often allocating
134217730 words
[1476.740s][warning][gc,alloc] Executor task launch worker for task 0.0 in
stage 1165.0 (TID 20825): Retried waiting for GCLocker too often allocating
134217730 words
[1476.754s][warning][gc,alloc] Executor task launch worker for task 1.0 in
stage 1165.0 (TID 20826): Retried waiting for GCLocker too often allocating
134217730 words
Stopped after 2 iterations, 26371 ms
Running case: join_semi: Comet (Scan)
[1490.216s][warning][gc,alloc] Executor task launch worker for task 2.0 in
stage 1171.0 (TID 20882): Retried waiting for GCLocker too often allocating
134217730 words
[1515.531s][warning][gc,alloc] Executor task launch worker for task 1.0 in
stage 1183.0 (TID 20991): Retried waiting for GCLocker too often allocating
134217730 words
Stopped after 2 iterations, 25521 ms
Running case: join_semi: Comet (Scan, Exec)
[1541.053s][warning][gc,alloc] Executor task launch worker for task 2.0 in
stage 1195.0 (TID 21102): Retried waiting for GCLocker too often allocating
134217730 words
Stopped after 2 iterations, 25372 ms
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.5
Apple M1 Max
TPCDS Micro Benchmarks: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
join_semi 13164 13186
30 5.5 182.8 1.0X
join_semi: Comet (Scan) 12522 12761
338 5.8 173.9 1.1X
join_semi: Comet (Scan, Exec) 12276 12686
580 5.9 170.5 1.1X
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
