alamb commented on PR #19364:
URL: https://github.com/apache/datafusion/pull/19364#issuecomment-3670114568
I benchmarked with/without the no buffers optimization and it seems to be
worth around 1% (10ms out of 990ms)
```shell
hyperfine --warmup 3 " ./datafusion-cli-starting -c \"select
l_returnflag,l_linestatus, count(*) as count_order from 'lineitem.parquet'
group by l_returnflag, l_linestatus;\" "
```
Starting (before
https://github.com/apache/datafusion/pull/19364/commits/da34c4cd52033ad04bb2da93133636f019f62eeb)
```
andrewlamb@Andrews-MacBook-Pro-3:~/Downloads$ hyperfine --warmup 3 "
./datafusion-cli-starting -c \"select l_returnflag,l_linestatus, count(*) as
count_order from 'lineitem.parquet' group by l_returnflag, l_linestatus;\" "
Benchmark 1: ./datafusion-cli-starting -c "select
l_returnflag,l_linestatus, count(*) as count_order from 'lineitem.parquet'
group by l_returnflag, l_linestatus;"
Time (mean ± σ): 989.4 ms ± 9.2 ms [User: 11962.4 ms, System:
620.9 ms]
Range (min … max): 980.0 ms … 1007.0 ms 10 runs
```
After
https://github.com/apache/datafusion/pull/19364/commits/da34c4cd52033ad04bb2da93133636f019f62eeb
```
Benchmark 1: ./datafusion-cli-no-buffers-opt -c "select
l_returnflag,l_linestatus, count(*) as count_order from 'lineitem.parquet'
group by l_returnflag, l_linestatus;"
Time (mean ± σ): 982.1 ms ± 10.0 ms [User: 11962.7 ms, System:
607.3 ms]
Range (min … max): 968.4 ms … 996.9 ms 10 runs
```
It seems reproducable:
```
andrewlamb@Andrews-MacBook-Pro-3:~/Downloads$ hyperfine --warmup 3 "
./datafusion-cli-starting -c \"select l_returnflag,l_linestatus, count(*) as
count_order from 'lineitem.parquet' group by l_returnflag, l_linestatus;\" "
Benchmark 1: ./datafusion-cli-starting -c "select
l_returnflag,l_linestatus, count(*) as count_order from 'lineitem.parquet'
group by l_returnflag, l_linestatus;"
Time (mean ± σ): 985.9 ms ± 11.2 ms [User: 11979.8 ms, System:
621.6 ms]
Range (min … max): 970.4 ms … 1002.0 ms 10 runs
andrewlamb@Andrews-MacBook-Pro-3:~/Downloads$ hyperfine --warmup 3 "
./datafusion-cli-no-buffers-opt -c \"select l_returnflag,l_linestatus,
count(*) as count_order from 'lineitem.parquet' group by l_returnflag,
l_linestatus;\" "
Benchmark 1: ./datafusion-cli-no-buffers-opt -c "select
l_returnflag,l_linestatus, count(*) as count_order from 'lineitem.parquet'
group by l_returnflag, l_linestatus;"
Time (mean ± σ): 995.5 ms ± 17.8 ms [User: 12140.3 ms, System:
627.2 ms]
Range (min … max): 964.1 ms … 1023.3 ms 10 runs
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
