zhuqi-lucas commented on issue #15375: URL: https://github.com/apache/datafusion/issues/15375#issuecomment-2747493793
I did some POC testing, the performance improvement is very promising, even 30% improvement for some queries: ```rust ┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Query ┃ main ┃ concat_batches_for_sort ┃ Change ┃ ┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ Q1 │ 2241.04ms │ 1816.69ms │ +1.23x faster │ │ Q2 │ 1841.01ms │ 1496.73ms │ +1.23x faster │ │ Q3 │ 12755.85ms │ 12770.18ms │ no change │ │ Q4 │ 4433.49ms │ 3278.70ms │ +1.35x faster │ │ Q5 │ 4414.15ms │ 4409.04ms │ no change │ │ Q6 │ 4543.09ms │ 4597.32ms │ no change │ │ Q7 │ 8012.85ms │ 9026.30ms │ 1.13x slower │ │ Q8 │ 6572.37ms │ 6049.51ms │ +1.09x faster │ │ Q9 │ 6734.63ms │ 6345.69ms │ +1.06x faster │ │ Q10 │ 9896.16ms │ 9564.17ms │ no change │ └──────────────┴────────────┴─────────────────────────┴───────────────┘ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓ ┃ Benchmark Summary ┃ ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩ │ Total Time (main) │ 61444.64ms │ │ Total Time (concat_batches_for_sort) │ 59354.33ms │ │ Average Time (main) │ 6144.46ms │ │ Average Time (concat_batches_for_sort) │ 5935.43ms │ │ Queries Faster │ 5 │ │ Queries Slower │ 1 │ │ Queries with No Change │ 4 │ └────────────────────────────────────────┴────────────┘ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
