zhuqi-lucas commented on PR #15380: URL: https://github.com/apache/datafusion/pull/15380#issuecomment-2888423022
> > I am confused why my benchmark for local Mac no regression for sort-tpch Q3, but the generated benchmark for linux we can reproduce the regression. > > It may be that the tests are unstable or that the memory / code made for x86 was different than for aarch. > > I'll run the benchmarks again to see if they are reproducable Thank you @alamb , the result still regression for linux run. > > I am confused why my benchmark for local Mac no regression for sort-tpch Q3, but the generated benchmark for linux we can reproduce the regression. > > I was wondering, maybe we only add the "concat-arrays-only-instead-of-full-record-batch" optimization and leave the rest for what it is (for now)? So don't change the `if self.reservation.size() < self.sort_in_place_threshold_bytes` condition to use different heuristics and keep `sort_in_place_threshold_bytes` to `1024 * 1024`. > > It might be of smaler significance, but it is more likely to have no or smaller regressions (and we can follow-up with better heuristics that work accros different machines). Good point @Dandandan , let me do the smallest optimization first, i will address it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
