alamb commented on PR #16398: URL: https://github.com/apache/datafusion/pull/16398#issuecomment-2980748403
> I've been trying to measure the impact of these changes, but for the life of me I can't get useful results out of the benchmark comparison script. The sample below is actually comparing the exact same commit on a mac mini M4 and yet I still see large fluctuations. Not sure what else I can do 🤷♂️. I agree the existing benchmarks are not perfect -- figuring out how to make them better would be good, though as you probably know it is a challenge of benchmarking in general. I'll kick off some runs against this PR with my scripts to see what it shows. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
