Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-04-05 Thread via GitHub
Shreyaskr1409 commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2769198630 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-04-03 Thread via GitHub
jayzhan211 commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2777328776 > You are also right here about the cost, but what if we can have 2 modes for benchmarks, one for the actual benchmarking purpose, and one with just to validate. If it is in

Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-04-03 Thread via GitHub
berkaysynnada commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2774860687 > Instead of running the benchmark, how about adding those benchmark query to tests, I don't think we need to actually "benchmark" the code for each merge. Keeping a

Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-04-02 Thread via GitHub
alamb commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2767683793 I see what you are saying now: https://github.com/apache/datafusion/pull/15500#issuecomment-2767216080 I now agree it would be good to test these somehow -- This

Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-04-01 Thread via GitHub
Omega359 commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2769848896 I disagree. If it's in the code base it should work or it should be marked as 'experimental' imho. No one wants to have to go through and fix a benchmark anytime they happen to

Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-04-01 Thread via GitHub
jayzhan211 commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2769754304 > > I don't think we need to actually "benchmark" the code for each merge. > > The issue [#5504](https://github.com/apache/datafusion/issues/5504) would require all ben

Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-04-01 Thread via GitHub
Shreyaskr1409 commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2769119055 > I don't think we need to actually "benchmark" the code for each merge How about we set a tag/label for performance related PRs and run benchmark tests for those sp

Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-04-01 Thread via GitHub
Omega359 commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2769284420 > > There has been a number of issues where benchmarks stopped working and no one noticed until someone happened to try and run them > > Instead of running the benchmark,

Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-04-01 Thread via GitHub
Shreyaskr1409 commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2769170980 > I don't think we need to actually "benchmark" the code for each merge. The issue #5504 would require all benchmarks to run after each merge. I think we could just

Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-03-31 Thread via GitHub
jayzhan211 commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2767724669 > There has been a number of issues where benchmarks stopped working and no one noticed until someone happened to try and run them Instead of running the benchmark, how

Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-03-31 Thread via GitHub
Omega359 commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2767313655 Sure. https://github.com/apache/datafusion/issues/5504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] Run all benchmarks on merge to main branch [datafusion]

2025-03-31 Thread via GitHub
alamb commented on issue #15511: URL: https://github.com/apache/datafusion/issues/15511#issuecomment-2767310181 I would persoanlly suggest if we are going to run the benchmarks we should also be actively tracking them / making sure they are measuring something useful. -- This is an auto