andygrove opened a new pull request, #2129: URL: https://github.com/apache/datafusion-ballista/pull/2129
# Which issue does this PR close? N/A — small benchmark tooling change, no issue filed. # Rationale for this change Sometimes one TPC-H query cannot complete on a given cluster and takes the rest of the suite down with it. At the moment the reference SF1000 cluster hits this on Q18, which exhausts the memory pool and OOM-kills the executors (#2025), wedging the queries that would have run after it. The workaround has been to drive the remaining queries as individual single-query jobs. That measures something different: each job starts against a cold page cache, which on that cluster cost even the join-free queries (Q1, Q6) roughly 20 s apiece compared to the same query inside a continuous suite run. Mixing the two methods in one results column silently misattributes that penalty to the engine. `--skip` lets the other 21 queries stay in a single run, so the column is internally consistent and the problem query can be investigated separately. # What changes are included in this PR? - Add `--skip <n>` to both the `ballista` and `datafusion` subcommands of the `tpch` benchmark binary. It is repeatable (`--skip 18 --skip 21`) and omits those queries from a whole-suite run. - An explicit `--query` still wins over `--skip`, so asking for a query directly always runs it rather than silently producing an empty run. - Skip values outside 1..=22, and a skip set that would exclude every query, are rejected with an error instead of being silently ignored — a typo like `--skip 42` would otherwise look like it took effect. - When queries are skipped, the run prints which ones, so it is visible in the benchmark output that the suite was not complete. - Both engine paths now share one `select_queries` helper instead of duplicating the "which queries do we run" logic. # Are there any user-facing changes? No library or API change. The `tpch` benchmark binary gains a `--skip` option; existing invocations behave exactly as before. Tested by seven unit tests over `select_queries` covering the default full suite, single and multiple skips, `--query` precedence, duplicate skips, out-of-range values, and skipping everything. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
