jayzhan211 opened a new issue, #26759: URL: https://github.com/apache/datafusion/issues/26759
### Is your feature request related to a problem or challenge? Two merge-queue jobs spend most of their test time waiting on one or two slow tests, while every other test finished long before: | Job | Runner | Test binary time | Other tests done by | Slow tests | |---|---|---|---|---| | `cargo test 'extended_tests'` | 32 vCPU | 229–238 s | 62–73 s | `leaf_pushdown_fuzz_extended` 229–238 s, `null_aware_join_matches_reference` 179–188 s | | `cargo test hash collisions` | 16 vCPU | 118–122 s (physical-plan unit tests) | 24–25 s | both cases of `clustered_keys_match_hash_grouping_across_batches_and_emits`, 117–123 s | These are from six merge-queue runs between Oct 24 and Oct 26. One Oct 24 run was faster overall (161 s and 132 s for the two extended tests) with the same shape. Why each test is slow: - `leaf_pushdown_fuzz_extended` runs 5,000 in-memory cases and then 1,000 Parquet cases, one after another (`datafusion/core/tests/fuzz_cases/leaf_pushdown_fuzz.rs`, the loop in `run` and the test at the bottom). - `null_aware_join_matches_reference` builds 120 sessions (20 seeds × 2 `target_partitions` × 3 batch sizes) and runs every query and its reference query in each one (`datafusion/core/tests/fuzz_cases/null_aware_join_fuzz.rs`). The reference result should be the same for every config, but it is recomputed for each. - `clustered_keys_match_hash_grouping_across_batches_and_emits` compares `GroupValuesClustered` with `GroupValuesColumn` over every key type, nullability, sort direction, 6 batch sizes and 4 emit limits. `GroupValuesClustered` does no hashing, so forcing hash collisions only slows down the `GroupValuesColumn` reference, and the hash-collision job gets no extra coverage of the code under test. ### Describe the solution you'd like - `leaf_pushdown_fuzz_extended`: run the seeds in a few concurrent chunks. The report already counts failures across all seeds and keeps the first one, so the chunks' results can be merged into the same report. - `null_aware_join_matches_reference`: compute each reference once per seed and compare every config against it, as #26644 did for the join fuzz test. - `clustered_keys_match_hash_grouping_across_batches_and_emits`: skip it under `force_hash_collisions`, as #26594 did for another test. Estimated effect: the extended test binary drops from about 235 s to about 70 s, and the physical-plan unit tests in the hash-collision job from about 120 s to about 25 s. The merge queue ran 198 times in the last week, so this frees about 165 s of a 32-vCPU runner and 95 s of a 16-vCPU runner per run, roughly 110 vCPU-minutes per run. The microbenchmark job (about 13 minutes) still decides how long a merge group takes, so this saves runner time, not queue time. ### Describe alternatives you've considered - Lower the case counts (fewer seeds, a smaller grid). Simpler, but it drops coverage; chunking and reusing the reference keep every case. - For the clustered test, a smaller grid would also help the regular `cargo test` job, where these two cases are the last physical-plan unit tests to finish (40 s). That changes what the test covers, so it is a call for the test's author. ### Additional context Part of #13813. ### Willingness to contribute I plan to contribute to this feature independently. I'll open a PR with one commit per test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
