Rachelint opened a new pull request, #25852: URL: https://github.com/apache/datafusion/pull/25852
## Which issue does this PR close? Related to #25724. This draft is an experiment based on its sixth commit (`f9df89e`) and does not close an issue. ## Rationale for this change Evaluate whether a half-full hashbrown table or a half-full linear-probing index improves finished aggregation buckets. Local synthetic results do not show a general benefit; a ClickBench run is needed before deciding on an implementation. ## What changes are included in this PR? - Add an environment-selected bucket-only index variant (`DATAFUSION_BUCKET_INDEX=hashbrown-half` or `linear`); unset uses the existing hashbrown policy. - Add reproducible Criterion and physical final-aggregate benchmarks, raw CSV results, and [experiment report](datafusion/physical-plan/benches/bucket_group_index_experiment.md). - Add correctness coverage for collisions, group keys, split recursion, and spill. This PR includes #25724 commits because it targets `main` to allow DataFusion benchmark automation. The final experiment commit is `ba374e2d1`. ## What is the testing strategy for this PR? - `cargo fmt --all --check` - `cargo clippy --all-targets --all-features -- -D warnings` - `./ci/scripts/doc_prettier_check.sh --write --allow-dirty` - `cargo test -p datafusion-physical-plan --lib`: 2,351 passed, 0 failed, 1 ignored. - Release Criterion benchmark and five process-level rounds of the physical aggregate benchmark; numbers and limits are in the report. Real ClickBench data was unavailable locally; I will request `clickbench_partitioned` from the benchmark bot with threshold `262144`. ## Are there any user-facing changes? Only the opt-in experimental environment selector. No API change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
