xudong963 opened a new pull request, #25821: URL: https://github.com/apache/datafusion/pull/25821
## Which issue does this PR close? - Fixes #24807 in the `branch-55` release line. - Part of the 55.2.0 release tracking issue #25758. - Backports the merged fix in #24809. ## Rationale for this change In DataFusion 55.1.0, running `EnsureRequirements` again on a plan with a pushed-down `fetch` can remove the global limit and return too many rows. This is a correctness fix for a bug present in the 55 release line, so it belongs in the 55.2.0 patch release. The incorrect result and expected behavior are described in #24807. ## What changes are included in this PR? - Cherry-picks the merged commit from #24809 onto `branch-55`. - Keeps fetched distribution operators as row-selection boundaries during reoptimization. - Preserves a fetched ordered merge's TopK selection when replacing order-preserving variants. - Carries over the original regression tests and adjusts the backport for the 55 branch. ## What is the testing strategy for this PR? - `cargo fmt --all` - `cargo test -p datafusion --test core_integration physical_optimizer::enforce_distribution` (87 passed) - `cargo clippy --all-targets --all-features -- -D warnings` - `./dev/rust_lint.sh` (passed with the CI-pinned `hawkeye 6.2.0`) - Initialized the repository's test-data submodules, then ran the documented extended workspace test command. One unrelated macOS RSS-threshold test failed: `smj_with_mem_limit_1_runner` measured 154.2 MB against a 152.6 MB allowance; it failed again in isolation at 158.6 MB. This test exercises sort-merge join memory accounting, which this backport does not change. - Ran the extended workspace suite with only that RSS test skipped: all `datafusion` library, integration, and fuzz tests passed. The custom SQLLogic runner does not accept libtest's `--skip` flag, so I ran all other workspace packages separately without a skip; those passed, including all 503 SQLLogic files. - No local benchmark directly exercises reoptimization of a fetched physical plan. ## Are there any user-facing changes? Queries keep their requested global limit when distribution requirements are optimized more than once. This corrects query results without changing public APIs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
