discord9 opened a new pull request, #25067: URL: https://github.com/apache/datafusion/pull/25067
## Which issue does this PR close? No linked issue. This is an independently reproduced physical-optimizer correctness fix. ## Rationale for this change A new parent predicate must not be moved below an already-enforced file-scan fetch. For an ordered Parquet scan containing `[0, 1]`, `Filter(a = 1) → Scan(fetch=1)` returns no rows. Filter pushdown currently installs the predicate inside the capped scan, changing the result to `[1]`. This was reproduced through the physical-plan API, isolated FilterPushdown, and one/two complete default physical-optimizer passes. An ordinary SQL-planning reproduction is not established here. ## What changes are included in this PR? Reject new parent predicates at `FileScanConfig::try_pushdown_filters` when the scan already has a limit, using the existing unsupported propagation result without updating the source. This preserves existing source predicates and prevents admission through the pruning-only path as well as exact row filtering. Uncapped scans retain their existing filter-pushdown behavior. The production change is one source-local guard; no new configuration or optimizer machinery. This is independent of #23800 and #25065. ## What is the testing strategy for this PR? Two focused Parquet tests cover: - an ordered single-row-group scan with row filtering enabled and statistics pruning disabled, including fetch 1 and 0; - original/optimized results and one/two full optimizer passes; - uncapped exact filter pushdown; - pruning-only predicate admission and preservation of an existing internal scan predicate. Removing only the guard makes the row-filtering regression return `[1]` instead of `[]`. The pruning-only ablation fails the no-new-predicate plan assertion; it is not claimed as a separate wrong-row reproduction. The production guard was restored byte-exactly. Verified locally: - `cargo check -p datafusion-datasource` - `cargo test -p datafusion --test parquet_integration parquet::filter_pushdown::` — 10 passed - `cargo fmt --all --check` - `cargo clippy --workspace --all-targets --all-features -- -D warnings` The full workspace runtime test suite was not run. ## Are there any user-facing changes? Preserves results when optimizing an already-capped file scan beneath a parent filter. No public API or configuration changes. This conservatively declines new predicate pushdown at an existing scan limit rather than changing the scan's current predicate semantics. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
