goutamadwant opened a new pull request, #25819: URL: https://github.com/apache/datafusion/pull/25819
## Which issue does this PR close? - Part of #25437. This first slice does not close the broader issue. ## Rationale for this change A `ListingTable` scan with declared Range partitioning currently retains every file group even when a range-key filter proves that some groups cannot contain matching rows. For example, `range_key < 10` still planned all four groups for split points `(10), (20), (30)`. ## What changes are included in this PR? - Prune file groups that are provably disjoint from simple literal comparisons on a single integer range key. - Leave pruned groups empty at their original partition indexes so range-partitioned joins retain alignment. - Recompute scan statistics after pruning. Unsupported predicates, types, and uncertain bounds retain their file groups. This does not yet compact partition counts or remove all downstream work for empty partitions. ## What is the testing strategy for this PR? - SQLLogic plan tests verify `range_key < 10` and equality at the `10` split point retain only the expected file group; a join test verifies the resulting rows. - Unit tests cover ascending and descending boundaries, reversed comparisons, compound predicates, NULL split points, and conservative fallbacks. - `cargo fmt --all --check`, `cargo clippy --all-targets --all-features -- -D warnings`, focused tests, and the extended workspace test suite passed locally. ## Are there any user-facing changes? Yes. Supported range-key filters can avoid reading file groups known not to match. There is no public API change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
