alamb opened a new pull request, #25832: URL: https://github.com/apache/datafusion/pull/25832
## Which issue does this PR close? - Related to https://github.com/apache/arrow-rs/pull/10859 - Related to https://github.com/apache/arrow-rs/issues/10926 **DO NOT MERGE**: this PR exists only to run the DataFusion tests and benchmarks against a pre-release arrow-rs branch. ## Rationale for this change https://github.com/apache/arrow-rs/pull/10859 changes the parquet push decoder to fuse consecutive `RowFilter` predicates that use the same projection, so a column is decoded once and later predicates see only the surviving rows. I want to (1) confirm the existing DataFusion parquet filter pushdown tests pass against it and (2) measure its effect on ClickBench with `pushdown_filters` enabled, using `datafusion-cli` and the benchmark runner. ## What changes are included in this PR? Based on `main` (arrow/parquet 60.0.0), plus a `[patch.crates-io]` section pinning all arrow/parquet crates to https://github.com/alamb/arrow-rs/tree/pr-10859-fused-predicates, which is the head of apache/arrow-rs#10859 (commit `257753e2d32f6f398dc6db14f8195b3447dd4abe`) pushed to my fork so the pin stays stable if the PR branch is force pushed. ## What is the testing strategy for this PR? Tests and benchmarks only: - The existing parquet filter pushdown tests in `datafusion/core/tests/parquet/filter_pushdown.rs` - `clickbench_pushdown` via the benchmark bot, and local `datafusion-cli` runs with `datafusion.execution.parquet.pushdown_filters = true` ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
