Rich-T-kid opened a new pull request, #26946: URL: https://github.com/apache/datafusion/pull/26946
## Which issue does this PR close? - Closes #26945. ## Rationale for this change For dictionary string columns, rewriting a short `IN` list into equality chains repeats dictionary evaluation and key expansion for each value. Keeping the membership expression evaluates the dictionary values once and expands one result through the keys. TPC-H Q12 has two of these expressions when Parquet RLE-to-dictionary conversion is enabled. ## What changes are included in this PR? The short `IN` simplifier now retains literal `IN` and `NOT IN` expressions for dictionaries with `Utf8` or `Utf8View` values. Other string columns keep the existing rewrite behavior. The existing type-based simplifier test now covers a dictionary string column. ## What is the testing strategy for this PR? - `cargo test -p datafusion-optimizer simplify_short_inlist_types` - `cargo clippy --all-targets --all-features -- -D warnings` - Local release A/B on TPC-H SF1 with `DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY=true`, 12 partitions, 8192-row batches, and a 32G memory limit. Against main `926901f9001977e85df617b96328801b42f6af4a`, focused Q12 median improved from 12.42 ms to 12.03 ms (3.1%). A separate full-suite pass improved Q12 from 11.57 ms to 11.16 ms (3.5%); unchanged-plan Q13 and Q22 were flat. Upstream TPC-H, TPC-DS, and partitioned ClickBench runs will follow with dictionary conversion enabled on both sides. ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
