goutamadwant opened a new pull request, #24917: URL: https://github.com/apache/datafusion/pull/24917
## Which issue does this PR close? - Closes #22185. ## Rationale for this change Pathological constant regexes can spend the regex engine's full runtime compilation budget during constant folding, making logical planning take seconds. Disabling folding for every regex would also penalize ordinary patterns, so this change gives planning a smaller compilation budget and defers only expensive literals to execution. ## What changes are included in this PR? - Add a defaulted `ScalarUDFImpl::should_evaluate_const` hook so immutable UDFs can defer expensive literal evaluation in both logical and physical simplification. - Apply a 256 KiB planning-time regex size limit to `regexp_like` and regex operators while retaining the existing runtime limit. - Prevent deferred regex subtrees from being evaluated indirectly through parent expressions. - Keep nonconstant regex expressions off the compilation-preflight path and inspect literal arguments by reference without cloning them. - Preserve ordinary constant folding and existing flag, syntax-error, and NULL behavior. ## What is the testing strategy for this PR? - Added logical optimizer, physical simplifier, scalar-UDF, and SQL planner regressions for ordinary and pathological patterns, aliases, flags, NULL inputs, and nonconstant expressions. - `cargo test -p datafusion-expr -p datafusion-functions -p datafusion-optimizer -p datafusion-physical-expr` - `cargo test -p datafusion expensive_regexp_like_is_deferred_to_execution --test core_integration` - `cargo test --profile=ci --test sqllogictests -- regexp/regexp_like.slt` - Contributor-guide extended workspace test suite with `avro`, `json`, `backtrace`, `extended_tests`, `recursive_protection`, and `parquet_encryption` features. - Repository lint steps: rustfmt, workspace Clippy, Taplo, license headers, typo checks, Prettier, workflow policy checks, and rustdoc with warnings denied. - A local debug planning harness running the issue-shaped expression 20 times took 5.37 seconds with the 10 MiB runtime budget and 0.17 seconds with the 256 KiB planning budget, approximately 31.6 times faster. ## Are there any user-facing changes? Planning now defers unusually expensive constant regexes to execution instead of spending the runtime compilation budget while optimizing the query. Ordinary constant regexes continue to fold. This also adds a defaulted `ScalarUDFImpl::should_evaluate_const` extension point; existing UDF implementations keep their current behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
