sam-1112 opened a new issue, #5246:
URL: https://github.com/apache/datafusion-comet/issues/5246
### What is the problem the feature request solves?
Native RLike documents that `is_match` hand-rolls a loop over the input
string array instead of calling Arrow's `regexp_is_match(_scalar)`, which
recompiles the pattern on every batch (see arrow-string `regexp.rs`). #5102
(and follow-up PRs) rewrote that loop (e.g. `iter`/`map`/`collect`). The
rationale lives in docs, but nothing in `native/spark-expr/benches/` measures
it, so future changes to this hot path can evaporate without a baseline.
### Describe the potential solution
Add `native/spark-expr/benches/rlike.rs` alongside the existing
`regexp_extract.rs` Criterion bench, and register it in
`native/spark-expr/Cargo.toml` (`[[bench]]`, `harness = false`).
Suggested coverage (mirroring review discussion):
- Scalar pattern, Utf8 subject array (common path through `is_match`)
- Mix of matches / non-matches / nulls
- Optionally Dictionary(Int32, Utf8), since that is the reachable
dictionary shape in Comet today
This does not need to land in the string-layout / dictionary fix PR;
it is follow-up so the rewritten loop has a measurable baseline.
### Additional context
Requested in review on the RLike string-layout PR: the null-check removal is
likely lost in noise next to `Regex::is_match`, so this is not a merge blocker,
but worth tracking rather than dropping.
Reference: `native/spark-expr/benches/regexp_extract.rs`
Related PR:
https://github.com/apache/datafusion-comet/pull/5215#pullrequestreview-4849576694
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
