airborne12 opened a new pull request, #70592:
URL: https://github.com/apache/doris/pull/70592

   ### Background
   
   `MATCH_REGEXP` on a text index expands to the dictionary terms the 
expression matches. An expression without a fixed prefix walks the whole 
dictionary and runs the regular expression on every term.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: #70373 (the same prefilter for leading-wildcard globs)
   PR Type: Performance
   
   Problem Summary:
   
   Unanchored `MATCH_REGEXP` ran the regular expression on every dictionary 
term, although only terms holding its literal can match. This PR derives the 
literal every match must contain and skips the other terms before matching, 
cutting the expansion's CPU by 30-38% on V3 and 10-20% on SNII.
   
   ### Details
   
   - **Why it was slow:** every term went through the UTF conversion (V3) and a 
Hyperscan scan.
   - **Optimization:** the longest literal run of the expression's top-level 
concatenation becomes the pattern's required text, only for an expression 
without an enumeration prefix (CLucene tests the text before the prefix, so 
`^cach.` would otherwise walk past its range).
   - **Results:** 100M rows, caches off: V3 5.6-5.9 to 3.5-4.1 CPU-seconds, 
SNII 3.7-4.0 to 3.0-3.5; anchored and literal-free expressions unchanged; 
identical results.
   - **Applicability and cost:** unanchored regular expressions with a literal; 
one parse per pattern.
   
   ### Testing
   
   BE UT `TermPatternRegexpLiteral*`; regression 
`test_storage_format_snii_regexp_literal` (19 expressions, SNII against V3, 
expected output generated without this change); clang-tidy and the hygiene gate.
   
   ### Release note
   
   Unanchored MATCH_REGEXP queries skip the dictionary terms that cannot match 
before running the expression.
   
   ### Check List (For Author)
   
   - Behavior changed: No
   - Does this need documentation: No
   - Pre-submission status: Ready
   
   ### AI Disclosure
   
   - Tool: Claude Code (Claude Opus 5.5)
   - Generated: code, tests and this description
   - Human review: design and benchmark results reviewed by the author; code 
review pending
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to