github-actions[bot] commented on code in PR #70592:
URL: https://github.com/apache/doris/pull/70592#discussion_r4624168142
##########
be/src/storage/index/inverted/query/exec/term_pattern.cpp:
##########
@@ -113,6 +174,12 @@ Status TermPattern::create(TermPatternKind kind,
std::string_view pattern, TermP
}
out->_scratch.reset(scratch);
out->_enumeration_prefix = regexp_enumeration_prefix(pattern);
+ // As for a glob: an expression with a prefix already stops where the
prefix ends, and CLucene
+ // checks the prefix only after this text, so '^cach.' would walk on past
"cach" through every
+ // later term that lacks "cach". Only an expression that enumerates the
whole dictionary keeps it.
+ if (out->_enumeration_prefix.empty()) {
+ out->_text = regexp_required_literal(pattern);
Review Comment:
[P1] Skip the literal prefilter for patterns containing NUL
A MATCH_REGEXP pattern containing `cache`, then a raw NUL byte, then
`missing` still compiles as `cache`, because `hs_compile` above consumes a C
string. SQL string literals can contain this byte through the single-backslash
zero escape. The new `parse_regex` call instead reads the entire `string_view`
and derives all 13 bytes as `required_text`. Both SNII's byte substring check
and CLucene's wide substring check then discard the valid term `cache` before
Hyperscan sees it, silently losing matching rows that were returned before this
change. Derive the literal from the same prefix that Hyperscan compiles, or
return empty required text for raw NULs, as `RegexGramCompiler::compile_regexp`
already does, and add coverage using a pattern with an explicit string length.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]