Doris-Breakwater commented on issue #70588:
URL: https://github.com/apache/doris/issues/70588#issuecomment-6892847414

   Source-based triage: the reported inconsistency is supported by the code, 
and declining an incapable index is a reasonable fix direction for plain 
`MATCH_PHRASE`. There is one important correction to the reproduction, and the 
change needs regression coverage before treating all phrase variants as 
equivalent.
   
   **Verified in source**
   
   - In the local `4.1.4` tag, `FunctionMatchBase::evaluate_inverted_index` 
returns `INDEX_INVALID_PARAMETERS` for a fulltext reader without phrase 
support; a missing iterator instead returns OK without a result. The same 
rejection exists at the cited master revision, although master correctly checks 
the reader selected for the query/analyzer. Preserve that reader selection in 
any patch. See [4.1.4 
evaluation](https://github.com/apache/doris/blob/4.1.4/be/src/exprs/function/match.cpp#L49-L109)
 and [master reader 
selection](https://github.com/apache/doris/blob/42b148bac33/be/src/exprs/function/match.cpp#L319-L352).
   - The fallback contract supports the proposed approach: 
`VExpr::_evaluate_inverted_index` records an exact result and marks the column 
as index-evaluated only when a result object has been populated. With no 
result, `VMatchPredicate` can execute the row function. Here 
`InvertedIndexResultBitmap::is_empty()` means **both bitmap pointers are 
null**, not “zero matching rows.” Return with the output untouched; do not 
populate a zero-cardinality bitmap, which would mean an exact no-match result. 
See [result 
registration](https://github.com/apache/doris/blob/42b148bac33/be/src/exprs/vexpr.cpp#L1056-L1079)
 and [bitmap 
representation](https://github.com/apache/doris/blob/42b148bac33/be/src/storage/index/inverted/inverted_index_reader.h#L215-L220).
   - Row execution calls `FunctionMatchBase::check`, so 
`enable_match_without_inverted_index=false` still rejects the slow path when it 
is actually executed. No broader suppression of `INDEX_INVALID_PARAMETERS` is 
needed. The normal segment downgrade handler does not classify that error as a 
fallback condition.
   
   **Reproduction correction**
   
   For a newly created index with `"parser" = "unicode"`, omitting 
`support_phrase` does **not** reliably create the intended non-positional 
index: the `4.1.4` FE `Index` constructor inserts `"support_phrase" = "true"` 
when a parser is specified and the property is absent. See [index 
defaults](https://github.com/apache/doris/blob/4.1.4/fe/fe-core/src/main/java/org/apache/doris/catalog/Index.java#L70-L100).
 Please use this explicitly in the reproducer and regression test:
   
   ```sql
   INDEX idx_msg (msg) USING INVERTED
   PROPERTIES ("parser" = "unicode", "support_phrase" = "false")
   ```
   
   This does not invalidate the reported behavior for an index that actually 
lacks positions, but the supplied DDL alone does not establish that condition.
   
   **Suggested next steps for the PR**
   
   1. Limit the behavioral change to the known lack-of-phrase-capability 
branch, leaving the bitmap untouched and retaining existing validation for 
other errors. The shared guard also covers `MATCH_PHRASE_PREFIX` and 
`MATCH_PHRASE_EDGE`; either explicitly cover those functions or scope the 
change accordingly.
   2. Add result-based regression tests with explicit `support_phrase=false`: 
direct filtering versus row evaluation, AND/OR/NOT, a projection or join case, 
NULL and zero-match inputs, and mixed indexed/unindexed segments. Verify the 
two-row example returns only `id=1`, and verify the existing slow-path error 
with fallback disabled. Retain a positive test proving a compatible positional 
index is still used, including multiple indexes/analyzers.
   3. Define the phrase-options boundary. The fulltext row path does not 
interpret slop suffixes as the index phrase path does; at the cited master 
revision, slop stripping in row execution is restricted to the keyword case. 
Thus the proposed fallback does not establish general semantic parity for 
queries such as `'a b ~2'`. Cover/document the supported behavior rather than 
claiming equivalence for all phrase syntax.
   4. Keep the `MATCH_ALL` prefilter as a separate optimization. For a plain 
exact phrase, matching all analyzed terms can provide a candidate superset, but 
the phrase must remain for row verification. Do not register that candidate 
bitmap as an exact phrase result or generalize the rewrite to 
prefix/edge/options without proving it cannot discard valid rows.
   
   **Remaining evidence / limits**
   
   I inspected source, including `42b148bac33`, but did not execute the SQL on 
a running cluster. Please attach the corrected reproducer's actual outputs, 
`SHOW CREATE TABLE`, exact FE/BE build revisions, both `EXPLAIN VERBOSE` plans, 
the relevant session settings, and the failing query ID with its BE error 
excerpt. These will establish which path the production queries actually took 
and whether their segments lack an index or contain an index without positions. 
An unrelated OR is not a reliable workaround: compound-expression error 
handling also differs between `4.1.4` and the cited master revision, and the 
optimizer may still attempt index evaluation. The source evidence supports 
proceeding with a focused PR; it does not yet prove every reported production 
plan or segment configuration.
   
   Breakwater-GitHub-Analysis-Slot: slot_d7943bc41e46
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to