airborne12 commented on code in PR #69790:
URL: https://github.com/apache/doris/pull/69790#discussion_r4421379602
##########
be/src/storage/index/inverted/reader/collection_statistics.cpp:
##########
@@ -73,6 +73,15 @@ Result<std::shared_ptr<lucene::search::IndexSearcher>>
open_optional_clucene_sta
}
return ResultError(status);
}
+ // A segment can index a field with BKD although the tablet schema calls
it text: a VARIANT
+ // path is typed per segment, so CAST(v['n'] AS STRING) MATCH '1' collects
statistics for v.n,
+ // which a segment holding numbers indexed as BKD. That index has no terms
and no segments file
+ // to open, so it adds no statistics, as a segment without the path does;
its rows match
+ // without the index and score nothing.
+ if (compound.value()->fileExists(
+
InvertedIndexDescriptor::get_temporary_bkd_index_data_file_name())) {
+ return std::shared_ptr<lucene::search::IndexSearcher> {};
+ }
Review Comment:
Confirmed and fixed in df74dd4ca4. Reproduced with a path that only one of
two segments holds: V3 returns the score, SNII fails with "SNII scoring
requires every logical index". It also failed with an analyzed VARIANT index,
so it predates this PR, which made the plain VARIANT index reach it.
`snii_field_has_no_terms()` now treats a missing entry with a non-empty
suffix as an absent VARIANT path: the segment adds no statistics, as on V2/V3.
A missing index without a suffix still means it was not built and keeps
failing, which `SniiScoringRejectsMissingLogicalIndexForWholeCollection` pins.
New UT `SniiScoringSkipsVariantPathMissingFromSegment`, and a sparse-path case
in `test_match_keyword_score` on V2/V3/SNII.
The same commit fixes a crash in my previous commit that Cloud P0 found
twice: a numeric predicate leaves the BKD searcher in the searcher cache, and
on a cache hit `std::get<FulltextIndexSearcherPtr>` threw `bad_variant_access`
and aborted the BE, because only the cache miss checked for BKD. Reproduced
locally, fixed with `get_if`, and the regression now runs the score query after
a numeric predicate; with the old code the suite fails with the same abort.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
