airborne12 opened a new pull request, #69053:
URL: https://github.com/apache/doris/pull/69053
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
With score() in a Boolean search query, inverted-index leaves published BM25
scores before their parent AND, OR, or NOT predicate resolved its bitmap. A row
returned through one OR branch could receive a score from another branch whose
AND condition failed. Negated terms could also contribute scores. Equivalent
SEARCH and MATCH predicates could therefore return different scores.
Root cause: All compound branches wrote to the same CollectionSimilarity
before the compound predicate knew which rows matched.
Changes:
- Collect scores separately for each compound predicate and merge only
scores for rows in its final positive bitmap.
- Discard scores from NOT predicates and return an explicit inverted-index
error when a scored compound requires incomplete row fallback.
- Add BE unit tests and an ordered regression suite for matching, failed
AND, reversed AND, NOT, cached scoring, and the explicit error.
### Release note
score() now excludes BM25 contributions from failed Boolean branches and
negated terms. Scored compound predicates that cannot be fully evaluated by the
inverted index return an explicit error instead of a partial score.
### Check List (For Author)
- Test
- [x] Regression test: test_search_score_boolean_branches and
test_search_score_cache (1 suite each, 0 failures).
- [x] Unit Test: VCompoundPredScoreTest and CollectionSimilarityTest (30
tests passed).
- [x] Manual test: the ten-row SEARCH and MATCH reproduction returns the
same scores after rounding to four decimals.
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
Static analysis: clang-tidy found no warnings in the CollectionSimilarity
files. Analysis of the compound predicate header and new unit test was blocked
by parser errors in unchanged OpenMP and JNI code. BE build, header hygiene,
clang-format, and check-format passed.
- Behavior changed:
- [ ] No.
- [x] Yes. Scores from failed or negated branches are excluded, and
unsupported scored row fallback returns an error.
- Does this need documentation?
- [x] No. No syntax or configuration changed.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
