airborne12 opened a new pull request, #69730:
URL: https://github.com/apache/doris/pull/69730
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: A VARIANT column can declare predefined fields whose
patterns overlap, for example the glob 'a.*' and the name 'a.b'. The BE writer
(variant_util.cpp) routes a subcolumn to the first predefined field whose
pattern matches its path and builds the index of that field.
`OlapTable.getInvertedIndex` looped over all the fields without stopping and
kept the last match, so a query on the same path chose the index of a different
field than the one the data was written with, and fell back to another index or
to a scan.
The loop now stops at the first match, the same order the BE uses, so a
write and a query on one path land on the same index.
### Release note
Fix the index chosen for a Variant subpath when the column declares
overlapping field patterns.
### Compatibility
- FE only, and nothing is persisted: `getInvertedIndex` computes its answer
from the table schema on every call.
- A table whose predefined field patterns do not overlap sees no change,
since a path with a single match has the same first and last match.
- A table with overlapping patterns now resolves a path to the first
matching field. The BE already built the index of the first matching field
(`variant_util.cpp`), so the index that holds the data is the one a query now
looks for, and no re-index is needed. Queries on such paths that missed the
index and fell back to another index or a scan can use it.
### Performance
The loop over the predefined fields now stops at the first match, so it
never does more work than before.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- Added
InvertedIndexPropertiesTest.testOverlappingFieldPatternsPickTheFirstPredefinedFieldLikeTheBe;
InvertedIndexPropertiesTest passes. With the early exit removed, the new test
fails.
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [ ] No.
- [x] Yes. With overlapping field patterns, the first declared match
decides the index instead of the last.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
