HappenLee opened a new pull request, #68583:
URL: https://github.com/apache/doris/pull/68583
### What problem does this PR solve?
Issue Number: N/A
Related PR: #68446
Primary-key lookup prunes segments by key bounds, but then opens every
segment in each selected rowset and eagerly loads their PK indexes and Bloom
filters. Row-store reads and point-query column fallback also load the entire
rowset to fetch one located row. This adds unnecessary cold IO and shared
segment-cache accesses for point lookups, including the storage path used by
batch point queries.
Load candidates only when they are probed, and retain them in a
request-local cache across keys. Index cache slots by rowset metadata position
while preserving physical segment IDs for loading and row locations. Read only
the located segment in row-store/column-store paths; column fallback uses the
rowset already pinned by the key lookup. Full scans retain the existing dense
segment handle. Shared MoW callers are adapted to the sparse cache without
changing sequence, delete bitmap, or search-order semantics.
The new storage tests demonstrate a cold seven-segment rowset
loading/indexing one candidate instead of all seven, and successful reads with
an unrelated segment file absent. This is a reduction in storage work, not an
end-to-end latency benchmark. No customer latency or throughput improvement is
claimed.
### Release note
Reduce unnecessary segment and primary-key index loading for point lookups
in multi-segment rowsets.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test: 73 ASAN BE tests passed, including 10 new cases
covering legacy and non-contiguous IDs, pruning, cache reuse, open failures and
retry, row-store/column-store reads, overlapping segments and delete bitmap
behavior. Existing key-probe, row-cache, historical-row, fixed/flexible
partial-update tests passed.
- [ ] Manual test
- [ ] No need to test
- Behavior changed:
- [ ] No.
- [x] Yes. Load only segments actually probed/read; SQL result semantics
and persisted formats are unchanged.
- Does this need documentation?
- [x] No. Internal storage optimization with no new setting or protocol
field.
- [ ] Yes.
Validation command:
```bash
./run-be-ut.sh -j 64 --run
--filter='*PointQuerySegmentTest*:*KeyProbeTest*:*RowCacheProbeTest*:*HistoricalRowFetcherTest*:*HistoricalRowRetrieverTest*:*FixedPartialUpdateTest*:*FlexiblePartialUpdateTest*'
```
clang-format 16, build-header hygiene and `git diff --check` passed.
clang-tidy analyzed all 14 changed files with no diagnostics on modified lines.
The installed tool required an explicit compiler resource directory; analysis
also used a local VFS overlay removing only an existing unmatched `NOLINTEND`
comment in `be/src/core/types.h` (no C++ tokens changed). Existing diagnostics
on unchanged lines remain; this is not a claim that the whole files are
warning-free.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
