airborne12 opened a new pull request, #67024:
URL: https://github.com/apache/doris/pull/67024
### What problem does this PR solve?
Issue Number: None
Related PR: #61160
Problem Summary: An IVF-on-disk search with no `AnnIndexIVFListCache`
installed never returns — it recurses until the stack is exhausted.
**Reproduction**: build an `IVF_ON_DISK` index, save it, load it, and search
it without calling `AnnIndexIVFListCache::create_global_cache()` first. Under
ASAN the process dies with `AddressSanitizer: stack-overflow`; on a large stack
it simply spins.
**Root cause**: `CachedRandomAccessReader::borrow()` falls back to
`RandomAccessReader::borrow()` when `AnnIndexIVFListCache::instance()` is null.
faiss documents that base implementation as *"allocates a buffer and calls
read_at()"*, and `CachedRandomAccessReader::read_at()` is overridden to call
`borrow()` — so the two call each other without bound:
```
CachedRandomAccessReader::borrow (no cache)
-> faiss::RandomAccessReader::borrow (base: allocate + read_at)
-> CachedRandomAccessReader::read_at (override: calls borrow)
-> CachedRandomAccessReader::borrow
-> ...
```
The branch reads like a fallback but cannot work at all.
**Reachability, stated plainly**: `exec_env_init` installs the cache
unconditionally at BE startup, so a running BE does not hit this today. What it
does hit is unit tests. Every existing IVF-on-disk case in
`faiss_vector_index_test.cpp` opens with
`AnnIndexIVFListCache::create_global_cache(...)`, and a new test that does not
know to do so gets a stack overflow rather than a diagnosable failure — which
is how this was found, while writing an unrelated ANN container test. It also
becomes live the moment the cache is made optional or its lifetime is shortened.
**Fix**: the no-cache path reads the region itself under `_io_mutex` and
returns an owning `ReadRef`. No cache means no cache accounting, so it uses a
plain buffer rather than a `DataPage`; the buffer lives exactly as long as the
ref the caller holds.
### Release note
None
### Check List (For Author)
- Test
- [x] Unit Test
`VectorSearchTest.IVFOnDiskSearchWithoutTheListCacheStillReads` searches
an IVF-on-disk index with no cache installed. On the parent commit it aborts
the binary with `AddressSanitizer: stack-overflow`; with the fix it passes in
milliseconds. The search runs on a worker with a 60s deadline so a regression
fails this one case instead of hanging the whole suite. Ran
`VectorSearchTest.*` + `*Ann*` + `*ANN*` + `AnnIndexIVFListCacheTest.*` — 152
passed.
- Behavior changed:
- [x] No. The path being fixed could not complete before.
- Does this need documentation?
- [x] No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
