Gabriel39 opened a new pull request, #68615: URL: https://github.com/apache/doris/pull/68615
Applies the dependency integration in #68613 to `branch-4.1`. An unfiltered vector search can still build an allowlist for every row in each index segment. Doris already assigns each scanner the selected segment's complete visible fragment coverage, so this work is redundant for that plan. Pin lance-c to an immutable upstream commit adopting the Lance development-branch fix in lance-format/lance#9599. Remove the superseded local lance-c patch chain; retain only the Foyer cache integration until its interface is accepted upstream. Rebase that cache integration onto the new object-store API, including bounded metadata reuse for cached immutable data reads. The fast path requires known, complete visible coverage of every selected index segment and no actual predicate. Partial coverage and predicates retain the row-ID prefilter, and deletion masks and unindexed fallback remain effective. No FE/BE planner or scanner API change is needed. Dependencies: lance-format/lance-c#89 and lance-format/lance#9599 are still under upstream review. The dependency pins include both changes and their regression tests. Validation: - Lance: 130 prefilter tests and three vector-segment contract tests passed; Rust formatting and targeted Clippy passed. - lance-c: a new C API regression covers 16 combinations of complete/partial segment coverage, physical/stable row IDs, predicates, and unindexed fallback, with deleted rows. It reproduces the old redundant loader and checks result IDs plus zero prefilter input rows, row IDs, build time, and load time on eligible scans. - lance-c: 442 tests passed, including 360 C API tests. With the retained Foyer integration: 455 Rust tests passed (including 364 C API tests), plus all three C/C++ compile-and-execute and static OSS transport tests. Rust formatting and Clippy passed. - The third-party integration regression exercises the actual downloader: fresh extraction, repeated invocation, re-extraction, an existing generic marker, and patch rejection without marking success. - Shell syntax, downloader fallback regression, archive checksum, and full diff checks passed. Validation uses synthetic regression data; no production-table latency claim is made. Full Doris builds are delegated to PR CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
