github-actions[bot] commented on code in PR #69346:
URL: https://github.com/apache/doris/pull/69346#discussion_r4344176493
##########
be/src/format_v2/table/lance_reader.h:
##########
@@ -82,6 +82,14 @@ class LanceTableReader final : public TableReader {
std::string _snapshot_description() const;
Status _prepare_fts_query_context();
Status _open_scanner(const TFileRangeDesc& range);
+ struct RowIdPrefilter {
+ Status status;
+ // Empty selects no rows; nullopt keeps the provider's native filter
for a broad result.
+ std::optional<std::vector<uint64_t>> row_ids;
Review Comment:
[P2] Track the cached row-ID buffer against query memory
A supported prefilter matching 65,536 rows retains at least 512 KiB in this
plain `std::vector`: `_read_row_id_prefilter()` moves the accumulated IDs into
`RowIdPrefilter`, and `ShardedKVCache` keeps them until `FileScanLocalState` is
destroyed. Standard allocations on this branch only pass through the jemalloc
hook, whereas `DorisVector` uses the allocator that charges the query tracker.
These retained buffers therefore go unaccounted across parallel scan instances,
so query memory reporting and limit enforcement miss their capacity. Use
tracked ownership for both the accumulation buffer and the cached IDs,
preserving the existing survivor bound.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
