github-actions[bot] commented on code in PR #65602:
URL: https://github.com/apache/doris/pull/65602#discussion_r3579610115
##########
be/src/format_v2/parquet/parquet_scan.cpp:
##########
@@ -1314,8 +1333,17 @@ bool
ParquetScanScheduler::prepare_current_row_group_reader(
if (file_context.metadata == nullptr) {
return false;
}
- const auto ranges = build_row_group_prefetch_ranges(
- *file_context.metadata, file_schema,
request_scan_columns(request), row_group_idx);
+ // MergeRangeFileReader can merge forward from the range containing the
current ReadAt() into
+ // later registered ranges. Registering lazy chunks before predicate
evaluation would therefore
+ // let a predicate read fetch output bytes that a fully filtered row group
never materializes.
+ // Keep filtered scans predicate-only here. Reads for a surviving batch's
lazy columns fall
+ // outside these ranges and use the underlying reader directly; unfiltered
scans can still
+ // coalesce all projected chunks because every output column is guaranteed
to be consumed.
+ const auto merge_range_columns = request.conjuncts.empty() &&
request.delete_conjuncts.empty()
+ ? request_scan_columns(request)
Review Comment:
The predicate-only range list still is not enough to keep lazy chunks cold
for filtered scans. If the Parquet row group is laid out as `pred0,
lazy_output, pred1`, `build_row_group_prefetch_ranges()` registers the two
predicate chunks, but `MergeRangeFileReader` may bridge the gap between
registered ranges as hollow data and issue one `_fill_box()` read over `pred0 +
gap + pred1`. That gap can be the unregistered lazy output chunk, so an
all-filtered batch can still fetch output bytes before any survivor exists. The
added regression only proves the single-predicate-column case. Please prevent
filtered-scan merge-range from crossing unregistered gaps between predicate
ranges, or disable merge-range until the lazy side is allowed, and add a test
with two predicate chunks surrounding a lazy output chunk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
