github-actions[bot] commented on code in PR #65498:
URL: https://github.com/apache/doris/pull/65498#discussion_r3566365948
##########
be/src/format_v2/table_reader.cpp:
##########
@@ -777,7 +780,10 @@ Status TableReader::prepare_split(const SplitReadOptions&
options) {
_current_task = std::make_unique<ScanTask>();
_current_task->data_file = create_file_description(options.current_range);
_current_file_description = *_current_task->data_file;
- if (_push_down_agg_type == TPushAggOp::type::COUNT &&
+ // A table-level row count is only equivalent to scanning the split when
no row predicate is
+ // active. In particular, a runtime filter may arrive after init() and
replace `_conjuncts`
+ // above. Returning synthetic rows here would bypass that fresh split
snapshot completely.
Review Comment:
The later-split case is fixed, but this shortcut can still span scheduler
turns within one large split. `_read_table_level_count()` emits at most
`runtime_state->batch_size()` rows and leaves `_remaining_table_level_count`
active with `eos=false`; if a runtime filter arrives before the next scheduler
read, `try_append_late_arrival_runtime_filter()` refreshes
`Scanner::_conjuncts`, but `FileScannerV2` does not call `prepare_split()`
again while `_has_prepared_split` is true. The remaining count rows therefore
stay on the synthetic metadata path, and the final scanner filter can only
evaluate the RF on default synthetic columns rather than real file rows. Please
either make the table-level count shortcut ineligible until all runtime filters
for the scanner have arrived, or reset/reprepare the active split when a late
RF arrives before emitting more metadata-count rows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
