github-actions[bot] commented on code in PR #65934:
URL: https://github.com/apache/doris/pull/65934#discussion_r3635129816
##########
be/src/format_v2/parquet/reader/native/column_chunk_reader.cpp:
##########
@@ -672,6 +673,23 @@ class PlainPredicateConsumer final : public
ParquetFixedValueConsumer {
_data_type,
_column_id,
_matches->data() + old_size));
}
+ if (_projected_column != nullptr) {
+ size_t row = 0;
+ while (row < num_values) {
+ while (row < num_values && (*_matches)[old_size + row] == 0) {
+ ++row;
+ }
+ const size_t run_begin = row;
+ while (row < num_values && (*_matches)[old_size + row] != 0) {
+ ++row;
+ }
+ if (row != run_begin) {
+ _projected_column->insert_many_raw_data(
Review Comment:
[P2] Keep native timing complete on projected direct scans
Projected predicate columns now append survivors in this branch and no
longer pass through `materialize_values()`, which is where `decode_value_time`
and `materialization_time` are incremented. `filter_plain_values()` calls
`decode_selected_fixed_values()` and this line copies survivors, but neither
contributes to the separately published
`DecodeValueTime`/`MaterializationTime`, so every eligible projected direct
scan underreports both stages. Please mirror the existing timing convention: an
inclusive batch/span decode timer around the direct decoder/consumer call and a
nested materialization timer around projected survivor writes, avoiding per-row
timers.
##########
be/src/format_v2/parquet/reader/native/column_chunk_reader.cpp:
##########
@@ -672,6 +673,23 @@ class PlainPredicateConsumer final : public
ParquetFixedValueConsumer {
_data_type,
_column_id,
_matches->data() + old_size));
}
+ if (_projected_column != nullptr) {
Review Comment:
[P2] Avoid per-match-run column appends
With alternating predicate results, this loop invokes virtual
`insert_many_raw_data()` once per survivor (`N/2` calls), and each call resizes
the vector before copying only 4/8 bytes; nullable output also grows its null
map and nested column each time. The previous projected path materialized the
batch and used vectorized filtering, so ordinary unsorted values can turn this
fast path into thousands of virtual calls and tiny copies per batch. Both added
tests have a single contiguous survivor run. Please reserve/grow once and
compact survivors into the destination in a bounded batch/span operation, and
cover an alternating-match case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
