github-actions[bot] commented on code in PR #68649:
URL: https://github.com/apache/doris/pull/68649#discussion_r4135457648
##########
be/src/core/data_type_serde/data_type_datev2_serde.cpp:
##########
@@ -65,22 +62,44 @@ Status decode_date_orc_values(const DataTypeSerDe& serde,
IColumn& column,
orc_serde_utils::orc_decode_row_count(orc_view.rows,
orc_view.selected_rows);
std::vector<int32_t> date_values;
date_values.resize(output_rows);
- auto& date_dict = date_day_offset_dict::get();
for (size_t row = 0; row < output_rows; ++row) {
const auto source_row = orc_serde_utils::orc_source_row_at(row,
orc_view.selected_rows);
- const auto date =
date_dict[cast_set<int>(orc_batch->data[source_row])];
- date_values[row] = cast_set<int32_t>(date.daynr() -
DORIS_DATE_EPOCH_DAYNR);
+ // The payload of a null slot is undefined, so it is zeroed rather
than range-checked;
+ // `read_column_from_decoded_values` never decodes it.
+ if (view.null_map != nullptr && view.null_map[row] != 0) {
+ date_values[row] = 0;
+ continue;
+ }
+ // ORC DATE is days since 1970-01-01 in the proleptic Gregorian
calendar, the same encoding
+ // `decode_epoch_days()` expects, but ORC hands it over in an int64
batch. Reject a value
+ // that does not fit the int32 view here, while the real file value is
still available for
+ // the message, instead of narrowing it blindly.
+ const int64_t file_days = orc_batch->data[source_row];
Review Comment:
[P1] Guard ORC COUNT against DATE conversion errors. This decoder now
rejects -719469 (0000-02-29), but `OrcReader::get_aggregate_result(COUNT)`
still sums stripe `getNumberOfValues()` without checking the DATE data. For a
no-filter `COUNT(d)` over a valid date plus -719469, pushdown returns 2 while
materializing `d` fails. The same branch accepts `COUNT` of a STRUCT/ARRAY
containing DATE and skips its invalid child. This is a separate ORC path from
the existing Parquet COUNT thread. Before using stripe counts, prove every DATE
leaf of the COUNT argument has a representable range in each selected stripe,
including invalid interior ordinals, or fall back to row materialization; test
scalar and nested cases.
##########
be/src/format_v2/parquet/parquet_reader.cpp:
##########
@@ -1175,6 +1175,28 @@ Status ParquetReader::get_aggregate_result(const
format::FileAggregateRequest& r
}
const auto& count_projection = request.columns[0].projection;
const auto& root_schema = projected_root_schema(_state->file_schema,
count_projection);
+ if (remove_nullable(root_schema.type)->get_primitive_type() ==
TYPE_DATEV2) {
Review Comment:
[P2] Check nested DATE leaves before levels-only COUNT. This guard only
tests the root type, so `COUNT(a)` for a Parquet `ARRAY<DATE>` containing
-719469 bypasses it. FE permits the complex COUNT slot, and `CountColumnReader`
reads only levels and returns 1, while a strict materialized scan of `a` fails
during DATE conversion. The existing Parquet thread covers scalar `COUNT(d)`
and its logical NULL count; this complex-root error path is separate. In strict
mode, prove each selected DATE child range is representable or decline
pushdown, and add a nested invalid-ordinal regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
