github-actions[bot] commented on code in PR #65562:
URL: https://github.com/apache/doris/pull/65562#discussion_r3575716194
##########
be/src/format_v2/orc/orc_reader.cpp:
##########
@@ -605,8 +629,12 @@ bool build_zone_map_from_orc_statistics(const ::orc::Type&
type,
return set_string_zone_map(type, statistics, zone_map);
case ::orc::TypeKind::DATE:
return set_date_zone_map(statistics, zone_map);
- case ::orc::TypeKind::TIMESTAMP:
- return set_timestamp_zone_map(statistics, timezone, false, zone_map);
+ case ::orc::TypeKind::TIMESTAMP: {
+ // ORC stores timestamp statistics as wall-clock values in UTC
coordinates. Restore them
+ // with UTC so the session timezone does not shift the civil time.
+ static const auto utc_time_zone = cctz::utc_time_zone();
+ return set_timestamp_zone_map(statistics, utc_time_zone, false,
zone_map);
Review Comment:
Could we guard this path for legacy TIMESTAMP stats before using it for
MIN/MAX pushdown? ORC has two timestamp-stat encodings: pre-ORC-135 files
stored min/max with the local timezone offset, while post-ORC-135 added UTC
min/max fields. Here we always decode `TimestampColumnStatistics` with UTC, and
`get_aggregate_result()` uses that for every stripe without checking the ORC
writer version or stats capability. A normal scan still decodes row batches
through `_state->timezone_obj`, so for a legacy Hive/ORC file written in a
non-UTC timezone, aggregate pushdown can return shifted min/max values relative
to the full scan. ORC's own Java predicate pushdown disables TIMESTAMP stats
when the writer version does not include ORC-135 for the same compatibility
reason. Please either gate TIMESTAMP MIN/MAX pushdown to UTC-normalized stats
or return `NotSupported` when the file may use legacy timestamp stats, and add
a non-GMT/legacy fixture differential test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
