abhinav-phi opened a new pull request, #70724:
URL: https://github.com/apache/doris/pull/70724

   ### Background
   
   BE datetime parsing resolves time zone names through `TimezoneUtils`, which 
scans the system tzdata (`/usr/share/zoneinfo`, overridable via `TZDIR`) into a 
name cache and keeps a separate cache of fixed-offset zones. ISO 8601 / RFC 
3339 timestamps end in `Z` as the UTC designator, and today that designator 
only resolves when the system tzdata scan finds a legacy top-level `Zulu` file.
   
   ### What problem does this PR solve?
   
   Issue Number: close #69326
   Related PR: None
   PR Type: Bugfix
   
   Problem Summary: On hosts whose tzdata ships no top-level `Zulu` file (for 
example Ubuntu 24.04 ships only `Etc/Zulu`), `find_cctz_time_zone("Z")` and 
`parse_tz_offset_string("Z")` both fail. A valid `2015-01-01T03:47:41Z` input 
then converts to NULL, and a VARIANT path predefined as DATETIME filters the 
NULL out, so the field silently disappears from the stored VARIANT value while 
the load still reports success. The fix treats `z`/`zulu` as UTC aliases in 
`normalize_timezone_name` and seeds the name cache with UTC for both names 
unconditionally in `load_timezones_to_cache`, so the designator no longer 
depends on any system tzfile. Most of the diff is the regression test.
   
   ### Details
   
   - **Frequency:** every load or query that feeds a `Z`-suffixed timestamp 
into BE datetime parsing on hosts without the legacy `Zulu` tzfile; the VARIANT 
typed-path drop is the visible data-loss symptom.
   - **Root cause:** `load_timezones_to_cache()` installed the `z` cache alias 
only conditionally (`if (find("zulu")) emplace("z")`), and 
`normalize_timezone_name()` did not accept `z` at all, so with no system `Zulu` 
file both the cache path and the offset-parse path failed.
   - **Why existing tests missed it:** the test environment's tzdata contains a 
`Zulu` entry, so the conditional alias was always installed there; nothing 
exercised the "no Zulu file" configuration.
   - **Fix:** `normalize_timezone_name` now maps `z` to UTC alongside 
`utc`/`etc/utc`/`zulu`, and `load_timezones_to_cache` unconditionally emplaces 
`z` and `zulu` as `cctz::utc_time_zone()` instead of copying from a 
system-provided `zulu` entry. `find_cctz_time_zone("Z")` now works on an empty 
named cache too, because the normalized name falls through to the 
`parse_tz_offset_string` UTC shortcut.
   - **Other paths and branches:** `try_get_fixed_offset_seconds` benefits 
automatically, since a `Z`-resolved zone is `cctz::utc_time_zone()` whose name 
is `UTC` and reports offset 0. No other call sites changed.
   
   ### Testing
   
   New unit test `TimezoneUtilsTest.UtcZDesignatorResolvesToUtc` in 
`be/test/util/timezone_utils_test.cpp`: it asserts 
`parse_tz_offset_string("Z"/"z")` succeeds, then runs 
`find_cctz_time_zone("Z"/"z"/"zulu")` both with an emptied cache and with the 
preloaded cache, asserting the resolved zone is `UTC` in each case. On master 
(before the fix) `parse_tz_offset_string("Z")` returns false and the uncached 
`find_cctz_time_zone("Z")` fails, so the test fails before the fix and passes 
after it. No local BE build is available in my environment (Windows host), so 
the suite itself runs in CI.
   
   ### Notes for Reviewers
   
   The conditional `zulu`-copy block in `load_timezones_to_cache` was replaced 
rather than extended; `emplace` keeps any system-provided `zulu` entry intact 
since UTC has the same value. CST special case untouched.
   
   ### Release note
   
   Timestamps ending in `Z` now load and query correctly on hosts without the 
legacy `Zulu` tzfile, and VARIANT DATETIME typed paths no longer silently drop 
such fields.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to