Max Gekk created SPARK-57818:
--------------------------------
Summary: Support nanosecond-precision and LTZ timestamps in
convert_timezone
Key: SPARK-57818
URL: https://issues.apache.org/jira/browse/SPARK-57818
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
precision).
h2. Problem
{{ConvertTimezone}} (datetimeExpressions.scala ~L4192-4203) types its source as
{{TimestampNTZType}} only and returns {{TimestampNTZType}}, using
{{convertTimestampNtzToAnotherTz(Long, ...)}}. It rejects all nanosecond
timestamp types.
h2. Goal
Support {{TIMESTAMP_NTZ(p)}} at nanosecond precision (keeping the existing NTZ
contract), preserving {{nanosWithinMicro}}.
h2. Scope
Accept {{AnyTimestampNanoType}} (NTZ nanos) as source/result; add a
{{TimestampNanosVal}} conversion path; keep the microsecond NTZ path intact.
h2. Acceptance criteria
* {{convert_timezone(sourceTz, targetTz, ts_ntz_nanos)}} returns a nanosecond
NTZ value at the same precision with the remainder preserved.
h2. Testing
{{DateExpressionsSuite}}; nanos golden files.
h2. Dependencies
None - independent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
