Max Gekk created SPARK-57816:
--------------------------------
Summary: Support nanosecond-precision timestamps in date_format,
to_char and to_varchar
Key: SPARK-57816
URL: https://issues.apache.org/jira/browse/SPARK-57816
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
precision).
h2. Problem
{{DateFormatClass}} (datetimeExpressions.scala ~L1235-1252) has {{inputTypes =
Seq(TimestampType, AnyTimeType)}} (or string) and formats
{{timestamp.asInstanceOf[Long]}} micros; {{ToCharacterBuilder}}
(numberFormatExpressions.scala ~L255-256) routes any {{DatetimeType}} (which
includes the nanosecond types) into {{DateFormatClass}}. Nanosecond input is
therefore narrowed to microseconds and the sub-microsecond digits never reach
the formatter.
h2. Goal
Format nanosecond timestamps at full precision, honoring the pattern's
fractional-second placeholders up to 9 digits, with correct NTZ
(zone-independent) vs LTZ (session-zone) rendering.
h2. Scope
Accept {{AnyTimestampNanoType}} in {{DateFormatClass.inputTypes}}; add a
{{TimestampNanosVal}} formatting path reusing the nanosecond-aware
{{TimestampFormatter}} ({{formatNanos}} / {{formatWithoutTimeZoneNanos}}); wire
{{to_char}} / {{to_varchar}} accordingly.
h2. Acceptance criteria
* {{date_format(ts_nanos, "...SSSSSSSSS")}} and {{to_char(ts_nanos, ...)}}
render up to nanosecond digits; NTZ vs LTZ zone behavior matches the
microsecond types.
h2. Testing
{{DateFunctionsSuite}} / {{DateExpressionsSuite}}; nanos golden files.
h2. Dependencies
None - independent (uses the resolved nanosecond-aware formatter, SPARK-57162).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
