Max Gekk created SPARK-57850:
--------------------------------
Summary: Return the correct fractional-second scale from
EXTRACT(SECOND FROM TIME(p)) for p > 6
Key: SPARK-57850
URL: https://issues.apache.org/jira/browse/SPARK-57850
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
This sub-task is part of the umbrella SPARK-57550 (extend support for the TIME
data type).
h2. Problem
{{SecondsOfTimeWithFraction}} (timeExpressions.scala ~L364-377;
DateTimeUtils.scala ~L173-178) always returns {{DECIMAL(8,6)}}, truncating
sub-microsecond digits for {{TIME(7..9)}}. ANSI requires the {{EXTRACT(SECOND
...)}} result scale to be at least the source fractional-seconds precision (std
~L15080-15082). This is now relevant since TIME precision goes to 9
(SPARK-57551).
h2. Goal
Make the {{EXTRACT(SECOND FROM TIME(p))}} / {{date_part}} result decimal scale
track the TIME precision (e.g. {{DECIMAL(11,9)}} for p in [7,9]) so nanoseconds
are preserved. Mirror the nanosecond-timestamp {{SecondWithFractionNanos}}
behavior (SPARK-57340).
h2. Scope
Make {{SecondsOfTimeWithFraction}} (and its {{DateTimeUtils}} helper) return a
decimal whose scale is derived from the TIME precision.
h2. Acceptance criteria
* {{EXTRACT(SECOND FROM TIME(9) '..:..:05.123456789')}} returns
{{5.123456789}}; p<=6 unchanged.
h2. Testing
{{TimeExpressionsSuite}}; {{extract}} / {{date_part}} golden files.
h2. Dependencies
Builds on the resolved precision->9 (SPARK-57551).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
