Max Gekk created SPARK-57811:
--------------------------------
Summary: Support string to nanosecond-precision timestamp coercion
in comparisons and predicates
Key: SPARK-57811
URL: https://issues.apache.org/jira/browse/SPARK-57811
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
precision).
h2. Problem
{{findCommonTypeForBinaryComparison}} (analysis/TypeCoercion.scala ~L144-147)
special-cases only {{TimestampType + StringType}}, and
{{StringPromotionTypeCoercion}} (~L58-61) matches only the microsecond
{{TimestampTypeExpression}} (DataTypeExpression.scala ~L45). A string operand
compared to a nanosecond timestamp column is therefore not coerced like the
microsecond case: non-ANSI may promote the timestamp to string, ANSI may fail.
Column-vs-column nanos/micro is already correct (SPARK-57454); this is the
string-literal path only.
h2. Goal
Coerce a string operand compared to a nanosecond timestamp column to the
nanosecond type (parity with microsecond), in both the binary-comparison
common-type path and the equality string-promotion path, for ANSI and non-ANSI.
h2. Scope
Extend {{findCommonTypeForBinaryComparison}} and
{{StringPromotionTypeCoercion}} (and the {{DataTypeExpression}} matchers) to
include {{AnyTimestampNanoType}}.
h2. Acceptance criteria
* {{ts_nanos = '2020-01-02 03:04:05.123456789'}} and range predicates resolve
by casting the string to the nanosecond type; results correct in both ANSI and
non-ANSI modes.
h2. Testing
{{TypeCoercionSuite}} / {{AnsiTypeCoercionSuite}}; nanos golden files.
h2. Dependencies
None - independent (widening resolved in SPARK-57454). Feeds the
cross-precision test-coverage sub-task.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
