Max Gekk created SPARK-57841:
--------------------------------
Summary: Add cross-precision and nested test coverage for
nanosecond-precision timestamps
Key: SPARK-57841
URL: https://issues.apache.org/jira/browse/SPARK-57841
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
precision).
h2. Problem
The cross-precision widening logic is implemented (SPARK-57454), but several
operations lack dedicated nanosecond e2e tests: join on mixed-precision keys,
{{INTERSECT}} / {{EXCEPT}}, {{BETWEEN}}, {{if()}} / {{nvl()}} / {{ifnull()}},
{{InSubquery}}; nested {{explode(array<ts_nanos>)}}, {{element_at}},
struct-field extraction, and map-with-nanosecond-key lookup (note
{{GetMapValue}}'s hash fast-path lists only microsecond timestamps); and
{{show()}} / {{collect()}} rendering at p=7/8/9.
h2. Goal
Lock in behavior with tests, fixing any latent issue surfaced (e.g. the map-key
hash fast-path).
h2. Scope
Add e2e tests to the widening SQL suites and golden files; add nested-type
tests; add show/collect precision assertions.
h2. Acceptance criteria
* Tests cover the listed operations at nanosecond precision and pass.
h2. Testing
{{TimestampNanosWideningSuite}}, {{timestamp-*-nanos.sql}},
collection/complex-type suites.
h2. Dependencies
Do AFTER SPARK-57811 (string <-> nanosecond coercion) for the string-comparison
cases; otherwise exercises the resolved widening and the function sub-tasks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
