Max Gekk created SPARK-57810:
--------------------------------
Summary: Infer nanosecond-precision timestamp types in the XML
schema inference
Key: SPARK-57810
URL: https://issues.apache.org/jira/browse/SPARK-57810
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
precision).
h2. Problem
{{XmlInferSchema.compatibleType}} (datasources/xml/XmlInferSchema.scala
~L653-669) has a stale TODO and downgrades an inferred nanosecond+datetime
combination to {{TimestampType}}, inconsistent with {{findWiderDateTimeType}}
(SPARK-57454). XML read/write with an explicit schema is done (SPARK-57458);
this gap is inference-only.
h2. Goal
Use the shared {{findWiderDateTimeType}} in XML schema inference so inferred
nanosecond precision is preserved.
h2. Scope
Replace the microsecond-only downgrade in {{compatibleType}} with the shared
widening.
h2. Acceptance criteria
* Schema inference over XML with sub-microsecond timestamps infers a nanosecond
timestamp type.
h2. Testing
{{XmlInferSchemaSuite}} / XML suite.
h2. Dependencies
None - independent (XML explicit-schema support resolved in SPARK-57458;
widening resolved in SPARK-57454).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
