[
https://issues.apache.org/jira/browse/SPARK-57810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-57810:
-----------------------------
Labels: starter (was: )
> Infer nanosecond-precision timestamp types in the XML schema inference
> ----------------------------------------------------------------------
>
> Key: SPARK-57810
> URL: https://issues.apache.org/jira/browse/SPARK-57810
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Max Gekk
> Priority: Major
> Labels: starter
>
> This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
> precision).
> h2. Problem
> {{XmlInferSchema.compatibleType}} (datasources/xml/XmlInferSchema.scala
> ~L653-669) has a stale TODO and downgrades an inferred nanosecond+datetime
> combination to {{TimestampType}}, inconsistent with {{findWiderDateTimeType}}
> (SPARK-57454). XML read/write with an explicit schema is done (SPARK-57458);
> this gap is inference-only.
> h2. Goal
> Use the shared {{findWiderDateTimeType}} in XML schema inference so inferred
> nanosecond precision is preserved.
> h2. Scope
> Replace the microsecond-only downgrade in {{compatibleType}} with the shared
> widening.
> h2. Acceptance criteria
> * Schema inference over XML with sub-microsecond timestamps infers a
> nanosecond timestamp type.
> h2. Testing
> {{XmlInferSchemaSuite}} / XML suite.
> h2. Dependencies
> None - independent (XML explicit-schema support resolved in SPARK-57458;
> widening resolved in SPARK-57454).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
