Yossi Eliaz created SPARK-60099:
-----------------------------------
Summary: Pin the XML reader's StAX input factory instead of
resolving it through the service loader
Key: SPARK-60099
URL: https://issues.apache.org/jira/browse/SPARK-60099
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 5.0.0
Reporter: Yossi Eliaz
StaxXmlParserUtils.factory and XmlOptions.buildXmlFactory() call
XMLInputFactory.newInstance(). That call picks the implementation through
META-INF/services, so any jar on the classpath that registers a StAX provider
changes the parser that Spark's XML reader uses.
snowflake-jdbc 4.4.0 registers its shaded Woodstox as the provider for
javax.xml.stream.XMLInputFactory, XMLOutputFactory, and XMLEventFactory. With
that jar on the sql/core test classpath, XmlSuite parsed with Woodstox. Nested
arrays came back as _corrupt_record, and numeric type inference changed.
StaxXmlGenerator already avoids this for output. It constructs the shaded
WstxOutputFactory directly instead of calling XMLOutputFactory.newInstance().
https://github.com/apache/spark/pull/59155 works around the problem in tests
only. The sbt build and sql/core/pom.xml rewrite the snowflake-jdbc test jar
without those three service files. If the read path builds a fixed factory the
way StaxXmlGenerator does, both rewrites can be deleted.
The same lookup runs outside tests, so a user jar that registers a StAX
provider would likely change XML parsing in the same way. That case is not
reproduced yet.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
