[
https://issues.apache.org/jira/browse/SPARK-59545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk resolved SPARK-59545.
------------------------------
Fix Version/s: 4.4.0
Resolution: Fixed
Issue resolved by pull request 58836
[https://github.com/apache/spark/pull/58836]
> AvroSchemaHelperSuite map-key overflow test fails deterministically on Java
> 25
> -------------------------------------------------------------------------------
>
> Key: SPARK-59545
> URL: https://issues.apache.org/jira/browse/SPARK-59545
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Max Gekk
> Assignee: Max Gekk
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.4.0
>
>
> h3. Problem
> Since the follow-up to SPARK-59311 (apache/spark#58722, merged 2026-09-11),
> one test in {{AvroSchemaHelperSuite}} fails on every Java 25 run of
> {{build_java25.yml}} on master (2026-09-13 and 2026-09-15), while the default
> Java 17 job passes:
> {code}
> - SPARK-59311: a pathologically nested map-key type names the map-key
> property on overflow *** FAILED ***
> java.lang.IndexOutOfBoundsException: Index -1 out of bounds for length 0
> was not instance of
> org.apache.spark.sql.avro.IncompatibleSchemaException expected
> IncompatibleSchemaException
> {code}
> The test parses a Catalyst type nested 1000 deep from the
> {{spark.sql.catalyst.mapKey.type}} property on a 256 KB thread stack and
> expects the StackOverflowError to surface as {{IncompatibleSchemaException}}
> naming that property. On JDK 25 the overflow is absorbed somewhere inside the
> ANTLR parse and what escapes is an {{IndexOutOfBoundsException}} from the
> parser's own state, so the conversion in {{SchemaConverters}} never sees a
> {{StackOverflowError}}.
> The sibling test "a pathologically nested Catalyst type fails as a schema
> error, not StackOverflowError" passes on the same JDK, so the difference is
> the map-key path.
> h3. Reproduction
> JDK 25 (openjdk 25.0.4), x86_64, three of three runs:
> {code}
> build/sbt "avro/testOnly *AvroSchemaHelperSuite"
> Tests: succeeded 9, failed 1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
