[
https://issues.apache.org/jira/browse/SPARK-58673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-58673:
-----------------------------------
Labels: pull-request-available (was: )
> Upgrade Avro to 1.12.2
> ----------------------
>
> Key: SPARK-58673
> URL: https://issues.apache.org/jira/browse/SPARK-58673
> Project: Spark
> Issue Type: Dependency upgrade
> Components: Build
> Affects Versions: 5.0.0
> Reporter: Ismaël Mejía
> Priority: Major
> Labels: pull-request-available
>
> Upgrade the Avro dependency from 1.12.1 to 1.12.2.
> Notable changes required by the upgrade:
> - Avro 1.12.2 enforces a ClassSecurityValidator / SERIALIZABLE_PACKAGES
> restriction when deserializing schemas. Spark tests that ship reader
> schemas to executors need "org.apache.spark" allowlisted via
> org.apache.avro.SERIALIZABLE_PACKAGES.
> - AVRO-4257 changes Schema.readResolve() to use the non-validating internal
> parser (JsonSchemaParser.parseInternal). As a result, a reader schema that
> carries invalid but unused default values is no longer re-validated after
> crossing the driver/executor boundary, so reads succeed end to end. This is
> consistent with the goal of SPARK-39775; the corresponding test is updated
> to assert the read returns the expected rows instead of failing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
