[
https://issues.apache.org/jira/browse/SPARK-56043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-56043:
-----------------------------------
Labels: pull-request-available (was: )
> Wrap NullPointerException from Avro 1.12.x ParseContext.resolve() in
> SchemaParseException
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-56043
> URL: https://issues.apache.org/jira/browse/SPARK-56043
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 4.1.0, 4.0.0, 4.0.1, 4.0.2, 4.2.0, 4.1.1, 4.1.2, 4.0.3
> Reporter: Boyang Jerry Peng
> Priority: Major
> Labels: pull-request-available
>
> After the Avro 1.11.3 → 1.12.0 upgrade (SPARK-49014),
> {{Schema.Parser().parse()}} can throw {{NullPointerException}} from
> {{ParseContext.resolve()}} for certain invalid user-provided schemas. In Avro
> 1.11.x, the same schemas threw {{{}SchemaParseException{}}}, which was caught
> by the existing {{NonFatal(e)}} handler in
> {{AvroDataToCatalyst.nullSafeEval}} and reported as
> {{{}MALFORMED_AVRO_MESSAGE{}}}. The NPE bypasses nothing — it's still
> {{NonFatal}} — but it surfaces as an unclassified internal error rather than
> a schema parse error because it was never expected from schema parsing.
>
> *Root cause:* AVRO-3666 replaced the {{Names}} lookup (which threw
> {{{}SchemaParseException("Undefined name: ..."){}}}) with
> {{{}ParseContext.resolve(){}}}, which calls
> {{Objects.requireNonNull(oldSchemas.get(fullName))}} — throwing a raw
> {{NullPointerException}} when named types can't be resolved.
> *Verified behavioral difference* (standalone test against both jars):
> ||Schema pattern||Avro 1.11.3||Avro 1.12.1||
> |Undefined named type in record
> field|{{SchemaParseException}}|{{AvroTypeException}}|
> |Undefined type in union|{{SchemaParseException}}|{{AvroTypeException}}|
> |Bare string reference
> ({{{}"com.test.Missing"{}}})|{{SchemaParseException}}|*{{NullPointerException}}*|
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
