[
https://issues.apache.org/jira/browse/FLINK-40263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40263:
-----------------------------------
Labels: pull-request-available (was: )
> Enum symbol validation
> ----------------------
>
> Key: FLINK-40263
> URL: https://issues.apache.org/jira/browse/FLINK-40263
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 2.4.0
> Reporter: Pritam Kumar
> Assignee: Pritam Kumar
> Priority: Major
> Labels: pull-request-available
>
> h2. Problem
> FLINK-39053 taught {{RowDataToAvroConverters}} to map a
> {{CHAR}}/{{VARCHAR}} column onto an Avro {{ENUM}} field:
> {code:java}
> if (schema.getType() == Schema.Type.ENUM) {
> return new GenericData.EnumSymbol(schema, object.toString());
> }
> {code}
> {{GenericData.EnumSymbol}} does not validate its argument. A string that is
> not one of the declared symbols therefore survives the conversion and only
> fails much later, inside {{GenericDatumWriter}}:
> {noformat}
> org.apache.avro.AvroTypeException: Not an enum: ...
> {noformat}
> wrapped in {{Failed to serialize row.}} That message names neither the
> column, nor the offending value, nor the symbols that would have been
> accepted, which makes it hard to act on for a value that occurs once in a
> stream.
> Separately, the enum {{"default"}} attribute (Avro 1.9+) is ignored. Avro's
> own schema resolution maps an unknown symbol onto the declared default; a
> schema author who sets one is explicitly asking for that, but here the job
> fails instead.
> h2. Proposal
> Validate before constructing the symbol:
> * a declared symbol converts exactly as today;
> * an unknown symbol falls back to {{Schema#getEnumDefault()}} when the enum
> declares one;
> * otherwise fail with a message naming the value, the enum and the allowed
> symbols.
> h2. Note
> FLINK-39053 is not in a release yet, so this can land before 2.4.0 ships
> and no released behaviour changes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
