[
https://issues.apache.org/jira/browse/FLINK-40255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Nuyanzin resolved FLINK-40255.
-------------------------------------
Fix Version/s: 2.4.0
Resolution: Fixed
> COMPILE PLAN fails for any plan containing a VARIANT type
> ---------------------------------------------------------
>
> Key: FLINK-40255
> URL: https://issues.apache.org/jira/browse/FLINK-40255
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Reporter: Ramin Gharib
> Assignee: Ramin Gharib
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0
>
>
> VARIANT plans and executes correctly, but serializing a plan containing it
> throws. {{{}COMPILE PLAN{}}}, {{StatementSet#compilePlan}} and
> {{TableEnvironment#compilePlanSql}} are unusable with VARIANT, as is plan
> restore.
> h3. Symptoms
> A VARIANT-typed {{{}RexNode{}}}, for example the return type of
> {{PARSE_JSON}} in a projection:
>
> {code:java}
> TableException: Unsupported RelDataType: VARIANT ExecNodeGraphJsonSerializer
> -> RexNodeJsonSerializer#serializeCall -> RelDataTypeJsonSerializer ->
> LogicalRelDataTypeConverter#toLogicalTypeNotNull{code}
> A VARIANT in an {{ExecNode}} output type:
> {code:java}
> TableException: Unable to serialize logical type 'VARIANT'.
> ExecNodeGraphJsonSerializer -> LogicalTypeJsonSerializer{code}
> h3. Root cause
> Two {{RelDataType}} to {{LogicalType}} converters exist and only one knows
> VARIANT. {{{}o.a.f.table.planner.calcite.FlinkTypeFactory{}}}, used during
> planning, handles it at {{FlinkTypeFactory.java:490-491}} and {{{}:861{}}}.
> {{{}o.a.f.table.planner.typeutils.LogicalRelDataTypeConverter{}}}, used only
> by plan serde, has no VARIANT handling at all. Nothing forces the two to
> agree.
> h3. Locations on master
> under {{{}flink-table/flink-table-planner/.../table/planner/{}}}:
> * {{typeutils/LogicalRelDataTypeConverter.java:609}} —
> {{toLogicalTypeNotNull}} hits {{default:}} and throws
> * {{typeutils/LogicalRelDataTypeConverter.java:473}} — reverse direction
> throws, breaks restore
> * {{plan/nodes/exec/serde/LogicalTypeJsonSerializer.java:507-550}} — VARIANT
> missing from {{CompactSerializationChecker}}
> * {{plan/nodes/exec/serde/LogicalTypeJsonSerializer.java:253}} and
> {{LogicalTypeJsonDeserializer.java:169-170}} — no {{case VARIANT}}
> h3. Fix
> Handle VARIANT in {{LogicalRelDataTypeConverter}} both ways and add it to
> {{{}CompactSerializationChecker{}}}; that is sufficient, since the compact
> path already round-trips VARIANT via {{{}LogicalTypeParser:585-586{}}}. The
> explicit switch cases are cheap defensive coverage.
> Add VARIANT to {{LogicalTypeJsonSerdeTest}} and
> {{{}RelDataTypeJsonSerdeTest{}}}, plus a {{restore-tests}} program, all at
> zero coverage today. {{VariantSemanticTest}} cannot catch this because
> {{SemanticTestBase}} never serializes a plan.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
