raminqaf opened a new pull request, #29466:
URL: https://github.com/apache/flink/pull/29466
## What is the purpose of the change
Every `CAST(v AS T)` from `VARIANT` generated its own code, spread over six
planner rules. The write side already goes through `ToVariantConverter` since
FLINK-40826. This PR adds its read-side twin, `FromVariantConverter`, so the
read logic lives in one place in flink-table-common, where a format can reuse
it.
Generated code for `CAST(v AS ARRAY<INT>)`, before:
```java
int arraySize$2 = VariantCastUtils.arraySize(variant$1, "ARRAY<INT>");
java.lang.Integer[] objArray$3 = new java.lang.Integer[arraySize$2];
for (int i$4 = 0; i$4 < arraySize$2; i$4++) {
Variant element$5 = variant$1.getElement(i$4);
if (!element$5.isNull()) {
result$6 = ((int) VariantCastUtils.toIntegral(element$5,
-2147483648L, 2147483647L, "INTEGER"));
objArray$3[i$4] = result$6;
}
}
result$0 = new GenericArrayData(objArray$3);
```
After:
```java
result$1 =
((org.apache.flink.table.data.ArrayData)(fromVariantConverter$2.convert(variant$0)));
```
This is a refactor. Every result, error message and null rule stays the
same, and the existing tests pass unchanged apart from the rule names in
`CastRuleProviderTest`.
## Brief change log
- Add `FromVariantConverter` to flink-table-common. `create(LogicalType,
ZoneId)` builds a tree of readers once per target type, and `convert(Variant)`
returns internal data.
- Merge `VariantToPrimitiveCastRule`, `VariantToArrayCastRule`,
`VariantToRowCastRule`, `VariantToMapCastRule`, `VariantToStringCastRule` and
`AbstractVariantToConstructedCastRule` into `FromVariantCastRule`. Printing
still calls `VariantCastUtils#toPrintString`, because a printed value must
never fail.
- Move `VariantCastUtils` from flink-table-runtime to flink-table-common,
because the converter calls it. The class is `@Internal`. Only its package
changes, apart from the CHAR padding in `variantKey`, which no longer uses
runtime's `BinaryStringDataUtil`. The new padding gives the same bytes.
- Add `CodeGeneratorCastRule.Context#getSessionZoneId`, mirroring
`CastRule.Context#getSessionZoneId`. The converter needs the session zone when
it is created.
Internal differences reviewers may notice:
- A primitive target now boxes its value through the converter, the same
trade-off as the write side.
- A `ROW` result is a `GenericRowData` instead of a `BinaryRowData`, because
`BinaryRowWriter` is not visible from flink-table-common. Every `VARIANT` below
a `ROW` is still copied out of its parent, as the binary row did.
- An `ARRAY` of a `NOT NULL` primitive holds boxed values instead of a
primitive array. `toIntArray()` and the other accessors still work.
- A `DISTINCT` target casts like its source type. It used to fail during
code generation. SQL cannot create such a type.
## Verifying this change
This change is covered by the existing tests, such as `CastRulesTest`,
`CastRuleProviderTest`, `CastFunctionITCase`, `CastFunctionMiscITCase`,
`VariantSemanticTest`, `JsonFunctionsITCase` and
`PushFilterIntoTableSourceScanRuleTest`.
It also adds:
- `FromVariantConverterTest`, covering what the planner tests do not reach:
CHAR and VARCHAR map keys, typed arrays, the `ROW` copy rule, a top-level
variant null, `DISTINCT` and `STRUCTURED` targets, Java serialization, and a
round trip through `ToVariantConverter` over a type matrix.
- A `CastFunctionITCase` case for `CAST(CAST(f5 AS VARIANT) AS STRING)`,
which renders a `TIMESTAMP_LTZ` in the session time zone.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): yes. A cast
from `VARIANT` now calls the converter, and a primitive target boxes its value.
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code 2.1.296 (Claude Opus 5.5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
