Ramin Gharib created FLINK-40794:
------------------------------------
Summary: CAST in Table API ignores input nullability when the
target type is NOT NULL
Key: FLINK-40794
URL: https://issues.apache.org/jira/browse/FLINK-40794
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Reporter: Ramin Gharib
Assignee: Ramin Gharib
In the Table API, a CAST to a target written with .notNull() is planned as NOT
NULL even when the input may be null. The CAST type strategy already derives
the result as nullable, but CastConverter builds the RexNode from the literal
target type. The optimizer trusts the wrong NOT NULL and can fold null checks
away, which drops rows or produces wrong values.
h3. Reproduce
{code:java}
Table t = tEnv.fromValues(
ROW(FIELD("i", INT())),
Row.of((Object) null),
Row.of(1));
t.select($("i").cast(BIGINT().notNull()).as("c"))
.filter($("c").isNull())
.execute()
.print();
{code}
Expected: one row with c = NULL.
Actual: empty result. The plan collapses to an empty Values because IS NULL is
folded to FALSE.
The same happens for a nested VARIANT input, e.g.
parseJson(s).cast(ARRAY(VARIANT())).cast(ARRAY(INT()).notNull()). FLINK-40672
only covered a top-level VARIANT input.
SQL is not affected. A SQL CAST cannot declare NOT NULL on the target, and
SqlCastFunction.deriveType takes the nullability from the input.
h3. Proposed fix
CastConverter should use the resolved output type of the call, which the type
strategy derives from the input, instead of the literal target type. This
aligns the Table API with SQL for all casts. As a side effect, casting a NOT
NULL input to a nullable target now yields NOT NULL, as it already does in SQL.
Also document on cast() that the top-level nullability of the target type is
ignored.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
