Dear devs,
Several issues [1][2][3] have been identified regarding the inconsistent
treatment of ROW type nullability between SQL and TableAPI. However,
addressing these discrepancies might necessitate updates to the public API.
Therefore, I'm initiating this discussion to engage the community in
forging a unified approach to resolve these challenges.
To summarize, SQL prohibits ROW types such as ROW<a INT NOT NULL, b
STRING>, which is implicitly rewritten to ROW<a INT, b STRING> by
Calcite[4]. In contrast, TableAPI permits such types, resulting in
inconsistency.
[image: image.png]
For a comprehensive issue breakdown, please refer to the comment of [1].
According to CALCITE-2464[4], ROW<f0 INT NOT NULL> is not a valid type. As
a result, the behavior of TableAPI is incorrect and needs to be consistent
with SQL, which means the implantation for the following public API needs
to be changed.
- RowType#copy(boolean nullable) should also set the inner fields to
null if nullable is true.
- RowType's constructor should also check nullability.
- FieldsDataType#nullable() should also set the inner fields to null.
In addition to the necessary changes in the implementation of the
aforementioned API, the semantics of the following API have also been
impacted.
- `DataTypes.ROW(DataTypes.FIELD("f0",
DataTypes.INT().notNull())).notNull()` cannot create a type like `ROW<INT
NOT NULL>NOT NULL`.
- Idempotence for chained calls `notNull().nullable().notNull()` for
`Row` cannot be maintained.
Sergey and I have engaged in a discussion regarding the solution [1]. I'm
interested in gathering additional perspectives on the fix.
Look forward to your ideas!
Best,
Jane
[1] https://issues.apache.org/jira/browse/FLINK-31830
[2] https://issues.apache.org/jira/browse/FLINK-31829
[3] https://issues.apache.org/jira/browse/FLINK-33217
[4] https://issues.apache.org/jira/browse/CALCITE-2464
