[
https://issues.apache.org/jira/browse/FLINK-40236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu closed FLINK-40236.
---------------------------
Fix Version/s: 1.20.6
2.3.1
2.4.0
Resolution: Fixed
Fixed in:
- master via ea801695739a4238cf2edfec1ff855100cb1e58b
- release-2.3 via 7365c778d0d9f4781a1e5ad600e1369116eca1f4
- release-1.20 via 029a14f151850ca46bb225b9e8d6b3cb151b9436
> _infer_type infers ARRAY<NULL> for a list whose first element is None
> ---------------------------------------------------------------------
>
> Key: FLINK-40236
> URL: https://issues.apache.org/jira/browse/FLINK-40236
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Reporter: Nikolaus Schuetz
> Assignee: Nikolaus Schuetz
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.20.6, 2.3.1, 2.4.0
>
>
> [_infer_type|https://github.com/apache/flink/blob/release-2.3.0/flink-python/pyflink/table/types.py#L1494]
> in *flink-python/pyflink/table/types.py* infers a list's element type from
> obj[0] instead of the first non-None element it scans for. When the first
> element is None, the
> array element type collapses to NULL, so schema inference becomes
> order-dependent:
> {code}
> _infer_type([1, None]) -> ArrayType(BigIntType) # correct
> _infer_type([None, 1]) -> ArrayType(NullType) # wrong; should be
> BigIntType
> {code}
> The dict branch just above handles this correctly (it infers from the found
> non-None value). from_elements([Row(c=[None, 1])]) without an explicit schema
> infers ARRAY<NULL>, which
> then typically errors downstream.
> Fix: infer from the scanned non-None element v rather than obj[0]. Verified
> against the released apache-flink 2.3.0 wheel: the array element type is
> NullType before the fix and
> BigIntType after.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
