[
https://issues.apache.org/jira/browse/SPARK-59528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uroš Bojanić resolved SPARK-59528.
----------------------------------
Fix Version/s: 4.4.0
Resolution: Fixed
Issue resolved by pull request 58827
[https://github.com/apache/spark/pull/58827]
> ASOF JOIN MATCH_CONDITION rejects arrays with coercible element types
> ---------------------------------------------------------------------
>
> Key: SPARK-59528
> URL: https://issues.apache.org/jira/browse/SPARK-59528
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.4.0
> Reporter: Luka Zdravic
> Assignee: Luka Zdravic
> Priority: Minor
> Labels: pull-request-available
> Fix For: 4.4.0
>
>
> Inside a MATCH_CONDITION, comparing array<int> to array<long> fails with
> ASOF_JOIN_MATCH_CONDITION_INVALID_TYPE, because ASOF requires identical array
> element types. The same comparison outside ASOF works: Spark coerces the
> element types. ASOF should apply the same coercion.
> -- fails:
> SELECT * FROM (VALUES (array(1))) l(a)
> ASOF JOIN (VALUES (array(CAST(1 AS BIGINT)))) r(b)
> MATCH_CONDITION (l.a >= r.b); -- ASOF_JOIN_MATCH_CONDITION_INVALID_TYPE
> -- works:
> SELECT array(1) >= array(CAST(1 AS BIGINT)); -- true
> Expected: MATCH_CONDITION accepts arrays whose element types differ but are
> coercible, same as the comparison operator.
> Reproduces under ANSI on and off.
> Impact: a valid array as-of condition that is a legal comparison cannot be
> written.
>
> Umbrella: SPARK-59526
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
