Josh Rosen created SPARK-20329:
----------------------------------
Summary: Resolution error when HAVING clause uses GROUP BY
expression that involves implicit type coercion
Key: SPARK-20329
URL: https://issues.apache.org/jira/browse/SPARK-20329
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.2.0
Reporter: Josh Rosen
Priority: Blocker
The following example runs without error on Spark 2.0.x and 2.1.x but fails in
the current Spark master:
{code}
create temporary view foo (a, b) as values (cast(1 as bigint), 2), (cast(3 as
bigint), 4);
select a + b from foo group by a + b having (a + b) > 1
{code}
The error is
{code}
Error in SQL statement: AnalysisException: cannot resolve '`a`' given input
columns: [(a + CAST(b AS BIGINT))]; line 1 pos 45;
'Filter (('a + 'b) > 1)
+- Aggregate [(a#249243L + cast(b#249244 as bigint))], [(a#249243L +
cast(b#249244 as bigint)) AS (a + CAST(b AS BIGINT))#249246L]
+- SubqueryAlias foo
+- Project [col1#249241L AS a#249243L, col2#249242 AS b#249244]
+- LocalRelation [col1#249241L, col2#249242]
{code}
I think what's happening here is that the implicit cast is breaking things: if
we change the types so that both columns are integers then the analysis error
disappears. Similarly, adding explicit casts, as in
{code}
select a + cast(b as bigint) from foo group by a + cast(b as bigint) having (a
+ cast(b as bigint)) > 1
{code}
works so I'm pretty sure that the resolution problem is being introduced when
the casts are automatically added by the type coercion rule.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
