[
https://issues.apache.org/jira/browse/SPARK-22491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16254431#comment-16254431
]
Liang-Chi Hsieh commented on SPARK-22491:
-----------------------------------------
For the query without aggregation, the exchanges are reused and there is only
one exchange. For the query with aggregation and adaptive execution is enabled,
the coordinated exchanges are not reused for now and it results three
exchanges. For the similar case, the coordinated exchanges can be reused. I've
opened a JIRA/PR SPARK-22527 for this issue.
> union all can't execute parallel with group by
> -----------------------------------------------
>
> Key: SPARK-22491
> URL: https://issues.apache.org/jira/browse/SPARK-22491
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.1.2
> Reporter: cen yuhai
> Priority: Minor
> Attachments: screenshot-1.png
>
>
> test sql
> set spark.sql.adaptive.enabled=true;
> {code}
> create table temp.test_yuhai_group
> as
> select
> city_name
> ,sum(amount)
> from temp.table_01
> group by city_name
> union all
> select
> city_name
> ,sum(amount)
> from temp.table_01
> group by city_name
> union all
> select city_name
> ,sum(amount)
> from temp.table_01
> group by city_name
> {code}
> if I remove group by ,it is ok
> {code}
> create table temp.test_yuhai_group
> as
> select
> city_name
> from temp.table_01
> union all
> select
> city_name
> from temp.table_01
> union all
> select city_name
> from temp.table_01
> {code}
> In the snapshot, the first time I execute ths sql, it run 3 job one by one
> (200 tasks per job).
> If I remove group by, it run just one job (600 tasks per job).
> workaround：
> set spark.sql.adaptive.enabled=false;
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
