e hundreds of partitions to union, creating a temp view
> for each of them might be slow?
>
> Sent using Zoho Mail <https://www.zoho.com/mail/>
>
>
> Forwarded message
> From : kathleen li
> To :
> Cc :
> Date : Wed, 21 Nov 20
tions to union, creating a temp view for each of them
might be slow? Sent using Zoho Mail Forwarded message
From : kathleen li To :
Cc : Date : Wed, 21 Nov 2018 10:16:21 +0330 Subject :
Re: spark-sql force parallel union Forwarded message
you
you might first write the code to construct query statement with "union
all" like below:
scala> val query="select * from dfv1 union all select * from dfv2 union all
select * from dfv3"
query: String = select * from dfv1 union all select * from dfv2 union all
select * from dfv3
then write loop to
