numbers in near future.
Thanks,
Yucai
From: "Wang, Carson"
Date: Tuesday, July 31, 2018 at 4:06 PM
To: Marco Gaido , Wenchen Fan
Cc: "xyliyuanj...@gmail.com" , Spark dev list
Subject: RE: [DISCUSS] Adaptive execution in Spark SQL
Thanks Marco and Wenchen for reviewing. It s
sql-adaptive-execution-at-100-tb>.
Thanks,
Carson
From: Marco Gaido [mailto:marcogaid...@gmail.com]
Sent: Tuesday, July 31, 2018 3:00 PM
To: Wenchen Fan
Cc: xyliyuanj...@gmail.com; Wang, Carson ; Spark dev
list
Subject: Re: [DISCUSS] Adaptive execution in Spark SQL
Hi all,
I also like thi
Hi all,
I also like this idea very much and I think it may bring also other
performance improvements in the future.
Thanks to everybody who worked on this.
I agree to target this feature for 3.0.
Thanks everybody,
Bests.
Marco
On Tue, 31 Jul 2018, 08:39 Wenchen Fan, wrote:
> Hi Carson and Yu
Hi Carson and Yuanjian,
Thanks for contributing to this project and sharing the production use
cases! I believe the adaptive execution will be a very important feature of
Spark SQL and will definitely benefit a lot of users.
I went through the design docs and the high-level design totally makes
s
Thanks Carson, great note!
Actually Baidu has ported this patch in our internal folk. I collected some
user cases and performance improve effect during Baidu internal usage of
this patch, summarize as following 3 scenario:
1. SortMergeJoin to BroadcastJoin
The SortMergeJoin transform to BroadcastJo
