> 2014/08/08 15:14
> To
> Jun Feng Liu/China/IBM@IBMCN,
> cc
> Patrick Wendell , "dev@spark.apache.org" <
> dev@spark.apache.org>
> Subject
> Re: Fine-Grained Scheduler on Yarn
>
>
>
>
> Hi Jun,
>
> Spark currently doesn't have that f
cc
Patrick Wendell , "dev@spark.apache.org"
Subject
Re: Fine-Grained Scheduler on Yarn
Hi Jun,
Spark currently doesn't have that feature, i.e. it aims for a fixed number
of executors per application regardless of resource usage, but it's
definitely worth considering. We co
om*
> [image: IBM]
>
> BLD 28,ZGC Software Park
> No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193
> China
>
>
>
>
>
> *Patrick Wendell >*
>
> 2014/08/08 13:10
> To
> Jun Feng Liu/China/IBM@IBMCN,
> cc
> "dev@spark.apache.org&
t;
Subject
Re: Fine-Grained Scheduler on Yarn
Hey sorry about that - what I said was the opposite of what is true.
The current YARN mode is equivalent to "coarse grained" mesos. There is no
fine-grained scheduling on YARN at the moment. I'm not sure YARN supports
s
Hey sorry about that - what I said was the opposite of what is true.
The current YARN mode is equivalent to "coarse grained" mesos. There is no
fine-grained scheduling on YARN at the moment. I'm not sure YARN supports
scheduling in units other than containers. Fine-grained scheduling requires
sche
The current YARN is equivalent to what is called "fine grained" mode in
Mesos. The scheduling of tasks happens totally inside of the Spark driver.
On Thu, Aug 7, 2014 at 7:50 PM, Jun Feng Liu wrote:
> Any one know the answer?
>
> Best Regards
>
>
> *Jun Feng Liu*
> IBM China Systems & Technolog
Any one know the answer?
Best Regards
Jun Feng Liu
IBM China Systems & Technology Laboratory in Beijing
Phone: 86-10-82452683
E-mail: liuj...@cn.ibm.com
BLD 28,ZGC Software Park
No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193
China
Jun Feng Liu/China/IBM
2014/08/07 15:
