Re: Dynamic allocation doesn't work on YARN

2016-03-10 Thread Jy Chen
edExecutorIdleTimeout=60s, "--conf" was lost >> when I copied it to mail. >> >> -- Forwarded message -- >> From: Jy Chen >> Date: 2016-03-10 10:09 GMT+08:00 >> Subject: Re: Dynamic allocation doesn't work on YARN >>

Re: Dynamic allocation doesn't work on YARN

2016-03-09 Thread Saisai Shao
quot;--conf" was lost > when I copied it to mail. > > -- Forwarded message -- > From: Jy Chen > Date: 2016-03-10 10:09 GMT+08:00 > Subject: Re: Dynamic allocation doesn't work on YARN > To: Saisai Shao , user@spark.apache.org > > > Hi, > My

Re: Dynamic allocation doesn't work on YARN

2016-03-09 Thread Jy Chen
Hi, My Spark version is 1.5.1 with Hadoop 2.5.0-cdh5.2.0. These are my configurations of dynamic allocation: --master yarn-client --conf spark.dynamicAllocation.enabled=true --conf spark.shuffle.service.enabled=true --conf spark.dynamicAllocation.minExecutors=0 --conf spark.dynamicAllocation.initia

Re: Dynamic allocation doesn't work on YARN

2016-03-09 Thread Saisai Shao
Would you please send out the configurations of dynamic allocation so we could know better. On Wed, Mar 9, 2016 at 4:29 PM, Jy Chen wrote: > Hello everyone: > > I'm trying the dynamic allocation in Spark on YARN. I have followed > configuration steps and started the shuffle service. > > Now it c