Re: Spark Job running on localhost on yarn cluster

2015-02-05 Thread Kostas Sakellis
> On Thu, Feb 5, 2015 at 12:49 PM, Felix C > wrote: > >> Is YARN_CONF_DIR set? >> >> --- Original Message --- >> >> From: "Aniket Bhatnagar" >> Sent: February 4, 2015 6:16 AM >> To: "kundan kumar" , "spark users" <

Re: Spark Job running on localhost on yarn cluster

2015-02-05 Thread kundan kumar
red to the standalone mode. Regards, Kundan On Thu, Feb 5, 2015 at 12:49 PM, Felix C wrote: > Is YARN_CONF_DIR set? > > --- Original Message --- > > From: "Aniket Bhatnagar" > Sent: February 4, 2015 6:16 AM > To: "kundan kumar" , "spark users" &

Re: Spark Job running on localhost on yarn cluster

2015-02-04 Thread Felix C
Is YARN_CONF_DIR set? --- Original Message --- From: "Aniket Bhatnagar" Sent: February 4, 2015 6:16 AM To: "kundan kumar" , "spark users" Subject: Re: Spark Job running on localhost on yarn cluster Have you set master in SparkConf/SparkContext in your code? Driv

Re: Spark Job running on localhost on yarn cluster

2015-02-04 Thread Michael Albert
of priority if the parameters disagree with the code). Good luck! -Mike From: kundan kumar To: spark users Sent: Wednesday, February 4, 2015 7:41 AM Subject: Spark Job running on localhost on yarn cluster Hi,  I am trying to execute my code on a yarn cluster The command which

Re: Spark Job running on localhost on yarn cluster

2015-02-04 Thread Aniket Bhatnagar
Have you set master in SparkConf/SparkContext in your code? Driver logs show in which mode the spark job is running. Double check if the logs mention local or yarn-cluster. Also, what's the error that you are getting? On Wed, Feb 4, 2015, 6:13 PM kundan kumar wrote: > Hi, > > I am trying to exec

Spark Job running on localhost on yarn cluster

2015-02-04 Thread kundan kumar
Hi, I am trying to execute my code on a yarn cluster The command which I am using is $SPARK_HOME/bin/spark-submit --class "EDDApp" target/scala-2.10/edd-application_2.10-1.0.jar --master yarn-cluster --num-executors 3 --driver-memory 6g --executor-memory 7g But, I can see that this program is