Strange. Does the application show up at all in the YARN web UI? Does application_1436314873375_0030 show up at all in the YARN ResourceManager logs?
-Sandy On Wed, Jul 8, 2015 at 3:32 PM, Juan Gordon <jgordo...@gmail.com> wrote: > Hello Sandy, > > Yes I'm sure that YARN has the enought resources, i checked it in the WEB > UI page of my cluster > > Also, i'm able to submit the same script in any of the nodes of the > cluster. > > That's why i don't understand whats happening. > > Thanks > > JG > > On Wed, Jul 8, 2015 at 5:26 PM, Sandy Ryza <sandy.r...@cloudera.com> > wrote: > >> Hi JG, >> >> One way this can occur is that YARN doesn't have enough resources to run >> your job. Have you verified that it does? Are you able to submit using >> the same command from a node on the cluster? >> >> -Sandy >> >> On Wed, Jul 8, 2015 at 3:19 PM, jegordon <jgordo...@gmail.com> wrote: >> >>> I'm trying to submit a spark job from a different server outside of my >>> Spark >>> Cluster (running spark 1.4.0, hadoop 2.4.0 and YARN) using the >>> spark-submit >>> script : >>> >>> spark/bin/spark-submit --master yarn-client --executor-memory 4G >>> myjobScript.py >>> >>> The think is that my application never pass from the accepted state, it >>> stuck on it : >>> >>> 15/07/08 16:49:40 INFO Client: Application report for >>> application_1436314873375_0030 (state: ACCEPTED) >>> 15/07/08 16:49:41 INFO Client: Application report for >>> application_1436314873375_0030 (state: ACCEPTED) >>> 15/07/08 16:49:42 INFO Client: Application report for >>> application_1436314873375_0030 (state: ACCEPTED) >>> 15/07/08 16:49:43 INFO Client: Application report for >>> application_1436314873375_0030 (state: ACCEPTED) >>> 15/07/08 16:49:44 INFO Client: Application report for >>> application_1436314873375_0030 (state: ACCEPTED) >>> 15/07/08 16:49:45 INFO Client: Application report for >>> application_1436314873375_0030 (state: ACCEPTED) >>> 15/07/08 16:49:46 INFO Client: Application report for >>> application_1436314873375_0030 (state: ACCEPTED) >>> 15/07/08 16:49:47 INFO Client: Application report for >>> application_1436314873375_0030 (state: ACCEPTED) >>> 15/07/08 16:49:48 INFO Client: Application report for >>> application_1436314873375_0030 (state: ACCEPTED) >>> 15/07/08 16:49:49 INFO Client: Application report for >>> application_1436314873375_0030 (state: ACCEPTED) >>> >>> But if i execute the same script with spark-submit in the master server >>> of >>> my cluster it runs correctly. >>> >>> I already set the yarn configuration in the remote server in >>> $YARN_CONF_DIR/yarn-site.xml like this : >>> >>> <property> >>> <name>yarn.resourcemanager.hostname</name> >>> <value>54.54.54.54</value> >>> </property> >>> >>> <property> >>> <name>yarn.resourcemanager.address</name> >>> <value>54.54.54.54:8032</value> >>> <description>Enter your ResourceManager hostname.</description> >>> </property> >>> >>> <property> >>> <name>yarn.resourcemanager.scheduler.address</name> >>> <value>54.54.54.54:8030</value> >>> <description>Enter your ResourceManager hostname.</description> >>> </property> >>> >>> <property> >>> <name>yarn.resourcemanager.resourcetracker.address</name> >>> <value>54.54.54.54:8031</value> >>> <description>Enter your ResourceManager hostname.</description> >>> </property> >>> Where 54.54.54.54 is the IP of my resourcemanager node. >>> >>> Why is this happening? do i have to configure something else in YARN to >>> accept remote submits? or what am i missing? >>> >>> Thanks a lot >>> >>> JG >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/Remote-spark-submit-not-working-with-YARN-tp23728.html >>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>> For additional commands, e-mail: user-h...@spark.apache.org >>> >>> >> > > > -- > Saludos, > Juan Gordon >
