@syepes I just run Spark 1.6 (881f254) on YARN with Hadoop 2.4.0. I was able to run a simple application in cluster mode successfully.
Can you verify whether the org.apache.spark.yarn.ApplicationMaster class exists in your assembly jar? jar -tf assembly.jar | grep ApplicationMaster -Andrew 2015-12-17 7:44 GMT-08:00 syepes <sye...@gmail.com>: > -1 (YARN Cluster deployment mode not working) > > I have just tested 1.6 (d509194b) on our HDP 2.3 platform and the cluster > mode does not seem work. It looks like some parameter are not being passed > correctly. > This example works correctly with 1.5. > > # spark-submit --master yarn --deploy-mode cluster --num-executors 1 > --properties-file $PWD/spark-props.conf --class > org.apache.spark.examples.SparkPi > /opt/spark/lib/spark-examples-1.6.0-SNAPSHOT-hadoop2.7.1.jar > > Error: Could not find or load main class > org.apache.spark.deploy.yarn.ApplicationMaster > > spark-props.conf > ----------------------------- > spark.driver.extraJavaOptions -Dhdp.version=2.3.2.0-2950 > spark.driver.extraLibraryPath > > /usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64 > spark.executor.extraJavaOptions -Dhdp.version=2.3.2.0-2950 > spark.executor.extraLibraryPath > > /usr/hdp/current/hadoop-client/lib/native:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64 > ----------------------------- > > I will try to do some more debugging on this issue. > > > > > > -- > View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-6-0-RC3-tp15660p15692.html > Sent from the Apache Spark Developers List mailing list archive at > Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > >
