shihj@master:/usr/local/spark/spark-1.6.1-bin-hadoop2.6$ ./bin/spark-submit
--class org.apache.spark.examples.SparkPi --master yarn-client --driver-memory
512m --num-executors 2 --executor-memory 512m --executor-cores 2
/user/shihj/spark_lib/spark-examples-1.6.1-hadoop2.6.0.jar 10
Warning: Local jar /user/shihj/spark_lib/spark-examples-1.6.1-hadoop2.6.0.jar
does not exist, skipping.
java.lang.ClassNotFoundException: org.apache.spark.examples.SparkPi
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:174)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:689)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
get error at once
------------------ ???????? ------------------
??????: "Yash Sharma";<[email protected]>;
????????: 2016??6??22??(??????) ????2:04
??????: "????????"<[email protected]>;
????: "user"<[email protected]>;
????: Re: Could not find or load main class
org.apache.spark.deploy.yarn.ExecutorLauncher
How about supplying the jar directly in spark submit -
./bin/spark-submit \
--class org.apache.spark.examples.SparkPi \
--master yarn-client \
--driver-memory 512m \
--num-executors 2 \
--executor-memory 512m \
--executor-cores 2 \
/user/shihj/spark_lib/spark-examples-1.6.1-hadoop2.6.0.jar
On Wed, Jun 22, 2016 at 3:59 PM, ???????? <[email protected]> wrote:
i config this para at spark-defaults.conf
spark.yarn.jar
hdfs://master:9000/user/shihj/spark_lib/spark-examples-1.6.1-hadoop2.6.0.jar
then ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master
yarn-client --driver-memory 512m --num-executors 2 --executor-memory 512m
--executor-cores 2 10:
Error: Could not find or load main class
org.apache.spark.deploy.yarn.ExecutorLauncher
but i don't config that para ,there no error why???that para is only avoid
Uploading resource file(jar package)??
