It is working properly now, by command "spark-submit --master yarn-cluster "
.It seems that it will not run by way of "spark-submit --master yarn-client "
on yarn.
Thanks a lot for all your help .
On Saturday, September 26, 2015 2:27 PM, Gavin Yue
wrote:
It is working, We are
Hi All,
Would some expert help me some about the issue...
I shall appreciate you kind help very much!
Thank you!
Zhiliang
On Sunday, September 27, 2015 7:40 PM, Zhiliang Zhu
wrote:
Hi Alexis, Gavin,
Thanks very much for your kind comment.My spark command is :
spark-submit
Hi Alexis, Gavin,
Thanks very much for your kind comment.My spark command is :
spark-submit --class com.zyyx.spark.example.LinearRegression --master
yarn-client LinearRegression.jar
Both spark-shell and spark-submit will not run, all is hanging during the stage,
15/09/27 19:18:06 INFO yarn.Clie
It is working, We are doing the same thing everyday. But the remote server
needs to able to talk with ResourceManager.
If you are using Spark-submit, your will also specify the hadoop conf
directory in your Env variable. Spark would rely on that to locate where
the cluster's resource manager is.
I think gavin want you to print the env variables from the remote machine.
Can you provide the spark-submit command line.
Are you able to run the repl from the remote machine ?
./bin/spark-shell --master yarn-client
2015-09-26 10:11 GMT+08:00 Zhiliang Zhu :
> Hi Yue,
>
> Thanks very much for yo
Hi Yue,
Thanks very much for your kind reply.
I would like to submit spark job remotely on another machine outside the
cluster,and the job will run on yarn, similar as hadoop job is already done,
could youconfirm it could exactly work for spark...
Do you mean that I would print those variables on
Print out your env variables and check first
Sent from my iPhone
> On Sep 25, 2015, at 18:43, Zhiliang Zhu wrote:
>
> Hi All,
>
> I would like to submit spark job on some another remote machine outside the
> cluster,
> I also copied hadoop/spark conf files under the remote machine, then hado
