HI,
I'm a junior user of spark from China.
I have a problem about submit spark job right now. I want to submit job
from code.
In other words ,"How to submit spark job from within java program to yarn
cluster without using spark-submit"
I've learnt from official site
http://spark.apache.org/docs/latest/submitting-applications.html
that using bin/spark-submit script to submit a job to cluster is easy .
Because the script may does lots of complex work such as setting up
the classpath with Spark and its dependencies.
If I don't use the script ,I have to deal with all complex work by
myself.It makes me feel really frustrated.
I have search this problem from Google,but the answers may not suit
for me .
In hadoop developing ,I know that after setting up Configuration
,Job and resources ,
we can submit hadoop job by coding like this:
job.waitForCompletion
It is convenient for users to submit job programmatically
I want to know if there is a schedule( may be in spark 1.5+?)that provide
users variety ways of submitting job like hadoop .
Like monitoring ,In the recent release spark(1.4.0) We can get statements
about spark applications by REST API right now.
Thanks & Regards
GUO QIAN
