Hello friends:
It was mentioned in another (Y.A.R.N.-centric) email thread that
'SPARK_JAR' was deprecated,
and to use the 'spark.yarn.jar' property instead for YARN submission.
For example:
user$ pyspark [some-options] --driver-java-options
spark.yarn.jar=hdfs://namenode:8020/path/to/spark-assembly-*.jar
What is the equivalent property to use for the LOCAL MODE case?
spark.jar? spark.local.jar?
I searched for this, but can't find where the definitions for these
exist (perhaps a pointer
to that, too). :)
For completeness/explicitness, I like to specify things like this on the
CLI, even if there
are default settings them.
Thank you!
didata
//
