On Mon, Sep 8, 2014 at 3:54 PM, Dimension Data, LLC. < subscripti...@didata.us> wrote:
> You're probably right about the above because, as seen *below* for > pyspark (but probably for other Spark > applications too), once '-Dspark.master=[yarn-client|yarn-cluster]' is > specified, the app invocation doesn't even seem to > respect the property '-Dspark.yarn.jar=[file:/// | file:// |/...]' > setting. > > That's too bad because I have CDH5 Spark installed > If you're using CDH 5 (either 5.0 or 5.1) then spark.yarn.jar won't work. For CDH5 you want to set the SPARK_JAR environment variable instead. The change that added spark.yarn.jar is part of Spark 1.1 (which will be part of CDH 5.2). -- Marcelo
