Re: Cannot submit to a Spark Application to a remote cluster Spark 1.0

Andrew, thank you so much! That worked! I had to manually set the spark.home configuration in the SparkConf object using .set("spark.home","/cluster/path/to/spark/"), and then I was able to submit from my laptop to the cluster! Aris On Thu, Jul 10, 2014 at 11:41 AM, Andrew Or wrote: > Setting

Re: Cannot submit to a Spark Application to a remote cluster Spark 1.0

Setting SPARK_HOME is not super effective, because it is overridden very quickly by bin/spark-submit here . Instead you should set the config "spark.home". Here's why: Each of your executors inherit

Re: Cannot submit to a Spark Application to a remote cluster Spark 1.0

Thank you very much Yana for replying! So right now the set up is a single-node machine which is my "cluster", and YES you are right my submitting laptop has a different path to the spark-1.0.0 installation than the "cluster" machine. I tried to set SPARK_HOME on my submittor laptop using the act

Re: Cannot submit to a Spark Application to a remote cluster Spark 1.0

class java.io.IOException: Cannot run program "/Users/aris.vlasakakis/Documents/spark-1.0.0/bin/compute-classpath.sh" (in directory "."): error=2, No such file or directory By any chance, are your SPARK_HOME directories different on the machine where you're submitting from and the cluster? I'm on