Re: spark-submit inside script... need some bash help

2014-11-09 Thread Akhil Das
Not sure why that is failing, but i found a workaround like: #!/bin/bash -e SPARK_SUBMIT=/home/akhld/mobi/localcluster/spark-1/bin/spark-submit *export _JAVA_OPTIONS=-Xmx1g* OPTS+=" --class org.apache.spark.examples.SparkPi" echo $SPARK_SUBMIT $OPTS lib/spark-examples-1.1.0-hadoop1.0.4.jar ex

spark-submit inside script... need some bash help

2014-11-07 Thread Koert Kuipers
i need to run spark-submit inside a script with options that are build up programmatically. oh and i need to use exec to keep the same pid (so it can run as a service and be killed). this is what i tried: == #!/bin/bash -e SPARK_SUBMIT=/usr/loca