Re: Setting JVM options to Spark executors in Standalone mode

2015-01-16 Thread Marcelo Vanzin
On Fri, Jan 16, 2015 at 10:07 AM, Michel Dufresne wrote: > Thank for your reply, I've should have mentioned that spark-env.sh is the > only option i found because: > >- I'm creating the SpeakConf/SparkContext from a Play Application >(therefore I'm not using spark-submit script) Then you

Re: Setting JVM options to Spark executors in Standalone mode

2015-01-16 Thread Michel Dufresne
Thank for your reply, I've should have mentioned that spark-env.sh is the only option i found because: - I'm passing the public IP address of the slave (which is determined in the shell script) - I'm creating the SpeakConf/SparkContext from a Play Application (therefore I'm not using s

Re: Setting JVM options to Spark executors in Standalone mode

2015-01-16 Thread Zhan Zhang
You can try to add it in in conf/spark-defaults.conf # spark.executor.extraJavaOptions -XX:+PrintGCDetails -Dkey=value -Dnumbers="one two three” Thanks. Zhan Zhang On Jan 16, 2015, at 9:56 AM, Michel Dufresne wrote: > Hi All, > > I'm trying to set some JVM options to the executor process

Setting JVM options to Spark executors in Standalone mode

2015-01-16 Thread Michel Dufresne
Hi All, I'm trying to set some JVM options to the executor processes in a standalone cluster. Here's what I have in *spark-env.sh*: jmx_opt="-Dcom.sun.management.jmxremote" > jmx_opt="${jmx_opt} -Djava.net.preferIPv4Stack=true" > jmx_opt="${jmx_opt} -Dcom.sun.management.jmxremote.port=" > jmx