Hi, I'm using vagrant. I've built a cluster with MESOS and SPARK with the following structure:
- 2 zookeeper nodes - 2 master nodes - 3 slave nodes In each master node, I've installed MESOS and Spark. My $SPARK_HOME/conf/spark_env.sh contains: export MESOS_NATIVE_LIBRARY=/usr/local/lib/libmesos.so export SPARK_EXECUTOR_URI=http://dominio.com/spark/spark-0.9.0-bin-cdh4.tgz export MASTER=zk://192.168.30.101:2181,192.168.30.102:2181 The issue I'm having is that executors, launched by the slaves nodes, always download the spark distribution from http://dominio.com/spark/spark-0.9.0-bin-cdh4.tgz. Disk usage of slave nodes is continuously increasing, in addition to the introduced latency which is due to downloading and decompressing the tgz file. Is there any way to avoid it? Thanks in advance. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Problem-with-Spark-on-Mesos-tp2283.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
