Hi I’m trying to using a specific dir for spark working directory since I have limited space at /tmp. I tried: 1) export SPARK_LOCAL_DIRS=“/mnt/data/tmp” or 2) SPARK_LOCAL_DIRS=“/mnt/data/tmp” in spark-env.sh
But neither worked, since the output of spark still saying ERROR DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file /tmp/spark-local-20140809134509-0502/34/shuffle_0_436_1 java.io.FileNotFoundException: /tmp/spark-local-20140809134509-0502/34/shuffle_0_436_1 (No space left on device) anybody help with correctly setting up the “tmp” directory? Best, Baoqiang Cao Blog: http://baoqiang.org Email: bqcaom...@gmail.com
