Hi,
I am using spark 1.0.0. In my spark code i m trying to persist an rdd to
disk as rrd.persist(DISK_ONLY). But unfortunately couldn't find the
location where the rdd has been written to disk. I specified
SPARK_LOCAL_DIRS and SPARK_WORKER_DIR to some other location rather than
using the default /tmp directory, but still couldnt see anything in worker
directory andspark ocal directory.
I also tried specifying the local dir and worker dir from the spark code
while defining the SparkConf as conf.set("spark.local.dir",
"/home/padma/sparkdir") but the directories are not used.
In general which directories spark would be using for map output files,
intermediate writes and persisting rdd to disk ?
Thanks,
Padma Ch
