Re: incorrect message that path appears to be local

2016-10-12 Thread Sean Owen
I'm not sure this is applied consistently across Spark, but I'm dealing with another change now where an unqualified path is assumed to be a local file. The method Utils.resolvePath implements this logic and is used several places. Therefore I think this is probably intended behavior and you can wr

incorrect message that path appears to be local

2016-10-12 Thread Koert Kuipers
i see this warning when running jobs on cluster: 2016-10-12 14:46:47 WARN spark.SparkContext: Spark is not running in local mode, therefore the checkpoint directory must not be on the local filesystem. Directory '/tmp' appears to be on the local filesystem. however the checkpoint "directory" that