Without a scheme, it will be interpreted relative to the default FS
configured in the Hadoop configuration, which is almost surely HDFS.
No, the stack trace does not imply it was writing to HDFS. It would
use the HDFS FileSystem API in any event, but, you can see that the
path was a file: URI.
The goal is to write a local file. I don't see that multiple files are
a problem either.
The original problem is simply that mkdirs failed and that's almost
surely a permission issue. I don't see that this has been addressed by
the OP.
On Mon, Jan 12, 2015 at 9:18 AM, Prannoy <pran...@sigmoidanalytics.com> wrote:
> Have you tried simple giving the path where you want to save the file ?
>
> For instance in your case just do
>
> r.saveAsTextFile("home/cloudera/tmp/out1")
>
> Dont use file
>
> This will create a folder with name out1. saveAsTextFile always write by
> making a directory, it does not write data into a single file.
>
> Incase you need a single file you can use copyMerge API in FileUtils.
>
> FileUtil.copyMerge(fs, home/cloudera/tmp/out1, fs,home/cloudera/tmp/out2 ,
> true, conf,null);
>
> Now out2 will be a single file containing your data.
>
> fs is the configuration of you local file system.
>
> Thanks
>
>
>
> On Sat, Jan 10, 2015 at 1:36 AM, NingjunWang [via Apache Spark User List]
> <[hidden email]> wrote:
>>
>> No, do you have any idea?
>>
>>
>>
>> Regards,
>>
>>
>>
>> Ningjun Wang
>>
>> Consulting Software Engineer
>>
>> LexisNexis
>>
>> 121 Chanlon Road
>>
>> New Providence, NJ 07974-1541
>>
>>
>>
>> From: firemonk9 [via Apache Spark User List] [mailto:[hidden email][hidden
>> email]]
>> Sent: Friday, January 09, 2015 2:56 PM
>> To: Wang, Ningjun (LNG-NPV)
>> Subject: Re: Failed to save RDD as text file to local file system
>>
>>
>>
>> Have you found any resolution for this issue ?
>>
>> ________________________________
>>
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>>
>> http://apache-spark-user-list.1001560.n3.nabble.com/Failed-to-save-RDD-as-text-file-to-local-file-system-tp21050p21067.html
>>
>> To unsubscribe from Failed to save RDD as text file to local file system,
>> click here.
>> NAML
>>
>>
>>
>> ________________________________
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://apache-spark-user-list.1001560.n3.nabble.com/Failed-to-save-RDD-as-text-file-to-local-file-system-tp21050p21068.html
>> To start a new topic under Apache Spark User List, email [hidden email]
>> To unsubscribe from Apache Spark User List, click here.
>> NAML
>
>
>
> ________________________________
> View this message in context: Re: Failed to save RDD as text file to local
> file system
>
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
