;
> -Original Message-
> From: Patrick Wendell [mailto:pwend...@gmail.com]
> Sent: Thursday, December 25, 2014 3:22 PM
> To: Shao, Saisai
> Cc: user@spark.apache.org; d...@spark.apache.org
> Subject: Re: Question on saveAsTextFile with overwrite option
>
> Is it suffic
Message-
> From: Patrick Wendell [mailto:pwend...@gmail.com]
> Sent: Thursday, December 25, 2014 3:22 PM
> To: Shao, Saisai
> Cc: user@spark.apache.org; d...@spark.apache.org
> Subject: Re: Question on saveAsTextFile with overwrite option
>
> Is it sufficient to set &quo
; d...@spark.apache.org
Subject: Re: Question on saveAsTextFile with overwrite option
Is it sufficient to set "spark.hadoop.validateOutputSpecs" to false?
http://spark.apache.org/docs/latest/configuration.html
- Patrick
On Wed, Dec 24, 2014 at 10:52 PM, Shao, Saisai wrote:
> Hi,
&g
Is it sufficient to set "spark.hadoop.validateOutputSpecs" to false?
http://spark.apache.org/docs/latest/configuration.html
- Patrick
On Wed, Dec 24, 2014 at 10:52 PM, Shao, Saisai wrote:
> Hi,
>
>
>
> We have such requirements to save RDD output to HDFS with saveAsTextFile
> like API, but need
Hi,
We have such requirements to save RDD output to HDFS with saveAsTextFile like
API, but need to overwrite the data if existed. I'm not sure if current Spark
support such kind of operations, or I need to check this manually?
There's a thread in mailing list discussed about this
(http://apach
