Hi Evan,
Could you please provide a code-snippet? Because it not clear for me, in
Hadoop you need to engage addNamedOutput method and I'm in stuck how to use
it from Spark
Thank you,
Konstantin Kudryavtsev
On Fri, Apr 4, 2014 at 5:27 PM, Evan Sparks <evan.spa...@gmail.com> wrote:
> Have a look at MultipleOutputs in the hadoop API. Spark can read and write
> to arbitrary hadoop formats.
>
> > On Apr 4, 2014, at 6:01 AM, dmpour23 <dmpou...@gmail.com> wrote:
> >
> > Hi all,
> > Say I have an input file which I would like to partition using
> > HashPartitioner k times.
> >
> > Calling rdd.saveAsTextFile(""hdfs://"); will save k files as part-00000
> > part-0000k
> > Is there a way to save each partition in specific folders?
> >
> > i.e. src
> > part0/part-00000
> > part1/part-00001
> > part1/part-0000k
> >
> > thanks
> > Dimitri
> >
> >
> >
> >
> >
> > --
> > View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/how-to-save-RDD-partitions-in-different-folders-tp3754.html
> > Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
