Re: Write 1 RDD to multiple output paths in one go

2014-09-15 Thread Nicholas Chammas
> /path/prefix/b [/part-1, /part-2, etc] > >> /path/prefix/f [/part-1, /part-2, etc] > >> > >> How would you do that? > >> > >> I suspect I need to use saveAsNewAPIHadoopFile or saveAsHadoopFile along > >> with the MultipleTextOutputFormat output format class, but I’m not sure > how. > >> > >> By the way, there is a very similar question to this here on Stack > >> Overflow. > >> > >> Nick > >> > >> > >> > >> View this message in context: Write 1 RDD to multiple output paths in > one > >> go > >> Sent from the Apache Spark User List mailing list archive at Nabble.com. > > > > >

Re: Write 1 RDD to multiple output paths in one go

2014-09-15 Thread Davies Liu
t;> /path/prefix/f [/part-1, /part-2, etc] >> >> How would you do that? >> >> I suspect I need to use saveAsNewAPIHadoopFile or saveAsHadoopFile along >> with the MultipleTextOutputFormat output format class, but I’m not sure how. >> >> By the way, there is a very simi

Re: Write 1 RDD to multiple output paths in one go

2014-09-15 Thread Sean Owen
would be something like: > > /path/prefix/n [/part-1, /part-2, etc] > /path/prefix/b [/part-1, /part-2, etc] > /path/prefix/f [/part-1, /part-2, etc] > > How would you do that? > > I suspect I need to use saveAsNewAPIHadoopFile or saveAsHadoopFile along > with the MultipleTextOutpu

Re: Write 1 RDD to multiple output paths in one go

2014-09-15 Thread Nicholas Chammas
ith the MultipleTextOutputFormat output format class, but I’m not > sure how. > > By the way, there is a very similar question to this here on Stack > Overflow > <http://stackoverflow.com/questions/23995040/write-to-multiple-outputs-by-key-spark-one-spark-job> > . > > Nick > ​ > >

Write 1 RDD to multiple output paths in one go

2014-09-13 Thread Nick Chammas
with the MultipleTextOutputFormat output format class, but I’m not sure how. By the way, there is a very similar question to this here on Stack Overflow <http://stackoverflow.com/questions/23995040/write-to-multiple-outputs-by-key-spark-one-spark-job> . Nick ​ -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Write-1-RDD-to-multiple-output-paths-in-one-go-tp14174.html Sent from the Apache Spark User List mailing list archive at Nabble.com.