Re: saving rdd to multiple files named by the key

There is also SPARK-3533 , which proposes to add a convenience method for this. ​ On Mon Jan 26 2015 at 10:38:56 PM Aniket Bhatnagar < aniket.bhatna...@gmail.com> wrote: > This might be helpful: > http://stackoverflow.com/questions/23995040/write-

Re: saving rdd to multiple files named by the key

This might be helpful: http://stackoverflow.com/questions/23995040/write-to-multiple-outputs-by-key-spark-one-spark-job On Tue Jan 27 2015 at 07:45:18 Sharon Rapoport wrote: > Hi, > > I have an rdd of [k,v] pairs. I want to save each [v] to a file named [k]. > I got them by combining many [k,v]

saving rdd to multiple files named by the key

Hi, I have an rdd of [k,v] pairs. I want to save each [v] to a file named [k]. I got them by combining many [k,v] by [k]. I could then save to file by partitions, but that still doesn't allow me to choose the name, and leaves me stuck with foo/part-... Any tips? Thanks, Sharon