Thanks for the replies. We decided to use concurrency in Scala to do the two mappings using the same source RDD in parallel. So far, it seems to be working. Any comments?
On Wednesday, May 6, 2015, Evo Eftimov <[email protected]> wrote: > RDD1 = RDD.filter() > > RDD2 = RDD.filter() > > > > *From:* Bill Q [mailto:[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>] > *Sent:* Tuesday, May 5, 2015 10:42 PM > *To:* [email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');> > *Subject:* Map one RDD into two RDD > > > > Hi all, > > I have a large RDD that I map a function to it. Based on the nature of > each record in the input RDD, I will generate two types of data. I would > like to save each type into its own RDD. But I can't seem to find an > efficient way to do it. Any suggestions? > > > > Many thanks. > > > > > > Bill > > > > -- > > Many thanks. > > Bill > > > -- Many thanks. Bill
