Depends on the size of the arrays, but is it what you want to achieve similar to a join?
> On 15 Aug 2016, at 20:12, Eric Ho <e...@analyticsmd.com> wrote: > > Hi, > > I've two nested-for loops like this: > > for all elements in Array A do: > > for all elements in Array B do: > > compare a[3] with b[4] see if they 'match' and if match, return that element; > > If I were to represent Arrays A and B as 2 separate RDDs, how would my code > look like ? > > I couldn't find any RDD functions that would do this for me efficiently. I > don't really want elements of RDD(A) and RDD(B) flying all over the network > piecemeal... > > THanks. > > -- > > -eric ho >
