Hi all, Can I group elements in RDD into different groups and let each group share elements? For example, I have 10,000 elements in RDD from e1 to e10000, and I want to group and aggregate them by another mapping with size of 2000, ex: ( (e1,e42), (e1,e554), (e3, e554)…… (2000th group))
My first approach was to filter the RDD with mapping rules for 2000 times, and then union them together. However, it ran forever. Does SPARK provide a way to group elements in RDD like this please? Thanks, Franz
