Re: special case of custom partitioning

Thanks Mayur - based on the doc-comments in source looks like this will work for the case. I will confirm. the dreamers of the day are dangerous men, for they may act their dream with open eyes, and make it possible On Fri, Mar 7, 2014 at 2:21 AM, Mayur Rustagi wrote: > How about Partition

Re: special case of custom partitioning

How about PartitionerAwareUnionRDD? Regards Mayur Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi On Thu, Mar 6, 2014 at 9:42 AM, Evan Chan wrote: > I would love to hear the answer to this as well. > > On Thu, Mar 6, 2014

Re: special case of custom partitioning

I would love to hear the answer to this as well. On Thu, Mar 6, 2014 at 4:09 AM, Manoj Awasthi wrote: > Hi All, > > > I have a three machine cluster. I have two RDDs each consisting of (K,V) > pairs. RDDs have just three keys 'a', 'b' and 'c'. > > // list1 - List(('a',1), ('b',2), >

special case of custom partitioning

Hi All, I have a three machine cluster. I have two RDDs each consisting of (K,V) pairs. RDDs have just three keys 'a', 'b' and 'c'. // list1 - List(('a',1), ('b',2), val rdd1 = sc.parallelize(list1).groupByKey(new HashPartitioner(3)) // list2 - List(('a',2), ('b',7), v