Re: Joining by values

2015-01-03 Thread Dilip Movva
tln) > > > This outputs the following . I think this may be essentially what u r looking > for > > (I have to understand how to NOT print as CompactBuffer) > > (2,CompactBuffer(1001,1000,1002,1003, 1004,1001,1006,1007)) > (3,CompactBuffer(1011,1012,1013,1010, 1007,1009,1005,1008)) > (

Re: Joining by values

2015-01-03 Thread Sanjay Subramanian
spark.apache.org" Sent: Saturday, January 3, 2015 8:15 PM Subject: Re: Joining by values call `map(_.toList)` to convert `CompactBuffer` to `List` Best Regards,Shixiong Zhu 2015-01-04 12:08 GMT+08:00 Sanjay Subramanian : hi Take a look at the code here I wrotehttps://raw.

Re: Joining by values

2015-01-03 Thread Shixiong Zhu
int as CompactBuffer) > > (2,CompactBuffer(1001,1000,1002,1003, 1004,1001,1006,1007)) > (3,CompactBuffer(1011,1012,1013,1010, 1007,1009,1005,1008)) > (1,CompactBuffer(1001,1000,1002,1003, 1011,1012,1013,1010, > 1004,1001,1006,1007, 1007,1009,1005,1008)) > > >

Re: Joining by values

2015-01-03 Thread Sanjay Subramanian
6,1007)) (3,CompactBuffer(1011,1012,1013,1010, 1007,1009,1005,1008)) (1,CompactBuffer(1001,1000,1002,1003, 1011,1012,1013,1010, 1004,1001,1006,1007, 1007,1009,1005,1008)) From: Sanjay Subramanian To: dcmovva ; "user@spark.apache.org" Sent: Saturday, January 3, 2015 12:19 PM Subject: Re:

Re: Joining by values

2015-01-03 Thread Sanjay Subramanian
This is my design. Now let me try and code it in Spark. rdd1.txt =1~4,5,6,72~4,53~6,7 rdd2.txt  4~1001,1000,1002,10035~1004,1001,1006,10076~1007,1009,1005,10087~1011,1012,1013,1010 TRANSFORM 1===map each value to key (like an inverted index)4~15~16~17~15~24~26~37~3 TRANSFOR