Re: Map-Side Join in Spark

2015-05-04 Thread ๏̯͡๏
vide your partitioner >>> On 21 Apr 2015 15:04, "ÐΞ€ρ@Ҝ (๏̯͡๏)" wrote: >>> >>>> What is re-partition ? >>>> >>>> On Tue, Apr 21, 2015 at 10:23 AM, ayan guha >>>> wrote: >>>> >>>>> In my under

Re: Map-Side Join in Spark

2015-04-21 Thread ayan guha
rtition ? >>> >>> On Tue, Apr 21, 2015 at 10:23 AM, ayan guha wrote: >>> >>>> In my understanding you need to create a key out of the data and >>>> repartition both datasets to achieve map side join. >>>> On 21 Apr 2015 14:10, "ÐΞ€ρ

Re: Map-Side Join in Spark

2015-04-20 Thread ayan guha
te a key out of the data and >> repartition both datasets to achieve map side join. >> On 21 Apr 2015 14:10, "ÐΞ€ρ@Ҝ (๏̯͡๏)" wrote: >> >>> Can someone share their working code of Map Side join in Spark + Scala. >>> (No Spark-SQL) >>> >>> Th

Re: Map-Side Join in Spark

2015-04-20 Thread ๏̯͡๏
>>> repartition both datasets to achieve map side join. >>> On 21 Apr 2015 14:10, "ÐΞ€ρ@Ҝ (๏̯͡๏)" wrote: >>> >>>> Can someone share their working code of Map Side join in Spark + Scala. >>>> (No Spark-SQL) >>>> >>>>

Re: Map-Side Join in Spark

2015-04-20 Thread ๏̯͡๏
re their working code of Map Side join in Spark + Scala. >> (No Spark-SQL) >> >> The only resource i could find was this (Open in chrome with Chinese to >> english translator) >> >> http://dongxicheng.org/framework-on-yarn/apache-spark-join-two-tables/ >> >> >> >> -- >> Deepak >> >> -- Deepak

Re: Map-Side Join in Spark

2015-04-20 Thread ayan guha
In my understanding you need to create a key out of the data and repartition both datasets to achieve map side join. On 21 Apr 2015 14:10, "ÐΞ€ρ@Ҝ (๏̯͡๏)" wrote: > Can someone share their working code of Map Side join in Spark + Scala. > (No Spark-SQL) > > The only resource

Re: Map-Side Join in Spark

2015-04-20 Thread ๏̯͡๏
automatically). With map side join you need to do it. I am >> assuming you do it with mapPartitions & yield. >> >> A working code will help me understand it better. >> >> On Tue, Apr 21, 2015 at 9:40 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) >> wrote: >> >>> Ca

Re: Map-Side Join in Spark

2015-04-20 Thread Punyashloka Biswal
matically). With map side join you need to do it. I am assuming > you do it with mapPartitions & yield. > > A working code will help me understand it better. > > On Tue, Apr 21, 2015 at 9:40 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) > wrote: > >> Can someone share their working code of Map

Re: Map-Side Join in Spark

2015-04-20 Thread ๏̯͡๏
d it better. On Tue, Apr 21, 2015 at 9:40 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) wrote: > Can someone share their working code of Map Side join in Spark + Scala. > (No Spark-SQL) > > The only resource i could find was this (Open in chrome with Chinese to > english translator) > > http://dongxich

Map-Side Join in Spark

2015-04-20 Thread ๏̯͡๏
Can someone share their working code of Map Side join in Spark + Scala. (No Spark-SQL) The only resource i could find was this (Open in chrome with Chinese to english translator) http://dongxicheng.org/framework-on-yarn/apache-spark-join-two-tables/ -- Deepak