Re: Need help for RDD/DF transformation.

2017-03-30 Thread Yong Zhang
logic order, so you can just pick the first element out from the Array "keys" of DF2, to join. Otherwise, I don't see any way to avoid a cartesian join. Yong From: Mungeol Heo Sent: Thursday, March 30, 2017 3:05 AM To: ayan guha Cc: Yong Zhang; user@spa

Re: Need help for RDD/DF transformation.

2017-03-30 Thread Mungeol Heo
he desired result for >> > >> > >> > RDD/DF 1 >> > >> > 1, a >> > 3, c >> > 5, b >> > >> > RDD/DF 2 >> > >> > [1, 2, 3] >> > [4, 5] >> > >> > >> > Yong >> > >&g

Re: Need help for RDD/DF transformation.

2017-03-29 Thread ayan guha
> > > > RDD/DF 1 > > > > 1, a > > 3, c > > 5, b > > > > RDD/DF 2 > > > > [1, 2, 3] > > [4, 5] > > > > > > Yong > > > > > > From: Mungeol Heo > > Sent: Wednesday, March 29, 2017 5:37 AM > &

Re: Need help for RDD/DF transformation.

2017-03-29 Thread Mungeol Heo
the desired result for > > > RDD/DF 1 > > 1, a > 3, c > 5, b > > RDD/DF 2 > > [1, 2, 3] > [4, 5] > > > Yong > > > From: Mungeol Heo > Sent: Wednesday, March 29, 2017 5:37 AM > To: user@spark.apache.org &

Re: Need help for RDD/DF transformation.

2017-03-29 Thread Yong Zhang
What is the desired result for RDD/DF 1 1, a 3, c 5, b RDD/DF 2 [1, 2, 3] [4, 5] Yong From: Mungeol Heo Sent: Wednesday, March 29, 2017 5:37 AM To: user@spark.apache.org Subject: Need help for RDD/DF transformation. Hello, Suppose, I have two RDD or data

Need help for RDD/DF transformation.

2017-03-29 Thread Mungeol Heo
Hello, Suppose, I have two RDD or data frame like addressed below. RDD/DF 1 1, a 3, a 5, b RDD/DF 2 [1, 2, 3] [4, 5] I need to create a new RDD/DF like below from RDD/DF 1 and 2. 1, a 2, a 3, a 4, b 5, b Is there an efficient way to do this? Any help will be great. Thank you.