Re: merge 3 different types of RDDs in one

On Tue, Dec 1, 2015 at 10:57 AM, Shams ul Haque wrote: > Thanks for the suggestion, i am going to try union. ...and please report your findings back. > And what is your opinion on 2nd question. Dunno. If you find a solution, let us know. Jacek

Re: merge 3 different types of RDDs in one

Hi, I have myself used union in a similar case. And applied reduceByKey on it. Union + reduceByKey will suffice join... but you will have to first use Map so that all values are of same datatype Regards, Sushrut Ikhar [image: https://]about.me/sushrutikhar

Re: merge 3 different types of RDDs in one

I think you should be able to join different rdds with same key. Have you tried that? On Dec 1, 2015 3:30 PM, "Praveen Chundi" wrote: > cogroup could be useful to you, since all three are PairRDD's. > > > https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.rdd.PairRDDFunct

Re: merge 3 different types of RDDs in one

cogroup could be useful to you, since all three are PairRDD's. https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.rdd.PairRDDFunctions Best Regards, Praveen On 01.12.2015 10:47, Shams ul Haque wrote: Hi All, I have made 3 RDDs of 3 different dataset, all RDDs are grou

Re: merge 3 different types of RDDs in one

Hi Jacek, Thanks for the suggestion, i am going to try union. And what is your opinion on 2nd question. Thanks Shams On Tue, Dec 1, 2015 at 3:23 PM, Jacek Laskowski wrote: > Hi, > > Never done it before, but just yesterday I found out about > SparkContext.union method that could help in your

Re: merge 3 different types of RDDs in one

Hi, Never done it before, but just yesterday I found out about SparkContext.union method that could help in your case. def union[T](rdds: Seq[RDD[T]])(implicit arg0: ClassTag[T]): RDD[T] http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkContext Pozdrawiam, Jacek --