How to join two PairRDD together?

2014-08-24 Thread Gefei Li
es() method and map the keys out, but it gives me an "out of memory" error. I think the "out of memory" error is caused by the few entries of my RDD, but I have no idea how to solve it. Can you help me? Regards, Gefei Li

Re: how to use the method saveAsTextFile of a RDD like javaRDD

2014-08-14 Thread Gefei Li
It is interesting to save a RDD on a disk or HDFS or somethings else as a set of objects, but I think it's more useful to save it as a text file for debugging or just as an output file. If we want to reuse a RDD, text file also works, but perhaps a set of object files will bring a decrease on execu

Re: how to use the method saveAsTextFile of a RDD like javaRDD

2014-08-14 Thread Gefei Li
Thank you! It works so well for me! Regards, Gefei On Thu, Aug 14, 2014 at 4:25 PM, Tathagata Das wrote: > FlatMap the JavaRDD to JavaRDD. Then it should > work. > > TD > > > On Thu, Aug 14, 2014 at 1:23 AM, Gefei Li wrote: > >> Hello, >> I wrote a c

how to use the method saveAsTextFile of a RDD like javaRDD

2014-08-14 Thread Gefei Li
Hello, I wrote a class named BooleanPair: public static class BooleanPairet implements Serializable{ public Boolean elementBool1; public Boolean elementBool2; BooleanPair(Boolean bool1, Boolean bool2){elementBool1 = bool1; elementBool2 = bool2;} public String to