Re: Proper caching method

2014-04-16 Thread Arpit Tak
Mon, Apr 14, 2014 at 8:32 PM, Joe L wrote: >>> >>>> Hi I am trying to cache 2Gbyte data and to implement the following >>>> procedure. >>>> In order to cache them I did as follows: Is it necessary to cache rdd2 >>>> since >>>> rdd1 is already cached? >>>> >>>> rdd1 = textFile("hdfs...").cache() >>>> >>>> rdd2 = rdd1.filter(userDefinedFunc1).cache() >>>> rdd3 = rdd1.filter(userDefinedFunc2).cache() >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> View this message in context: >>>> http://apache-spark-user-list.1001560.n3.nabble.com/Proper-caching-method-tp4206.html >>>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>>> >>> >>> >> >

Re: Proper caching method

2014-04-16 Thread Cheng Lian
yte data and to implement the following >>> procedure. >>> In order to cache them I did as follows: Is it necessary to cache rdd2 >>> since >>> rdd1 is already cached? >>> >>> rdd1 = textFile("hdfs...").cache() >>> >>&g

Re: Proper caching method

2014-04-15 Thread Arpit Tak
s already cached? >> >> rdd1 = textFile("hdfs...").cache() >> >> rdd2 = rdd1.filter(userDefinedFunc1).cache() >> rdd3 = rdd1.filter(userDefinedFunc2).cache() >> >> >> >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Proper-caching-method-tp4206.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> > >

Re: Proper caching method

2014-04-14 Thread Cheng Lian
this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Proper-caching-method-tp4206.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Re: Proper caching method

2014-04-14 Thread Marcelo Vanzin
; > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Proper-caching-method-tp4206.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. -- Marcelo

Proper caching method

2014-04-14 Thread Joe L
finedFunc2).cache() -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Proper-caching-method-tp4206.html Sent from the Apache Spark User List mailing list archive at Nabble.com.