Thanks Jerry. It looks like a good option, will try it. Regards, Ajay
On Friday, July 4, 2014 2:18 PM, "Shao, Saisai" <saisai.s...@intel.com> wrote: Hi Ajay, StorageLevel OFF_HEAP means for can cache your RDD into Tachyon, the prerequisite is that you should deploy Tachyon among Spark. Yes, it can alleviate GC, since you offload JVM memory into system managed memory. You can use rdd.persist(…) to use this level, details can be checked in BlockManager.scala, TachyonBlockManager and TachyonStore. StorageLevel NONE means the rdd will not be cached, and if you want to use this rdd again, you should re-compute from the source to get the data. Thanks Jerry From:Ajay Srivastava [mailto:a_k_srivast...@yahoo.com] Sent: Friday, July 04, 2014 2:19 PM To: user@spark.apache.org Subject: OFF_HEAP storage level Hi, I was checking different storage level of an RDD and found OFF_HEAP. Has anybody used this level ? If i use this level, where will data be stored ? If not in heap, does it mean that we can avoid GC ? How can I use this level ? I did not find anything in archive regarding this. Can someone also explain the behavior of storage level - NONE ? Regards, Ajay
