Re: Caching issue with msg: RDD block could not be dropped from memory as it does not exist

2014-07-22 Thread rindra
Hello Andrew, Thank you very much for your great tips. Your solution worked perfectly. In fact, I was not aware that the right option for local mode is --driver.memory 1g Cheers, Rindra On Mon, Jul 21, 2014 at 11:23 AM, Andrew Or-2 [via Apache Spark User List] < ml-node+s1001560n10336...

Caching issue with msg: RDD block could not be dropped from memory as it does not exist

2014-07-19 Thread rindra
utting block rdd_63_40 failed res29: Long = 5 It seems that I could not cache the data in memory even though my local machine has 16Gb RAM and the data is only 13MB with 100 partitions size. How to prevent this caching issue from happening? Thanks. Rindra -- View this message in context: http://