Re: How to avoid being killed by YARN node manager ?

Thank you for your reply. I'm sorry confirmation is slow. I'll try the tuning 'spark.yarn.executor.memoryOverhead'. Thanks, Yuichiro Sakamoto On 2015/03/25 0:56, Sandy Ryza wrote: Hi Yuichiro, The way to avoid this is to boost spark.yarn.executor.memoryOverhead until the executors have enou

Re: Can't cache RDD of collaborative filtering on MLlib

Hello. I tried `count()`, then `userJavaRDD` and `productJavaRDD` were cached, and the speed became faster. Thank you. On 2015/03/10 4:05, Xiangrui Meng wrote: cache() is lazy. The data is stored into memory after the first time it gets materialized. So the first time you call `predict` after