ALS still needs to load and deserialize the in/out blocks (one by one) from disk and then construct least squares subproblems. All happen in RAM. The final model is also stored in memory. -Xiangrui
On Wed, Oct 1, 2014 at 4:36 AM, Alex T <[email protected]> wrote: > Hi, thanks for the reply. > > I added the ALS.setIntermediateRDDStorageLevel and it worked well (a little > slow, but still did the job and i've made MF and get all the features). > > But even if I persist with DISK_ONLY, the system monitor shows on the memory > and swap history that Apache Spark is using RAM. How can I free this memory > after all tasks without destroying sparkContext? > > Thanks again for your time. > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/MLLib-ALS-question-tp15420p15473.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
