Re: ALS rating caching

2016-02-09 Thread Roberto Pagliari
Hi Nick, >From which version does that apply? I'm using 1.5.2 Thank you, From: Nick Pentreath mailto:nick.pentre...@gmail.com>> Date: Tuesday, 9 February 2016 07:02 To: "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Sub

Re: ALS rating caching

2016-02-08 Thread Nick Pentreath
In the "new" ALS intermediate RDDs (including the ratings input RDD after transforming to block-partitioned ratings) is cached using intermediateRDDStorageLevel, and you can select the final RDD storage level (for user and item factors) using finalRDDStorageLevel. The old MLLIB API now calls the n

ALS rating caching

2016-02-08 Thread Roberto Pagliari
When using ALS from mllib, would it be better/recommended to cache the ratings RDD? I'm asking because when predicting products for users (for example) it is recommended to cache product/user matrices. Thank you,