Re: questions about MLLib recommendation models

2014-08-08 Thread Jay Hutfles
Ah, that makes perfect sense. Thanks for the concise explanation! On Thu, Aug 7, 2014 at 9:14 PM, Xiangrui Meng wrote: > ratings.map{ case Rating(u,m,r) => { > val pred = model.predict(u, m) > (r - pred)*(r - pred) > } > }.mean() > > The code doesn't work because the userFeatures and

Re: questions about MLLib recommendation models

2014-08-07 Thread Xiangrui Meng
ratings.map{ case Rating(u,m,r) => { val pred = model.predict(u, m) (r - pred)*(r - pred) } }.mean() The code doesn't work because the userFeatures and productFeatures stored in the model are RDDs. You tried to serialize them into the task closure, and execute `model.predict` on an execu

Re: questions about MLLib recommendation models

2014-08-07 Thread Burak Yavuz
Hi Jay, I've had the same problem you've been having in Question 1 with a synthetic dataset. I thought I wasn't producing the dataset well enough. This seems to be a bug. I will open a JIRA for it. Instead of using: ratings.map{ case Rating(u,m,r) => { val pred = model.predict(u, m) (r

Re: questions about MLLib recommendation models

2014-08-07 Thread Sean Owen
On Thu, Aug 7, 2014 at 9:06 PM, Jay Hutfles wrote: > 0,0,5 > 0,1,5 > 0,2,0 > 0,3,0 > 1,0,5 > 1,3,0 > 2,1,4 > 2,2,0 > 3,0,0 > 3,1,0 > 3,2,5 > 3,3,4 > 4,0,0 > 4,1,0 > 4,2,5 > val rank = 10 This is likely the problem? your rank is actually larger than the number of users or items. The error could p