Re: Collaborative filtering steps in spark

2017-03-29 Thread chris snow
k)(random.nextGaussian().toFloat) > val nrm = blas.snrm2(rank, factor, 1) > blas.sscal(rank, 1.0f / nrm, factor, 1) > factor > } > (srcBlockId, factors) > } > } > > > factor is ~ N(0, 1) and then scaled by the L2 norm, but it looks to me the > abs value is

Collaborative filtering steps in spark

2017-03-26 Thread chris snow
In the paper “Large-Scale Parallel Collaborative Filtering for the Netflix Prize”, the following steps are described for ALS: Step 1 Initialize matrix M by assigning the average rating for that movie as the first row, and small random numbers for the remaining entries. Step 2 Fix M, Solve U by min

Re: Collaborative Filtering - scaling of the regularization parameter

2017-03-23 Thread chris snow
Thanks Nick. If this will help other users, I'll create a JIRA and send a patch. On 23 March 2017 at 13:49, Nick Pentreath wrote: > Yup, that is true and a reasonable clarification of the doc. > > On Thu, 23 Mar 2017 at 00:03 chris snow wrote: >> >> The docum

Collaborative Filtering - scaling of the regularization parameter

2017-03-23 Thread chris snow
The documentation for collaborative filtering is as follows: === Scaling of the regularization parameter Since v1.1, we scale the regularization parameter lambda in solving each least squares problem by the number of ratings the user generated in updating user factors, or the number of ratings th