Re: Spark Matrix Factorization

2014-06-28 Thread Krakna H
Hi Deb, Putting your code on github will be much appreciated -- it will give us a good starting point to adapt for our purposes. Regards. On Sat, Jun 28, 2014 at 10:57 AM, Debasish Das [via Apache Spark Developers List] wrote: > Factorization problems are non-convex and so both ALS and DSGD w

Re: Spark Matrix Factorization

2014-06-28 Thread Debasish Das
Factorization problems are non-convex and so both ALS and DSGD will converge to local minima and it is not clear which minima will be better than the other until we run both the algorithms and see... So I will still say get a DSGD version running in the test setup while you experiment with the Spa

Re: Spark Matrix Factorization

2014-06-28 Thread Krakna H
Hi Deb, Thanks so much for your response! At this point, we haven't determined which of DSGD/ALS to go with and were waiting on guidance like yours to tell us what the right option would be. It looks like ALS seems to be good enough for our purposes. Regards. On Fri, Jun 27, 2014 at 12:47 PM, D

Re: Spark Matrix Factorization

2014-06-27 Thread Debasish Das
Hi, In my experiments with Jellyfish I did not see any substantial RMSE loss over DSGD for Netflix dataset... So we decided to stick with ALS and implemented a family of Quadratic Minimization solvers that stays in the ALS realm but can solve interesting constraints(positivity, bounds, L1, equali

Re: Spark Matrix Factorization

2014-06-27 Thread Krakna H
Hi all, Just found this thread -- is there an update on including DSGD in Spark? We have a project that entails topic modeling on a document-term matrix using matrix factorization, and were wondering if we should use ALS or attempt writing our own matrix factorization implementation on top of Spar