Good clarification Sean. Diana, I was also referring to this example when setting up some of my bigger ALS runs. I don't this particular example is very helpful, as it is creating the initial matrix locally in memory before parallelizing in spark. So (unless I'm misunderstanding), it is an ok example at demonstrating basic spark functionality, but since it is limited by local memory you obviously can't demo it on bigger datasets.
The example at spark.apache.org/docs/0.9.1/mllib-guide.html I found to be much more helpful for good use cases. -----Original Message----- From: Sean Owen [mailto:so...@cloudera.com] Sent: Monday, April 28, 2014 1:41 PM To: user@spark.apache.org Subject: Re: running SparkALS Yeah you'd have to look at the source code in this case; it's not explained in the scaladoc or usage message as far as I can see either. The args refer specifically to the example of recommending Movies to Users. This example makes up a bunch of ratings and then makes recommendations using ALS. M = number of movies in the made-up data U = number of users F = number of latent factors in the ALS model iter = number of iterations to run ALS for slices = level of parallelism to use on Spark I'd say that could be explained in the scaladoc at least. As an aside though, looking at the code, this is not a demo of ALS from MLlib, but a different implementation and based on the CERN linear algebra libraries. Is this still a good example? -- Sean Owen | Director, Data Science | London On Mon, Apr 28, 2014 at 6:30 PM, Diana Carroll <dcarr...@cloudera.com> wrote: > Hi everyone. I'm trying to run some of the Spark example code, and > most of it appears to be undocumented (unless I'm missing something). > Can someone help me out? > > I'm particularly interested in running SparkALS, which wants parameters: > M U F iter slices > > What are these variables? They appear to be integers and the default > values are 100, 500 and 10 respectively but beyond that...huh? > > Thanks! > > Diana ________________________________________________________ The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
