Re: Constraint Solver for Spark

Sorry last one went out by mistake: Is not for users (0 to numUsers), fullXtX is same ? In the ALS formulation this is W^TW or H^TH which should be same for all the users ? Why we are reading userXtX(index) and adding it to fullXtX in the loop over all numUsers ? // Solve the least-squares proble

Re: Constraint Solver for Spark

Hi, I am bit confused wiht the code here: // Solve the least-squares problem for each user and return the new feature vectors Array.range(0, numUsers).map { index => // Compute the full XtX matrix from the lower-triangular part we got above fillFullMatrix(userXtX(index), fullXt

Re: Run ScalaTest inside Intellij IDEA

I also run into this problem when running examples in IDEA. The issue looks that it uses depends on too many jars and that the classpath seems to have length limit. So I import the assembly jar and put the head of the list dependent path and it works. Thanks, Qiuzhuang On Wed, Jun 11, 2014 at 10

Re: Suggestion: rdd.compute()

You can achieve an equivalent effect by calling rdd.foreach(x => {}), which is the lightest possible action that forces materialization of the whole RDD. Ankur

Suggestion: rdd.compute()

Hi, Regarding the following scenario, Would it be nice to have an action method named like 'compute()' that does nothing but computing/materializing the whole partitions of an RDD? It can also be useful for the profiling. -Original Message- From: innowireless TaeYun Kim [mailto:taeyun...

Run ScalaTest inside Intellij IDEA

Hi All, I want to run ScalaTest Suite in IDEA directly, but it seems didn’t pass the make phase before test running. The problems are as follows: /Users/yijie/code/apache.spark.master/core/src/main/scala/org/apache/spark/executor/MesosExecutorBackend.scala Error:(44, 35) type mismatch; found

Re: Emergency maintenace on jenkins

Hey just to update people - as of around 1pm PT we were back up and running with Jenkins slaves on EC2. Sorry about the disruption. - Patrick On Tue, Jun 10, 2014 at 1:15 AM, Patrick Wendell wrote: > No luck with this tonight - unfortunately our Python tests aren't > working well with Python 2.6

Re: debugger

Hi Josh, I came across this post when looking for a debugger or RDD visualization tool for Spark. I am using Spark 0.9.1 and upgrading soon to Spark 1.0. The links you posted are dead. Can you please direct me to how I can debug my existing Spark job. Will I need to edit my existing job's code in

Re: Emergency maintenace on jenkins

No luck with this tonight - unfortunately our Python tests aren't working well with Python 2.6 and some other issues made it hard to get the EC2 worker up to speed. Hopefully we can have this up and running tomororw. - Patrick On Mon, Jun 9, 2014 at 10:17 PM, Patrick Wendell wrote: > Just a head