Re: mLIb solving linear regression with sparse inputs

Well I did eventually write this code in Java, and it was very long! see https://github.com/insidedctm/sparse-linear-regression - Robin East Spark GraphX in Action Michael Malak and Robin East Manning Publications Co. http://www.

Re: mLIb solving linear regression with sparse inputs

Also in Java as well. Thanks again! Iman On Sun, Nov 6, 2016 at 8:28 AM Iman Mohtashemi wrote: Hi Robin, It looks like the linear regression model takes in a dataset not a matrix? It would be helpful for this example if you could set up the whole problem end to end using one of the columns of th

Re: mLIb solving linear regression with sparse inputs

Hi Robin, It looks like the linear regression model takes in a dataset not a matrix? It would be helpful for this example if you could set up the whole problem end to end using one of the columns of the matrix as b. So A is a sparse matrix and b is a sparse vector Best regards. Iman On Sun, Nov 6,

Re: mLIb solving linear regression with sparse inputs

Thank you! Would happen to have this code in Java?. This is extremely helpful! Iman On Sun, Nov 6, 2016 at 3:35 AM -0800, "Robineast [via Apache Spark User List]" wrote: Here’s a way of creating sparse vectors in MLLib: import org.apache.spark.mllib.linalg.Vectorsimp

Re: mLIb solving linear regression with sparse inputs

Here’s a way of creating sparse vectors in MLLib: import org.apache.spark.mllib.linalg.Vectors import org.apache.spark.rdd.RDD val rdd = sc.textFile("A.txt").map(line => line.split(",")). map(ary => (ary(0).toInt, ary(1).toInt, ary(2).toDouble)) val pairRdd: RDD[(Int, (Int, Int, Double))] =

Re: mLIb solving linear regression with sparse inputs

Any reason why you can’t use built in linear regression e.g. http://spark.apache.org/docs/latest/ml-classification-regression.html#regression or http://spark.apache.org/docs/latest/mllib-linear-methods.html#linear-least-squares-lasso-and-ridge-regression? ---