Hi, I see ALS is still using Array[Int] but for other mllib algorithm we moved to Vector[Double] so that it can support either dense and sparse formats...
ALS can stay in Array[Int] due to the Netflix format for input datasets which is well defined but it helps if we move ALS to Vector[Double] as well...that way all algorithms will be consistent... The second issue is that toString on SparseVector does not write libsvm format but something not very generic...can we change the SparseVector.toString to write as libsvm output ? I am dumping a sample of dataset to see how mllib glm compares with the glmnet-R package for QoR... Thanks. Deb On Mon, May 5, 2014 at 4:05 PM, David Hall <d...@cs.berkeley.edu> wrote: > >> On Mon, May 5, 2014 at 3:40 PM, DB Tsai <dbt...@stanford.edu> wrote: >> >> > David, >> > >> > Could we use Int, Long, Float as the data feature spaces, and Double for >> > optimizer? >> > >> >> Yes. Breeze doesn't allow operations on mixed types, so you'd need to >> convert the double vectors to Floats if you wanted, e.g. dot product with >> the weights vector. >> >> You might also be interested in FeatureVector, which is just a wrapper >> around Array[Int] that emulates an indicator vector. It supports dot >> products, axpy, etc. >> >> -- David >> >> >> > >> > >> > Sincerely, >> > >> > DB Tsai >> > ------------------------------------------------------- >> > My Blog: https://www.dbtsai.com >> > LinkedIn: https://www.linkedin.com/in/dbtsai >> > >> > >> > On Mon, May 5, 2014 at 3:06 PM, David Hall <d...@cs.berkeley.edu> >> wrote: >> > >> > > Lbfgs and other optimizers would not work immediately, as they require >> > > vector spaces over double. Otherwise it should work. >> > > On May 5, 2014 3:03 PM, "DB Tsai" <dbt...@stanford.edu> wrote: >> > > >> > > > Breeze could take any type (Int, Long, Double, and Float) in the >> matrix >> > > > template. >> > > > >> > > > >> > > > Sincerely, >> > > > >> > > > DB Tsai >> > > > ------------------------------------------------------- >> > > > My Blog: https://www.dbtsai.com >> > > > LinkedIn: https://www.linkedin.com/in/dbtsai >> > > > >> > > > >> > > > On Mon, May 5, 2014 at 2:56 PM, Debasish Das < >> debasish.da...@gmail.com >> > > > >wrote: >> > > > >> > > > > Is this a breeze issue or breeze can take templates on float / >> > double ? >> > > > > >> > > > > If breeze can take templates then it is a minor fix for >> Vectors.scala >> > > > right >> > > > > ? >> > > > > >> > > > > Thanks. >> > > > > Deb >> > > > > >> > > > > >> > > > > On Mon, May 5, 2014 at 2:45 PM, DB Tsai <dbt...@stanford.edu> >> wrote: >> > > > > >> > > > > > +1 Would be nice that we can use different type in Vector. >> > > > > > >> > > > > > >> > > > > > Sincerely, >> > > > > > >> > > > > > DB Tsai >> > > > > > ------------------------------------------------------- >> > > > > > My Blog: https://www.dbtsai.com >> > > > > > LinkedIn: https://www.linkedin.com/in/dbtsai >> > > > > > >> > > > > > >> > > > > > On Mon, May 5, 2014 at 2:41 PM, Debasish Das < >> > > debasish.da...@gmail.com >> > > > > > >wrote: >> > > > > > >> > > > > > > Hi, >> > > > > > > >> > > > > > > Why mllib vector is using double as default ? >> > > > > > > >> > > > > > > /** >> > > > > > > >> > > > > > > * Represents a numeric vector, whose index type is Int and >> value >> > > > type >> > > > > is >> > > > > > > Double. >> > > > > > > >> > > > > > > */ >> > > > > > > >> > > > > > > trait Vector extends Serializable { >> > > > > > > >> > > > > > > >> > > > > > > /** >> > > > > > > >> > > > > > > * Size of the vector. >> > > > > > > >> > > > > > > */ >> > > > > > > >> > > > > > > def size: Int >> > > > > > > >> > > > > > > >> > > > > > > /** >> > > > > > > >> > > > > > > * Converts the instance to a double array. >> > > > > > > >> > > > > > > */ >> > > > > > > >> > > > > > > def toArray: Array[Double] >> > > > > > > >> > > > > > > Don't we need a template on float/double ? This will give us >> > memory >> > > > > > > savings... >> > > > > > > >> > > > > > > Thanks. >> > > > > > > >> > > > > > > Deb >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
