Yup...this can be a spark community project...I saw a PR for that...interested users fine with lgpl/gpl code can make use of it...
On Mon, Sep 8, 2014 at 12:37 PM, Xiangrui Meng <men...@gmail.com> wrote: > I asked Tim whether he would change the license of SuiteSparse to an > Apache-friendly license couple months ago, but the answer was no. So I > don't think we can use SuiteSparse in MLlib through JNI. Please feel > free to create JIRAs for distributed linear programming and SOCP > solvers and run the discussion there. I'm very interested since I > don't really know how to do linear programming in a distributed way. > -Xiangrui > > On Mon, Sep 8, 2014 at 7:12 AM, Debasish Das <debasish.da...@gmail.com> > wrote: > > Xiangrui, > > > > Should I open up a JIRA for this ? > > > > Distributed lp/socp solver through ecos/ldl/amd ? > > > > I can open source it with gpl license in spark code as that's what our > legal > > cleared (apache + gpl becomes gpl) and figure out the right way to call > > it...ecos is gpl but we can definitely use the jni version of ldl and amd > > which are lgpl... > > > > Let me know. > > > > Thanks. > > Deb > > > > On Sep 8, 2014 7:04 AM, "Debasish Das" <debasish.da...@gmail.com> wrote: > >> > >> Durin, > >> > >> I have integrated ecos with spark which uses suitesparse under the hood > >> for linear equation solves....I have exposed only the qp solver api in > spark > >> since I was comparing ip with proximal algorithms but we can expose > >> suitesparse api as well...jni is used to load up ldl amd and ecos > libraries. > >> > >> Please follow ecos section of my spark summit talk. We can discuss more > >> but we can formulate interesting things like google's ceres solver's > trust > >> region formulation. > >> > >> > >> > http://spark-summit.org/2014/talk/quadratic-programing-solver-for-non-negative-matrix-factorization-with-spark > >> > >> Let me point you to the code so that you can take a look at it. > >> Suitesparse (ldl and amd) is lgpl but ecos is gpl and therefore I was > not > >> sure how straightforward it will be to add the solver to mllib. Our > legal > >> was not convinced to add lgpl/gpl code in apache project. > >> > >> Could you also detail the usecases you are looking for ? You want a > >> distributed lp / socp solver where each worker solves a partition of the > >> constraint and the full objective...and you want to converge to a global > >> solution using consensus ? Or your problem has more structure to > partition > >> the problem cleanly and don't need consensus step (which is what I > >> implemented in the code) > >> > >> Thanks > >> Deb > >> > >> On Sep 7, 2014 11:35 PM, "Xiangrui Meng" <men...@gmail.com> wrote: > >>> > >>> You can try LinearRegression with sparse input. It converges the least > >>> squares solution if the linear system is over-determined, while the > >>> convergence rate depends on the condition number. Applying standard > >>> scaling is popular heuristic to reduce the condition number. > >>> > >>> If you are interested in sparse direct methods as in SuiteSparse. I'm > >>> not aware of any effort to do so. > >>> > >>> -Xiangrui > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > >>> For additional commands, e-mail: user-h...@spark.apache.org > >>> > > >
