Re: Contributing to MLlib on GLM

2014-07-07 Thread Gang Bai
you, I'd be happy to send a PR to your branch. >>>> * In addition to the generated test data, We may use some real-world >> data for testing. In my implementation, I added the test data from >> https://onlinecourses.science.psu.edu/stat504/node/223. Please check my &g

Re: Contributing to MLlib on GLM

2014-07-07 Thread xwei
/223. Please check my > test suite. > >> > >> -Gang > >> Sent from my iPad > >> > >>> On 2014年6月27日, at 下午6:03, "xwei" <[hidden email]> wrote: > >>> > >>> > >>> Yes, that's what we did: adding

Re: Contributing to MLlib on GLM

2014-06-30 Thread Gang Bai
, at 下午6:03, "xwei" <[hidden email]> wrote: >>> >>> >>> Yes, that's what we did: adding two gradient functions to Gradient.scala >>> and >>> create PoissonRegression and GammaRegression using these gradients. We made >>> a PR on this

Re: Contributing to MLlib on GLM

2014-06-28 Thread xwei
t; create PoissonRegression and GammaRegression using these gradients. We made > > a PR on this. > > > > > > > > -- > > View this message in context: > > http://apache-spark-developers-list.1001551.n3.nabble.com/Contributing-to-MLlib-on-GLM-tp7033p7088.htm

Re: Contributing to MLlib on GLM

2014-06-27 Thread 白刚
is. > > > > -- > View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/Contributing-to-MLlib-on-GLM-tp7033p7088.html > Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

Re: Contributing to MLlib on GLM

2014-06-26 Thread xwei
Yes, that's what we did: adding two gradient functions to Gradient.scala and create PoissonRegression and GammaRegression using these gradients. We made a PR on this. -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Contributing-to-MLlib-o

Re: Contributing to MLlib on GLM

2014-06-25 Thread Sung Hwan Chung
Well, as you said, MLLib already supports GLM in a sense. Except they only support two link functions - identity (linear regression) and logit (logistic regression). It should not be too hard to add other link functions, as all you have to do is add a different gradient function for Poisson/Gamma,

Re: Contributing to MLlib on GLM

2014-06-17 Thread Andrew Ash
Hi Xiaokai, Also take a look through Xiangrui's slides from HadoopSummit a few weeks back: http://www.slideshare.net/xrmeng/m-llib-hadoopsummit The roadmap starting at slide 51 will probably be interesting to you. Andrew On Tue, Jun 17, 2014 at 7:37 PM, Sandy Ryza wrote: > Hi Xiaokai, > > I

Re: Contributing to MLlib on GLM

2014-06-17 Thread Sandy Ryza
Hi Xiaokai, I think MLLib is definitely interested in supporting additional GLMs. I'm not aware of anybody working on this at the moment. -Sandy On Tue, Jun 17, 2014 at 5:00 PM, Xiaokai Wei wrote: > Hi, > > I am an intern at PalantirTech and we are building some stuff on top of > MLlib. In P

Contributing to MLlib on GLM

2014-06-17 Thread Xiaokai Wei
Hi, I am an intern at PalantirTech and we are building some stuff on top of MLlib. In Particular, GLM is of great interest to us. Though GeneralizedLinearModel in MLlib 1.0.0 has some important GLMs such as Logistic Regression, Linear Regression, some other important GLMs like Poisson Regression