Thanks Bryan for that pointer : I will follow it. In the meantime the One vs Rest appears to satisfy the requirements.
2016-05-29 15:40 GMT-07:00 Bryan Cutler <cutl...@gmail.com>: > This is currently being worked on, planned for 2.1 I believe > https://issues.apache.org/jira/browse/SPARK-7159 > On May 28, 2016 9:31 PM, "Stephen Boesch" <java...@gmail.com> wrote: > >> Thanks Phuong But the point of my post is how to achieve without using >> the deprecated the mllib pacakge. The mllib package already has >> multinomial regression built in >> >> 2016-05-28 21:19 GMT-07:00 Phuong LE-HONG <phuon...@gmail.com>: >> >>> Dear Stephen, >>> >>> Yes, you're right, LogisticGradient is in the mllib package, not ml >>> package. I just want to say that we can build a multinomial logistic >>> regression model from the current version of Spark. >>> >>> Regards, >>> >>> Phuong >>> >>> >>> >>> On Sun, May 29, 2016 at 12:04 AM, Stephen Boesch <java...@gmail.com> >>> wrote: >>> > Hi Phuong, >>> > The LogisticGradient exists in the mllib but not ml package. The >>> > LogisticRegression chooses either the breeze LBFGS - if L2 only (not >>> elastic >>> > net) and no regularization or the Orthant Wise Quasi Newton (OWLQN) >>> > otherwise: it does not appear to choose GD in either scenario. >>> > >>> > If I have misunderstood your response please do clarify. >>> > >>> > thanks stephenb >>> > >>> > 2016-05-28 20:55 GMT-07:00 Phuong LE-HONG <phuon...@gmail.com>: >>> >> >>> >> Dear Stephen, >>> >> >>> >> The Logistic Regression currently supports only binary regression. >>> >> However, the LogisticGradient does support computing gradient and loss >>> >> for a multinomial logistic regression. That is, you can train a >>> >> multinomial logistic regression model with LogisticGradient and a >>> >> class to solve optimization like LBFGS to get a weight vector of the >>> >> size (numClassrd-1)*numFeatures. >>> >> >>> >> >>> >> Phuong >>> >> >>> >> >>> >> On Sat, May 28, 2016 at 12:25 PM, Stephen Boesch <java...@gmail.com> >>> >> wrote: >>> >> > Followup: just encountered the "OneVsRest" classifier in >>> >> > ml.classsification: I will look into using it with the binary >>> >> > LogisticRegression as the provided classifier. >>> >> > >>> >> > 2016-05-28 9:06 GMT-07:00 Stephen Boesch <java...@gmail.com>: >>> >> >> >>> >> >> >>> >> >> Presently only the mllib version has the one-vs-all approach for >>> >> >> multinomial support. The ml version with ElasticNet support only >>> >> >> allows >>> >> >> binary regression. >>> >> >> >>> >> >> With feature parity of ml vs mllib having been stated as an >>> objective >>> >> >> for >>> >> >> 2.0.0 - is there a projected availability of the multinomial >>> >> >> regression in >>> >> >> the ml package? >>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> >> >> ` >>> >> > >>> >> > >>> > >>> > >>> >> >>
