Thanks Ram. Your sample look is very helpful. (there is a minor bug that PipelineModel.stages is hidden under private[ml], just need a wrapper around it. :)
Justin On Sat, May 16, 2015 at 10:44 AM, Ram Sriharsha <sriharsha....@gmail.com> wrote: > Hi Justin > > The CrossValidatorExample here > https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/ml/CrossValidatorExample.scala > is a good example of how to set up an ML Pipeline for extracting a model > with the best parameter set. > > You set up the pipeline as in here: > > https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/ml/CrossValidatorExample.scala#L73 > > This pipeline is treated as an estimator and wrapped into a Cross > Validator to do grid search and return the model with the best parameters . > Once you have trained the best model as in here > > https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/ml/CrossValidatorExample.scala#L93 > > The result is a CrossValidatorModel which contains the best estimator > (i.e. the best pipeline above) and you can extract the best pipeline and > inquire its parameters as follows: > > // what are the best parameters? > val bestPipelineModel = cvModel.bestModel.asInstanceOf[PipelineModel] > val stages = bestPipelineModel.stages > > val hashingStage = stages(1).asInstanceOf[HashingTF] > println(hashingStage.getNumFeatures) > val lrStage = stages(2).asInstanceOf[LogisticRegressionModel] > println(lrStage.getRegParam) > > > > Ram > > On Sat, May 16, 2015 at 3:17 AM, Justin Yip <yipjus...@prediction.io> > wrote: > >> Hello, >> >> I am using MLPipeline. I would like to extract the best parameter found >> by CrossValidator. But I cannot find much document about how to do it. Can >> anyone give me some pointers? >> >> Thanks. >> >> Justin >> >> ------------------------------ >> View this message in context: Getting the best parameter set back from >> CrossValidatorModel >> <http://apache-spark-user-list.1001560.n3.nabble.com/Getting-the-best-parameter-set-back-from-CrossValidatorModel-tp22915.html> >> Sent from the Apache Spark User List mailing list archive >> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >> > >
