Re: How to save mllib model to hdfs and reload it

2014-09-13 Thread Yanbo Liang
, 2014 7:03 AM, "lancezhange" wrote: >>>> >>>>> my prediction codes are simple enough as follows: >>>>> >>>>> *val labelsAndPredsOnGoodData = goodDataPoints.map { point => >>>>> val prediction = model.predict(point.fe

Re: How to save mllib model to hdfs and reload it

2014-08-14 Thread lancezhange
; constructors,"This is part of SPARK-2495 to allow users construct linear models manually" -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-save-mllib-model-to-hdfs-and-reload-it-tp11953p12154.html Sent from the Apache Spark User List mailing li

Re: How to save mllib model to hdfs and reload it

2014-08-14 Thread Shixiong Zhu
" to see if that throws >>>>> any visible exceptions. >>>>> >>>>> Sent while mobile. Pls excuse typos etc. >>>>> On Aug 13, 2014 7:03 AM, "lancezhange" <[hidden email] >>>>> <http://user/Se

Re: How to save mllib model to hdfs and reload it

2014-08-14 Thread lancezhange
gt;>>> On Aug 13, 2014 7:03 AM, "lancezhange" <[hidden email] >>>> <http://user/SendEmail.jtp?type=node&node=12112&i=3>> wrote: >>>> >>>>> my prediction codes are simple enough as follows: >>>>> >&g

Re: How to save mllib model to hdfs and reload it

2014-08-14 Thread Shixiong Zhu
p { point => >>>> val prediction = model.predict(point.features) >>>> (point.label, prediction) >>>> }* >>>> >>>> when model is the loaded one, above code just can't work. Can you catch >>>> the >>>

Re: How to save mllib model to hdfs and reload it

2014-08-14 Thread Christopher Nguyen
atch >>> the >>> error? >>> Thanks. >>> >>> PS. i use spark-shell under standalone mode, version 1.0.0 >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list

Re: How to save mllib model to hdfs and reload it

2014-08-14 Thread Hoai-Thu Vuong
e just can't work. Can you catch >> the >> error? >> Thanks. >> >> PS. i use spark-shell under standalone mode, version 1.0.0 >> >> >> >> >> -- >> View this message in context: >> http://ap

Re: How to save mllib model to hdfs and reload it

2014-08-13 Thread Christopher Nguyen
model is the loaded one, above code just can't work. Can you catch the > error? > Thanks. > > PS. i use spark-shell under standalone mode, version 1.0.0 > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-s

Re: How to save mllib model to hdfs and reload it

2014-08-13 Thread Christopher Nguyen
+1 what Sean said. And if there are too many state/argument parameters for your taste, you can always create a dedicated (serializable) class to encapsulate them. Sent while mobile. Pls excuse typos etc. On Aug 13, 2014 6:58 AM, "Sean Owen" wrote: > PS I think that solving "not serializable" exc

Re: How to save mllib model to hdfs and reload it

2014-08-13 Thread lancezhange
. PS. i use spark-shell under standalone mode, version 1.0.0 -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-save-mllib-model-to-hdfs-and-reload-it-tp11953p12035.html Sent from the Apache Spark User List mailing list archive at Na

Re: How to save mllib model to hdfs and reload it

2014-08-13 Thread Sean Owen
PS I think that solving "not serializable" exceptions by adding 'transient' is usually a mistake. It's a band-aid on a design problem. transient causes the default serialization mechanism to not serialize the field when the object is serialized. When deserialized, this field will be null, which of

Re: How to save mllib model to hdfs and reload it

2014-08-13 Thread Jaideep Dhok
JoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)ed, > from pool > at > scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > at > > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > > Any help here? >

Re: How to save mllib model to hdfs and reload it

2014-08-13 Thread lancezhange
ng you have weights and intercept? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-save-mllib-model-to-hdfs-and-reload-it-tp11953p12030.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: How to save mllib model to hdfs and reload it

2014-08-12 Thread Xiangrui Meng
e tips. > > Thanks > > XiaoQinyu > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-save-mllib-model-to-hdfs-and-reload-it-tp11953.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > ---

How to save mllib model to hdfs and reload it

2014-08-11 Thread XiaoQinyu
re,if anyone can give some tips. Thanks XiaoQinyu -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-save-mllib-model-to-hdfs-and-reload-it-tp11953.html Sent from the Apache Spark User List mailing list archive at Nabbl