In that case, what is the strategy to train a model in some background
batch process and make recommendations for some other service in real
time? Run both processes in the same spark cluster?
Thanks.
--
Albert Manyà
alber...@eml.cc
On Mon, Dec 15, 2014, at 05:58 PM, Sean Owen wrote:
> This class is not going to be serializable, as it contains huge RDDs.
> Even if the right constructor existed the RDDs inside would not
> serialize.
>
> On Mon, Dec 15, 2014 at 4:33 PM, Albert Manyà <alber...@eml.cc> wrote:
> > Hi all.
> >
> > I'm willing to serialize and later load a model trained using mllib's
> > ALS.
> >
> > I've tried usign Java serialization with something like:
> >
> > val model = ALS.trainImplicit(training, rank, numIter, lambda, 1)
> > val fos = new FileOutputStream("model.bin")
> > val oos = new ObjectOutputStream(fos)
> > oos.writeObject(bestModel.get)
> >
> > But when I try to deserialize it using:
> >
> > val fos = new FileInputStream("model.bin")
> > val oos = new ObjectInputStream(fos)
> > val model = oos.readObject().asInstanceOf[MatrixFactorizationModel]
> >
> > I get the error:
> >
> > Exception in thread "main" java.io.IOException: PARSING_ERROR(2)
> >
> > I've also tried to serialize MatrixFactorizationModel's both RDDs
> > (products and users) and later create the MatrixFactorizationModel by
> > hand passing the RDDs by constructor but I get an error cause its
> > private:
> >
> > Error:(58, 17) constructor MatrixFactorizationModel in class
> > MatrixFactorizationModel cannot be accessed in object RecommendALS
> > val model = new MatrixFactorizationModel (8, userFeatures,
> > productFeatures)
> >
> > Any ideas?
> >
> > Thanks!
> >
> > --
> > Albert Manyà
> > alber...@eml.cc
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> > For additional commands, e-mail: user-h...@spark.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
