The thing about MatrixFactorizationModel, compared to other models, is
that it is huge. It's not just a few coefficients, but whole RDDs of
coefficients. I think you could save these RDDs of user/product
factors to persistent storage, load them, then recreate the
MatrixFactorizationModel that way.
Hi Albert,
There is some discussion going on here:
http://apache-spark-user-list.1001560.n3.nabble.com/MLLIB-model-export-PMML-vs-MLLIB-serialization-tc20324.html#a20674
I am also looking for this solution.But looks like until mllib pmml export
is ready, there is no full proof solution to export th
In that case, what is the strategy to train a model in some background
batch process and make recommendations for some other service in real
time? Run both processes in the same spark cluster?
Thanks.
--
Albert Manyà
alber...@eml.cc
On Mon, Dec 15, 2014, at 05:58 PM, Sean Owen wrote:
> This
This class is not going to be serializable, as it contains huge RDDs.
Even if the right constructor existed the RDDs inside would not
serialize.
On Mon, Dec 15, 2014 at 4:33 PM, Albert Manyà wrote:
> Hi all.
>
> I'm willing to serialize and later load a model trained using mllib's
> ALS.
>
> I've
Hi all.
I'm willing to serialize and later load a model trained using mllib's
ALS.
I've tried usign Java serialization with something like:
val model = ALS.trainImplicit(training, rank, numIter, lambda, 1)
val fos = new FileOutputStream("model.bin")
val oos = new ObjectOutputStream(f
