Hi ,
please i need help about that question
2017-05-15 10:32 GMT+02:00 issues solution :
> Hi,
> I am under Pyspark 1.6 i want save my model in hdfs file like parquet
>
> how i can do this ?
>
>
> My model it s a RandomForestClassifier performed with corssvalidation
>
> like this
>
>
>
> r
Hi,
I am under Pyspark 1.6 i want save my model in hdfs file like parquet
how i can do this ?
My model it s a RandomForestClassifier performed with corssvalidation
like this
rf_csv2 = CrossValidator()
how i can save it ?
thx for adavance
cv.fit is going to give you a CrossValidatorModel, if you want to extract
the real model built. You need to do
val cvModel = cv.fit(data)
val plmodel = cvModel.bestModel.asInstanceOf[PipelineModel]
val model = plmodel.stages(2).asInstanceOf[whatever_model]
then you can model.save
O
Hi,
Thanks Rezaul and Asher Krim.
The method suggested by Rezaul works fine for NaiveBayes but still fails
for RandomForest and Multi-layer perceptron classifier.
Everything properly is saved until this stage.
CrossValidator cv = new CrossValidator()
.setEstimator(pipeline)
.setE
What version of Spark are you on?
Although it's cut off, I think your error is with RandomForestClassifier,
is that correct? If so, you should upgrade to spark 2 since I think this
class only became writeable/readable in Spark 2 (
https://github.com/apache/spark/pull/12118)
On Thu, Jan 12, 2017 at
Hi Malshan,
The error says that one (or more) of the estimators/stages is either not
writable or compatible that supports overwrite/model write operation.
Suppose you want to configure an ML pipeline consisting of three stages
(i.e. estimator): tokenizer, hashingTF, and nb:
val nb = new Naive
Hi,
When I try to save a pipeline model using spark ML (Java) , the following
exception is thrown.
java.lang.UnsupportedOperationException: Pipeline write will fail on this
Pipeline because it contains a stage which does not implement Writable.
Non-Writable stage: rfc_98f8c9e0bd04 of type class
