problem with saving RandomForestClassifier model - Saprk Java

2018-05-22 Thread Donni Khan
Hi SPark users, I built Random forest model by using Spark 1.6 with Java. I'm getting the following exception: User class threw exception: java.lang.UnsupportedOperationException: Pipeline write will fail on this Pipeline because it contains a stage which does not implement Writable. Does anyon

Re: Spark ML's RandomForestClassifier OOM

2017-01-10 Thread Julio Antonio Soto de Vicente
n 10 Jan 2017 10:07 am, "Julio Antonio Soto" wrote: >> Hi, >> >> I am running into OOM problems while training a Spark ML >> RandomForestClassifier (maxDepth of 30, 32 maxBins, 100 trees). >> >> My dataset is arguably pretty big given the exec

Re: Spark ML's RandomForestClassifier OOM

2017-01-10 Thread Marco Mistroni
You running locally? Found exactly same issue. 2 solutions: _ reduce datA size. _ run on EMR Hth On 10 Jan 2017 10:07 am, "Julio Antonio Soto" wrote: > Hi, > > I am running into OOM problems while training a Spark ML > RandomForestClassifier (maxDepth of 30, 32 maxBi

Spark ML's RandomForestClassifier OOM

2017-01-10 Thread Julio Antonio Soto
Hi, I am running into OOM problems while training a Spark ML RandomForestClassifier (maxDepth of 30, 32 maxBins, 100 trees). My dataset is arguably pretty big given the executor count and size (8x5G), with approximately 20M rows and 130 features. The "fun fact" is tha

Re: how to print auc & prc for GBTClassifier, which is okay for RandomForestClassifier

2016-11-28 Thread Nick Pentreath
prediction value, or due to waiting for future multi-class support before implementing all the classifier methods. On Sun, 27 Nov 2016 at 19:52 Zhiliang Zhu wrote: > > Hi All, > > I need to print auc and prc for GBTClassifier model, it seems okay for > RandomForestClassifier but

how to print auc & prc for GBTClassifier, which is okay for RandomForestClassifier

2016-11-27 Thread Zhiliang Zhu
Hi All, I need to print auc and prc for GBTClassifier model, it seems okay for RandomForestClassifier but not GBTClassifier, though rawPrediction column is neither in original data. the codes are : ..    // Set up Pipeline    val stages = new

Re: RandomForestClassifier

2016-07-20 Thread Marco Mistroni
Hi afaik yes (other pls override ). Generally, in RandomForest and DecisionTree you have a column which you are trying to 'predict' (the label) and a set of features that are used to predict the outcome. i would assume that if you specify thelabel column and the 'features' columns, everything else

RandomForestClassifier

2016-07-20 Thread pseudo oduesp
hi , we have parmaters named labelCol="labe" ,featuresCol="features", when i precise the value here (label and features) if train my model on data frame with other columns tha algorithme choos only label columns and features columns ? thanks

Re: Spark issue with running CrossValidator with RandomForestClassifier on dataset

2015-07-13 Thread Feynman Liang
ing else that I'm > missing? > > Thanks! > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-issue-with-running-CrossValidator-with-RandomForestClassifier-on-dataset-tp23791.html > Sent from the Apache Spark User Li

Spark issue with running CrossValidator with RandomForestClassifier on dataset

2015-07-13 Thread shivamverma
with the evaluator, or is there something else that I'm missing? Thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-issue-with-running-CrossValidator-with-RandomForestClassifier-on-dataset-tp23791.html Sent from the A