How to convert a Random Forest model built in R to a similar model in Spark

kages/randomForest/randomForest.pdf > > mytry=3 > > ntree=500 > > importance=TRUE > > maxnodes = NULL > > On May 31, 2016 7:03 AM, "Sun Rui" wrote: >> >> I mean train random forest model (not using R) and use it for prediction together using Spark M

How to convert a Random Forest model built in R to a similar model in Spark

packages/randomForest/randomForest.pdf> mytry=3 ntree=500 importance=TRUE maxnodes = NULL On May 31, 2016 7:03 AM, "Sun Rui" wrote: I mean train random forest model (not using R) and use it for prediction together using Spark ML. On May 30, 2016, at 20:15, Neha Mehta wrote: Th

Re: Ignore features in Random Forest

> Regards, > Yuhao > > 2016-06-01 21:18 GMT+08:00 Neha Mehta : > >> Hi, >> >> I am performing Regression using Random Forest. In my input vector, I >> want the algorithm to ignore certain columns/features while training the >> classifier and also whi

Ignore features in Random Forest

Hi, I am performing Regression using Random Forest. In my input vector, I want the algorithm to ignore certain columns/features while training the classifier and also while prediction. These are basically Id columns. I checked the documentation and could not find any information on the same. Requ

Re: Can we use existing R model in Spark

t; Subject: Re: Can we use existing R model in Spark > To: Sun Rui > Cc: Neha Mehta , user > > > Try to invoke a R script from Spark using rdd pipe method , get the work > done & and receive the model back in RDD. > > > for ex :- > . rdd.pipe(&quo

Can we use existing R model in Spark

Hi, I have an existing random forest model created using R. I want to use that to predict values on Spark. Is it possible to do the same? If yes, then how? Thanks & Regards, Neha

Re: How to call a custom function from GroupByKey which takes Iterable[Row] as input and returns a Map[Int,String] as output in scala

rdd = sc.parallelize(arr) > > val prdd = rdd.map(a => (a._1,a)) > val totals = prdd.groupByKey.map(a => (a._1, a._2.size)) > > var n1 = rdd.map(a => ((a._1, a._2), 1) ) > var n2 = n1.reduceByKey(_+_).map(a => (a._1._1, (a._1._2, a._2))) > var n3 = n2.join(to

How to call a custom function from GroupByKey which takes Iterable[Row] as input and returns a Map[Int,String] as output in scala

C$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:112) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:114) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:116) .. Thanks for the help. Regards, Neha Mehta

How to call a custom function from GroupByKey which takes Iterable[Row] as input and returns a Map[Int,String] as output in scala

C$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:112) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:114) at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:116) .. Thanks for the help. Regards, Neha Mehta