Specifying Scala types when calling methods from SparkR

[Chris Freeman]

Hey everyone,

I’m currently looking at ways to save out SparkML model objects from SparkR and 
I’ve had some luck putting the model into an RDD and then saving the RDD as an 
Object File. Once it’s saved, I’m able to load it back in with something like:

sc.objectFile[LinearRegressionModel](“path/to/model”)

I’d like to try and replicate this same process from SparkR using the JVM 
backend APIs (e.g. “callJMethod”), but so far I haven’t been able to replicate 
my success and I’m guessing that it’s (at least in part) due to the necessity 
of specifying the type when calling the objectFile method.

Does anyone know if this is actually possible? For example, here’s what I’ve 
come up with so far:

loadModel <- function(sc, modelPath) {
  modelRDD <- SparkR:::callJMethod(sc,
                                                            
"objectFile[PipelineModel]",
                                modelPath,
        SparkR:::getMinPartitions(sc, NULL))
  return(modelRDD)
}

Any help is appreciated!

--
Chris Freeman