Retrieve best parameters from CrossValidator

2015-11-28 Thread BenFradet
Hi everyone, Is there a better way to retrieve the best model parameters obtained from cross validation than inspecting the logs issued while calling the fit method (with the call here: https://github.com/apache/spark/blob/branch-1.5/mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.s

Drop multiple columns in the DataFrame API

2015-11-20 Thread BenFradet
Hi everyone, I was wondering if there is a better way to drop mutliple columns from a dataframe or why there is no drop(cols: Column*) method in the dataframe API. Indeed, I tend to write code like this: val filteredDF = df.drop("colA") .drop("colB") .drop("colC") //etc which is a bit