This is because we do not have excellent coverage for Java-friendly
wrappers.
I found we only implement JavaParams who is the wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from
Scala Params.
For example, in Scala we have:
trait HasLabelCol extends Params
We should have the Java-friendly wrappers as follows:
class JavaHasLabelCol extends JavaParams
Then each params of the Estimator will be generated correctly and param
validation will success.
I think this should be further defined.
2016-05-26 18:54 GMT-07:00 Benjii519 <benjii.man.sm...@gmail.com>:
> Hello,
>
> Let me preface this with the fact that I am completely new to Spark and
> Scala, so I may be missing something basic.
>
> I have been looking at implementing a clustering algorithm on top of
> SparkML
> using Java, and ran into immediate problems. As a sanity check, I went to
> the Java API example, but encountered the same behavior: I am unable to set
> parameters on a Java defined Estimator
>
> Focusing on the JavaDeveloperApiExample, as I trust that more than my code,
> I encounter the exception pasted at end of post.
>
> Digging around the Spark code, it looks like adding parameters through Java
> is broken because the Scala params implementation is using reflection to
> determine valid parameters. This works fine in the Scala Estimators as they
> appear to use implementation specific params as a trait. In the Java case,
> the params are a generic base class and reflection on params won't find
> anything to populate (all defined on the Estimator class). Therefore, when
> I
> try to set a parameter on the estimator, the validation fails as an unknown
> parameter.
>
> Any feedback / suggestions? Is this a known issue?
>
> Thanks!
>
> Exception in thread "main" java.lang.IllegalArgumentException: requirement
> failed: Param myJavaLogReg_d3e770dacdc9__maxIter does not belong to
> myJavaLogReg_d3e770dacdc9.
> at scala.Predef$.require(Predef.scala:233)
> at
> org.apache.spark.ml.param.Params$class.shouldOwn(params.scala:740)
> at org.apache.spark.ml.param.Params$class.set(params.scala:618)
> at org.apache.spark.ml.PipelineStage.set(Pipeline.scala:43)
> at org.apache.spark.ml.param.Params$class.set(params.scala:604)
> at org.apache.spark.ml.PipelineStage.set(Pipeline.scala:43)
> at
>
> org.apache.spark.examples.ml.MyJavaLogisticRegression.setMaxIter(JavaDeveloperApiExample.java:144)
> at
>
> org.apache.spark.examples.ml.MyJavaLogisticRegression.init(JavaDeveloperApiExample.java:139)
> at
>
> org.apache.spark.examples.ml.MyJavaLogisticRegression.<init>(JavaDeveloperApiExample.java:111)
> at
>
> org.apache.spark.examples.ml.JavaDeveloperApiExample.main(JavaDeveloperApiExample.java:68)
>
>
>
> --
> View this message in context:
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-tp17710.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
> For additional commands, e-mail: dev-h...@spark.apache.org
>
>
