HI, I want to develop a library with custom Estimator / Transformers for spark. So far not a lot of documentation could be found but http://stackoverflow.com/questions/37270446/how-to-roll-a-custom-estimator-in-pyspark-mllib
Suggest that: Generally speaking, there is no documentation because as for Spark 1.6 / 2.0 most of the related API is not intended to be public. It should change in Spark 2.1.0 (see SPARK-7146 <https://issues.apache.org/jira/browse/SPARK-7146>). Where can I already find documentation today? Is it true that my library would require residing in Sparks`s namespace similar to https://github.com/collectivemedia/spark-ext to utilize all the handy functionality? Kind Regards, Georg
