Look at https://github.com/DataSystemsLab/GeoSpark/tree/master/sql/src/main/scala/org/apache/spark/sql/geospark sql for an example.
Using custom function registration and functions residing inside sparks private namespace should work. But I am not aware of a public user facing API. Is there any I am missing? Arwin Tio <arwin....@hotmail.com> schrieb am Do. 22. Aug. 2019 04:28: > Hi friends, > > I am looking into converting some UDFs/UDAFs to Spark-Native functions to > leverage Catalyst and codegen. > > Looking through some examples (for example: > https://github.com/apache/spark/pull/7214/files for Levenshtein) it seems > like we need to add these functions to the Spark framework itself. > > Is there a way to add custom Spark-Native functions in "userspace"? > > Thank you! > > Arwin >
