Hello Dear Spark User / Dev, I would like to pass Python user defined function to Spark Job developed using Scala and return value of that function would be returned to DF / Dataset API.
Can someone please guide me, which would be best approach to do this. Python function would be mostly transformation function. Also would like to pass Java Function as a String to Spark / Scala job and it applies to RDD / Data Frame and should return RDD / Data Frame. Thank you.
