Hi All, For the use case where the expensive UDF has constant inputs (literals) we have proposed the following JIRA and PR which calculates the UDF only once in the driver: https://issues.apache.org/jira/browse/SPARK-27692 https://github.com/apache/spark/pull/24593
If considering revisiting the optimization for UDFs the above PR can be considered as well. Thanks, Guy -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
