hvanhovell opened a new pull request, #49785: URL: https://github.com/apache/spark/pull/49785
### What changes were proposed in this pull request? This PR adds support for `UserDefinedAggregateFunction` to the Spark Connect Scala Client. While this is a deprecated feature, we still believe it is useful to support it to ensure we reduce incompatibilities between classic and connect. Implementation wise I opted to convert the `UserDefinedAggregateFunction` to an `Aggregator`, and use that code path for execution. This is probably not as fast as the original implementation (more allocations). ### Why are the changes needed? This reduces friction between the classic and connect implementations. ### Does this PR introduce _any_ user-facing change? Yes. It enabled Spark Connect Scala Client users to use `UserDefinedAggregateFunction`s. ### How was this patch tested? Added tests to ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
