This would also probably improve performance:
https://github.com/apache/spark/pull/9565
On Fri, Apr 15, 2016 at 8:44 AM, Hamel Kothari
wrote:
> Hi all,
>
> So we have these UDFs which take <1ms to operate and we're seeing pretty
> poor performance around them in practice, the overhead being >10m
Hi all,
So we have these UDFs which take <1ms to operate and we're seeing pretty
poor performance around them in practice, the overhead being >10ms for the
projections (this data is deeply nested with ArrayTypes and MapTypes so
that could be the cause). Looking at the logs and code for ScalaUDF, I
