I don’t think it is realistic to support codegen for UDFs. It’s hooked deep
into intervals.
On Fri, Dec 14, 2018 at 6:52 PM Matt Cheah wrote:
> How would this work with:
>
>1. Codegen – how does one generate code given a user’s UDF? Would the
>user be able to specify the code that is gen
How would this work with:
Codegen – how does one generate code given a user’s UDF? Would the user be
able to specify the code that is generated that represents their function? In
practice that’s pretty hard to get right.
Row serialization and representation – Will the UDF receive catalyst rows w
Having a way to register UDFs that are not using Hive APIs would be great!
On Fri, Dec 14, 2018 at 1:30 PM, Ryan Blue < rb...@netflix.com.invalid > wrote:
>
>
>
> Hi everyone,
> I’ve been looking into improving how users of our Spark platform register
> and use UDFs and I’d like to discuss a f
Hi everyone,
I’ve been looking into improving how users of our Spark platform register
and use UDFs and I’d like to discuss a few ideas for making this easier.
The motivation for this is the use case of defining a UDF from SparkSQL or
PySpark. We want to make it easy to write JVM UDFs and use them
I agree with the CatalystConf idea. On Fri, 14 Dec 2018
18:40:26 +0800 Wenchen Fan wrote IIRC, the reason we
did it is: `SQLConf` was in SQL core module. So we need to create methods in
`CatalystConf`, and `SQLConf` implements `CatalystConf`.Now the probl
IIRC, the reason we did it is: `SQLConf` was in SQL core module. So we need
to create methods in `CatalystConf`, and `SQLConf` implements
`CatalystConf`.
Now the problem has gone: we moved `SQLConf` to catalyst module. I think we
can remove these methods.
On Fri, Dec 14, 2018 at 3:45 PM Reynold X
