Re: Re: [DISCUSS] Donation of a User-Defined Function Framework for Apache Arrow

To be clear, if the arrow community thinks this would be better organized / administered in the Apache DataFusion project (especially if it is aligned with Rust) I think it would be good to discuss donating there On Fri, Jun 28, 2024 at 3:17 PM Andrew Lamb wrote: > I think there are two aspects:

Re: Re: [DISCUSS] Donation of a User-Defined Function Framework for Apache Arrow

I think there are two aspects: 1. The actual mechanics of implementing functions 2. The actual library of udf functions (e.g. sin, cos, nullif, etc) I agree 2 is not something that belongs naturally in the arrow project and is better aligned with query engines However I think 1 is worth consideri

RE: Re: [DISCUSS] Donation of a User-Defined Function Framework for Apache Arrow

I wonder if the DataFusion project might be a more natural home for this functionality? UDFs are more of a query engine concept, whereas arrow-rs is more focused on purely physical execution? On 28 June 2024 19:41:39 BST, Runji Wang wrote: >Hi Felipe, > >Vectorization will be applied whenever p

RE: Re: [DISCUSS] Donation of a User-Defined Function Framework for Apache Arrow

Hi Felipe, Vectorization will be applied whenever possible. When all input and output types of a function are primitive (int16, int32, int64, float32, float64) and do not involve any Option or Result, the macro will automatically generate code based on unary

RE: Re: [DISCUSS] Donation of a User-Defined Function Framework for Apache Arrow

Hi All, I am the initiator of this project. Thanks Xuanwo for helping to promote it and start this discussion. Regarding the location of the code, I prefer to keep everything in the same repository rather than spreading it across various language binding libraries. The current implementations