paleolimbot opened a new issue, #18201: URL: https://github.com/apache/datafusion/issues/18201
### Is your feature request related to a problem or challenge? In the FFI_ScalarUDF, I noticed that the Scalar/Array nature of an argument is not preserved. This is not a bug, as scalar values are correctly repeated to the number of rows, but it does make the FFI interface of limited value for what I'd like to do with it (because some geometry predicates are significantly faster if we "prepare" a scalar value once per batch instead of once per row). ### Describe the solution you'd like I think we can use a comparison between the `num_rows` argument and length of the wrapper array to decide if an argument is a scalar or an array (with a corner case when there's `num_rows` of 1). This would break the FFI on the "new function producer but old function caller" end although I don't think it would segfault anything if it's of interest. ### Describe alternatives you've considered Our own FFI (or just building everything at once) ### Additional context https://github.com/apache/datafusion/blob/b5b7f9b356a5363a71ee1d293df199c9e33206cd/datafusion/ffi/src/udf/mod.rs#L185-L191 https://github.com/apache/datafusion/blob/b5b7f9b356a5363a71ee1d293df199c9e33206cd/datafusion/ffi/src/udf/mod.rs#L389-L391 PR where I discovered this: https://github.com/apache/sedona-db/pull/228 cc @kylebarron who may run across this issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
