gstvg commented on issue #14205: URL: https://github.com/apache/datafusion/issues/14205#issuecomment-2621052052
Hi @rkrishn7, have you already started any work? Since creating this issue, I’ve been working on a PoC to refine the proposal, and I managed to get it working. I’d like to continue with it if you’re OK with that—I probably should have assigned the issue to myself earlier, my bad 🤦🏻♂️! Anyway, many thanks for your interest and for your questionings, and I would really appreciate your input in futures questions (I expect many to arise!) > How do implementers define a signature for a UDF that takes one or more lambdas? Do we just skip type checking of lambda arguments? How does this affect the ordering? I added a new [ScalarUDF method invoke_with_lambda_args](https://github.com/apache/datafusion/compare/main...gstvg:arrow-datafusion:lambda#diff-ce24057bb984fe97516c33a9b1e690e20c85a5943503caa65db40d78ca3fdc7aR128) that receives ScalarFunctionArgs<ColumnarValueOrLambda> instead of ScalarFunctionArgs<ColumnarValue>, and has a [default implementation](https://github.com/apache/datafusion/compare/main...gstvg:arrow-datafusion:lambda#diff-8a327db2db945bcf6ca2b4229885532feae127e94a450600d3fac6ecdc0eeb3fR702) calling invoke_with_args, returning an error if any arg is a lambda This should preserve ordering of any arg. But maybe we could add an `lambda_arguments` to `ScalarFunctionArgs` similtar to [here](https://github.com/apache/datafusion/compare/main...gstvg:arrow-datafusion:lambda#diff-8a327db2db945bcf6ca2b4229885532feae127e94a450600d3fac6ecdc0eeb3fR413), what do you think ? > How should one map lambda function arguments to concrete values in the body? If the function only operates on a list, this becomes easier to express. The lambda implementation should [build a RecordBatch](https://github.com/apache/datafusion/compare/main...gstvg:arrow-datafusion:lambda#diff-ce24057bb984fe97516c33a9b1e690e20c85a5943503caa65db40d78ca3fdc7aR147), likely using some other concrete argument, and pass it to the lambda [PhysicalExpr evaluate](https://github.com/apache/datafusion/compare/main...gstvg:arrow-datafusion:lambda#diff-ce24057bb984fe97516c33a9b1e690e20c85a5943503caa65db40d78ca3fdc7aR147) method That said, if you have any thoughts on the [current interface](https://github.com/apache/datafusion/compare/main...gstvg:arrow-datafusion:lambda#diff-ce24057bb984fe97516c33a9b1e690e20c85a5943503caa65db40d78ca3fdc7aR128), I’d love to hear your feedback. It's a `list_map([1,2], x -> x*2)` udf, you can see the [executed output here](https://github.com/apache/datafusion/compare/main...gstvg:arrow-datafusion:lambda#diff-53634af9d48017f60cb545dfe2aaab20d1f7fa3435947c161ef22515fa62ed8dR26). Thanks! 🙏 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
