GitHub user coderfender created a discussion: [datafusion-spark] Add physical implementations for functions that only have simplify()
Several functions in `datafusion-spark` currently only implement simplify() and throw errors in invoke_with_args(). This causes issues for downstream projects like Comet that convert physical plans directly without going through DataFusion's logical optimizer This discussion is to see if we could get consensus on the approach since the functions should both support direct datafusion use cases while also providing easy access to downstream projects like comet / lakesail etc Some open questions worth discussion 1. Should we keep simplify() alongside invoke_with_args() for optimization benefits when the logical optimizer does run? 2. Concerns about duplicating logic between simplify() and invoke_with_args()? Related issue: #21959 GitHub link: https://github.com/apache/datafusion/discussions/22012 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
