timsaucer commented on issue #14317: URL: https://github.com/apache/datafusion/issues/14317#issuecomment-2618893233
We have something similar in `datafusion-python` [here](https://github.com/apache/datafusion-python/blob/main/python/datafusion/dataframe.py#L835). It lets you do something like what you're describing but it operates on a dataframe and not a logical plan. In python you can currently do something like this (this is from our unit tests) ```python def add_string_col(df_internal) -> DataFrame: return df_internal.with_column("string_col", literal("string data")) def add_with_parameter(df_internal, value: Any) -> DataFrame: return df_internal.with_column("new_col", literal(value)) df = df.transform(add_string_col).transform(add_with_parameter, 3) ``` If you had a DataFrame `df` then this would add in two more columns. It's a trivial example, but it allows for some really nice chaining of operations like in the last line. I would definitely support adding something like that on the rust side. But what I'm doing here doesn't exactly match up with what the issue requests, which is to work on the LogicalPlan. So we could do something similar but `df.transform_plan` that will do the conversion from dataframe to plan, execute the function/closure, and convert back to dataframe. @phisn would that meet your needs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
