Malthe Borch created SPARK-53440:
------------------------------------
Summary: Transform function to accept SQL-defined function
Key: SPARK-53440
URL: https://issues.apache.org/jira/browse/SPARK-53440
Project: Spark
Issue Type: Wish
Components: PySpark
Affects Versions: 4.0.0
Reporter: Malthe Borch
The [transform
function|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.transform.html]
takes a Python _function_-argument (unary or binary) which is then converted
into a Catalyst expression using introspection.
In some situations, it's preferable to express such functions using SQL, e.g.
`x -> x + 1` since no introspection happens, just simple parsing.
It should be possible to specify such a function as a string or using some
other construct that ultimately lets the user provide the function simply as a
string.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
