Holden Karau created SPARK-60190:
------------------------------------
Summary: Pass the callable (`func`) to `_transpile_from_ast`
Key: SPARK-60190
URL: https://issues.apache.org/jira/browse/SPARK-60190
Project: Spark
Issue Type: Sub-task
Components: PySpark
Affects Versions: 4.4.0
Reporter: Holden Karau
For an out of tree transpiler it is missing the callable:
`func_from_spark_stack()` walks `sys._getframe`
looking for `_transpile_func`'s `func` local (`keys.py:213`) because nothing
else hands it over, and the variety needs it for capture extraction, canonical
source, and fingerprinting. `_transpile_func` has `func` in hand; adding it to
the hook signature deletes the frame walk. (The sampling wrap that mutates the
UDF instance's `.func` mid-construction is a further reach -- passing `func`
covers reading it; blessing instance mutation is a separate question, left to
the consumer's own devices for now.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
