Hi, I wanted to highlight the usefulness of now closed (unmerged) PR `[SPARK-49008][PYTHON] Use ParamSpec to propagate func signature in transform` - https://github.com/apache/spark/pull/47493. This change would add type-checking for the DataFrame `transform` method in PySpark using Python ParamSpec.
Personally, explicit mypy errors for that specific method would save me some time on multiple occasions, and I am confident others in the community would benefit from it as well. As I understand it, despite ParamSpec being introduced in Python 3.10, the change maintains compatibility with all PySpark supported Python versions. Is there a path to reopen and review the PR please? Best regards - Rafal
