Shujing Yang created SPARK-53378:
------------------------------------
Summary: [PYTHON] Enable @udtf decorator to support returnType as
first positional argument
Key: SPARK-53378
URL: https://issues.apache.org/jira/browse/SPARK-53378
Project: Spark
Issue Type: New Feature
Components: PySpark
Affects Versions: 4.1.0
Reporter: Shujing Yang
Currently, the @udtf decorator only supports returnType as a keyword-only
argument. This PR enables @udtf to accept returnType as the first positional
argument.
```
# Current - only keyword argument supported
@udtf(returnType="c1: int, c2: string")
class MyUDTF:
def eval(self, x):
yield x, x + 1
# Current - bare decorator only works with analyze method
@udtf
class MyUDTF:
@staticmethod
def analyze():
return AnalyzeResult(StructType().add("c1", IntegerType()))
def eval(self, x):
yield x
After This Change
# New - positional string schema
@udtf("c1: int, c2: string")
class MyUDTF:
def eval(self, x):
yield x, x + 1
# New - positional StructType
@udtf(StructType().add("c1", IntegerType()).add("c2", IntegerType()))
class MyUDTF:
def eval(self, x):
yield x, x + 1
```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
