[
https://issues.apache.org/jira/browse/SPARK-58244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tian Gao reassigned SPARK-58244:
--------------------------------
Assignee: Tian Gao
> Remove type: ignore[name-defined] in pyspark.sql
> ------------------------------------------------
>
> Key: SPARK-58244
> URL: https://issues.apache.org/jira/browse/SPARK-58244
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
> Affects Versions: 4.3.0
> Reporter: Haotian Sun
> Assignee: Tian Gao
> Priority: Minor
> Labels: pull-request-available
>
> 1. Some forward-reference annotations in pyspark.sql reference a type only
> imported inside a function body to avoid a circular import, so it is
> unresolvable to the type checker. Adding it to the module's if TYPE_CHECKING:
> block resolves the reference.
> 2. In the _test() doctest runners of streaming/query.py and
> streaming/listener.py, the except Py4JError: fallback calls SparkSession(sc)
> but never defines sc. The _test() helpers in readwriter.py define sc =
> SparkContext("local[4]", "PythonTest") before the try block. Applying the
> same fixes it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
