[
https://issues.apache.org/jira/browse/SPARK-60174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anna Liang updated SPARK-60174:
-------------------------------
Description:
`worker_run` in python/pyspark/sql/worker/utils.py suppresses a mypy
`union-attr` error on `sys.modules["__main__"]._{_}spec{_}_.name` with
`# type: ignore[union-attr]`, because `_{_}spec{_}_` is typed as
`ModuleSpec | None`.
Narrow the type explicitly by storing `_{_}spec{_}_` in a local variable and
checking `is not None`, then remove the `type: ignore`. Behavior is unchanged.
was:
`worker_run` in python/pyspark/sql/worker/utils.py suppresses a mypy
`union-attr` error on `sys.modules["__main__"].__spec__.name` with
`# type: ignore[union-attr]`, because `__spec__` is typed as
`ModuleSpec | None`.
Narrow the type explicitly by storing `__spec__` in a local variable and
checking `is not None`, then remove the `type: ignore`. Behavior is unchanged.
> Remove type: ignore[union-attr] in pyspark.sql.worker.utils
> -----------------------------------------------------------
>
> Key: SPARK-60174
> URL: https://issues.apache.org/jira/browse/SPARK-60174
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
> Affects Versions: 5.0.0
> Reporter: Anna Liang
> Priority: Minor
> Labels: pull-request-available
>
> `worker_run` in python/pyspark/sql/worker/utils.py suppresses a mypy
> `union-attr` error on `sys.modules["__main__"]._{_}spec{_}_.name` with
> `# type: ignore[union-attr]`, because `_{_}spec{_}_` is typed as
> `ModuleSpec | None`.
> Narrow the type explicitly by storing `_{_}spec{_}_` in a local variable and
> checking `is not None`, then remove the `type: ignore`. Behavior is unchanged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
