[
https://issues.apache.org/jira/browse/SPARK-51595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Macagno updated SPARK-51595:
--------------------------------------
Summary: pyspark pip install fails with latest setuptools (78.0.1) (was:
pyspark pip install fails with latest setuptools (78.x))
> pyspark pip install fails with latest setuptools (78.0.1)
> ---------------------------------------------------------
>
> Key: SPARK-51595
> URL: https://issues.apache.org/jira/browse/SPARK-51595
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 3.5.5
> Environment: pyenv virtualenv and docker-based python both ran into
> same error.
> Reporter: Francesco Macagno
> Priority: Minor
>
> setuptools 78.0.1 broke installing pyspark by no longer allowing
> `description-file` in setup.cfg. Thankfully, they have since reverted the
> change, but it seems odd that pyspark still has that issue given it seemed to
> have been fixed a while ago.
>
> ```
> Collecting pyspark==3.5.5 (from -r requirements.txt (line 18))
> Using cached pyspark-3.5.5.tar.gz (317.2 MB)
> Running command pip subprocess to install build dependencies
> Using pip 25.0.1 from
> /Users/fmacagno/.pyenv/versions/3.9.21/envs/airflow/lib/python3.9/site-packages/pip
> (python 3.9)
> Collecting setuptools>=40.8.0
> Obtaining dependency information for setuptools>=40.8.0 from
> https://files.pythonhosted.org/packages/42/c8/3faed884acdb2c1f2eb353cbacdd1ee4943de89a199d1f622ebefb6170e5/setuptools-78.0.1-py3-none-any.whl.metadata
> Using cached setuptools-78.0.1-py3-none-any.whl.metadata (6.6 kB)
> Using cached setuptools-78.0.1-py3-none-any.whl (1.3 MB)
> Installing collected packages: setuptools
> Successfully installed setuptools-78.0.1
> Installing build dependencies ... done
> Running command Getting requirements to build wheel
> Traceback (most recent call last):
> File
> "/Users/fmacagno/.pyenv/versions/3.9.21/envs/airflow/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py",
> line 389, in <module>
> main()
> File
> "/Users/fmacagno/.pyenv/versions/3.9.21/envs/airflow/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py",
> line 373, in main
> json_out["return_val"] = hook(**hook_input["kwargs"])
> File
> "/Users/fmacagno/.pyenv/versions/3.9.21/envs/airflow/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py",
> line 143, in get_requires_for_build_wheel
> return hook(config_settings)
> File
> "/private/var/folders/3x/02mhw__978919nbqh8z01r400000gn/T/pip-build-env-dxr75ldx/overlay/lib/python3.9/site-packages/setuptools/build_meta.py",
> line 334, in get_requires_for_build_wheel
> return self._get_build_requires(config_settings, requirements=[])
> File
> "/private/var/folders/3x/02mhw__978919nbqh8z01r400000gn/T/pip-build-env-dxr75ldx/overlay/lib/python3.9/site-packages/setuptools/build_meta.py",
> line 304, in _get_build_requires
> self.run_setup()
> File
> "/private/var/folders/3x/02mhw__978919nbqh8z01r400000gn/T/pip-build-env-dxr75ldx/overlay/lib/python3.9/site-packages/setuptools/build_meta.py",
> line 522, in run_setup
> super().run_setup(setup_script=setup_script)
> File
> "/private/var/folders/3x/02mhw__978919nbqh8z01r400000gn/T/pip-build-env-dxr75ldx/overlay/lib/python3.9/site-packages/setuptools/build_meta.py",
> line 320, in run_setup
> exec(code, locals())
> File "<string>", line 225, in <module>
> File
> "/private/var/folders/3x/02mhw__978919nbqh8z01r400000gn/T/pip-build-env-dxr75ldx/overlay/lib/python3.9/site-packages/setuptools/__init__.py",
> line 116, in setup
> _install_setup_requires(attrs)
> File
> "/private/var/folders/3x/02mhw__978919nbqh8z01r400000gn/T/pip-build-env-dxr75ldx/overlay/lib/python3.9/site-packages/setuptools/__init__.py",
> line 87, in _install_setup_requires
> dist.parse_config_files(ignore_option_errors=True)
> File
> "/private/var/folders/3x/02mhw__978919nbqh8z01r400000gn/T/pip-build-env-dxr75ldx/overlay/lib/python3.9/site-packages/setuptools/dist.py",
> line 730, in parse_config_files
> self._parse_config_files(filenames=inifiles)
> File
> "/private/var/folders/3x/02mhw__978919nbqh8z01r400000gn/T/pip-build-env-dxr75ldx/overlay/lib/python3.9/site-packages/setuptools/dist.py",
> line 599, in _parse_config_files
> opt = self._enforce_underscore(opt, section)
> File
> "/private/var/folders/3x/02mhw__978919nbqh8z01r400000gn/T/pip-build-env-dxr75ldx/overlay/lib/python3.9/site-packages/setuptools/dist.py",
> line 629, in _enforce_underscore
> raise InvalidConfigError(
> setuptools.errors.InvalidConfigError: Invalid dash-separated key
> 'description-file' in 'metadata' (setup.cfg), please use the underscore name
> 'description_file' instead.
> error: subprocess-exited-with-error
> ```
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
