Re: build package xrayutilities - wheel and pip with setuptools
Hello, I juste added PYBUILD_SYSTEM=distutils
so now pybuild use the setup.py file. (at least this is a workaround :).
now it failes with this error
/usr/bin/python3.10: No module named pip
so I am wondering if pip should not be added to the setuptools dependency ?
I will add mannualy pip in the B-D and see if it works at the end.
Cheers
Fred
dh clean --buildsystem=pybuild
dh_auto_clean -O--buildsystem=pybuild
install -d
/<>/pyfai-0.21.3\+dfsg1/debian/.debhelper/generated/_source/home
pybuild --clean -i python{version} -p 3.10
I: pybuild base:240: python3.10 setup.py clean
/<>/setup.py:43: DeprecationWarning: The distutils package is
deprecated and slated for removal in Python 3.12. Use setuptools or check PEP
632 for potential alternatives
from distutils.command.clean import clean as Clean
/usr/lib/python3/dist-packages/_distutils_hack/__init__.py:18: UserWarning:
Distutils was imported before Setuptools, but importing Setuptools also
replaces the `distutils` module in `sys.modules`. This may lead to undesirable
behaviors or errors. To avoid these issues, avoid using distutils directly,
ensure that setuptools is installed in the traditional way (e.g. not an
editable install), and/or make sure that setuptools is always imported before
distutils.
warnings.warn(
/usr/lib/python3/dist-packages/_distutils_hack/__init__.py:33: UserWarning:
Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
INFO: Disabling color, you really want to install colorlog.
INFO:pythran:Disabling color, you really want to install colorlog.
INFO:pyFAI.setup:Use setuptools with cython
INFO:pyFAI.setup:Use setuptools.setup
/usr/lib/python3/dist-packages/setuptools/installer.py:27:
SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements
should be satisfied by a PEP 517 installer.
warnings.warn(
/usr/lib/python3/dist-packages/setuptools/installer.py:34: UserWarning: Unbuilt
egg for pyFAI [unknown version] (/<>)
pkg_resources.get_distribution('wheel')
WARNING: The wheel package is not available.
/usr/lib/python3/dist-packages/setuptools/installer.py:60: UserWarning: Unbuilt
egg for pyFAI [unknown version] (/<>)
environment = pkg_resources.Environment()
/usr/bin/python3.10: No module named pip
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/setuptools/installer.py", line 82, in
fetch_build_egg
subprocess.check_call(cmd)
File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3.10', '-m', 'pip',
'--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmppdpybvy1',
'--quiet', 'setuptools<60.0.0']' returned non-zero exit status 1.
- Mail d’origine -
De: Scott Kitterman
À: debian-python@lists.debian.org
Envoyé: Tue, 01 Nov 2022 20:36:20 +0100 (CET)
Objet: Re: build package xrayutilities - wheel and pip with setuptools
On Tuesday, November 1, 2022 3:31:47 PM EDT PICCA Frederic-Emmanuel wrote:
> > I don't think it should do that, so we need to investigate.
Where
> > can I find the updated packaging?
>
> I did not push the change right now, I will push once I solve this issue
:).
>
> my opinion is that I should force via PYBUILD_SYSTEM=distutils
>
> Fred
If that works, I think it's fine, but I don't think it should be
necessary.
Let me know once you push the package and I'll see if there's a
pybuild issue.
Scott K
Re: build package xrayutilities - wheel and pip with setuptools
Ok, I understand better what is going on.
in pyproject.toml there is these lines.
[build-system]
requires = [
"wheel",
"setuptools<60.0.0",
"oldest-supported-numpy",
"scipy",
"sphinx",
"nbsphinx",
"silx>=0.10",
"Cython>=0.25"
]
indeed the setuptools < 60, can not be fullfill, So I think that this is why it
try to use pip in order to install the required version of setuptools.
the strack trace seems to config this.
dh clean --buildsystem=pybuild
dh_auto_clean -O--buildsystem=pybuild
install -d
/<>/pyfai-0.21.3\+dfsg1/debian/.debhelper/generated/_source/home
pybuild --clean -i python{version} -p 3.10
I: pybuild base:240: python3.10 setup.py clean
/<>/setup.py:43: DeprecationWarning: The distutils package is
deprecated and slated for removal in Python 3.12. Use setuptools or check PEP
632 for potential alternatives
from distutils.command.clean import clean as Clean
/usr/lib/python3/dist-packages/_distutils_hack/__init__.py:18: UserWarning:
Distutils was imported before Setuptools, but importing Setuptools also
replaces the `distutils` module in `sys.modules`. This may lead to undesirable
behaviors or errors. To avoid these issues, avoid using distutils directly,
ensure that setuptools is installed in the traditional way (e.g. not an
editable install), and/or make sure that setuptools is always imported before
distutils.
warnings.warn(
/usr/lib/python3/dist-packages/_distutils_hack/__init__.py:33: UserWarning:
Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
INFO: Disabling color, you really want to install colorlog.
INFO:pythran:Disabling color, you really want to install colorlog.
INFO:pyFAI.setup:Use setuptools with cython
INFO:pyFAI.setup:Use setuptools.setup
/usr/lib/python3/dist-packages/setuptools/installer.py:27:
SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements
should be satisfied by a PEP 517 installer.
warnings.warn(
/usr/lib/python3/dist-packages/setuptools/installer.py:60: UserWarning: Unbuilt
egg for pyFAI [unknown version] (/<>)
environment = pkg_resources.Environment()
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None,
status=None)) after connection broken by 'ProxyError('Cannot connect to
proxy.', NewConnectionError(': Failed to establish a new connection: [Errno 111]
Connection refused'))': /simple/setuptools/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None,
status=None)) after connection broken by 'ProxyError('Cannot connect to
proxy.', NewConnectionError(': Failed to establish a new connection: [Errno 111]
Connection refused'))': /simple/setuptools/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None,
status=None)) after connection broken by 'ProxyError('Cannot connect to
proxy.', NewConnectionError(': Failed to establish a new connection: [Errno 111]
Connection refused'))': /simple/setuptools/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None,
status=None)) after connection broken by 'ProxyError('Cannot connect to
proxy.', NewConnectionError(': Failed to establish a new connection: [Errno 111]
Connection refused'))': /simple/setuptools/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None,
status=None)) after connection broken by 'ProxyError('Cannot connect to
proxy.', NewConnectionError(': Failed to establish a new connection: [Errno 111]
Connection refused'))': /simple/setuptools/
ERROR: Could not find a version that satisfies the requirement
setuptools<60.0.0 (from versions: none)
ERROR: No matching distribution found for setuptools<60.0.0
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/setuptools/installer.py", line 82, in
fetch_build_egg
subprocess.check_call(cmd)
File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3.10', '-m', 'pip',
'--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpmx8yh9o_',
'--quiet', 'setuptools<60.0.0']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/<>/setup.py", line 1137, in
setup_package()
File "/<>/setup.py", line 1133, in setup_package
setup(**setup_kwargs)
File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 86, in
setup
_install_setup_requires(attrs)
File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 80, in
_install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 875, in
fetch_build_eggs
resolved_dists = pkg_resources.working_set.resolve(
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 789, in
resolve
dist = best[req.key] = env.best_match(
File "/usr/lib/py
Re: pyyaml 6
On 19/10/2022 22:30, Gordon Ball wrote: On 09/10/2022 21:39, Gordon Ball wrote: On 07/10/2022 01:13, Timo Röhling wrote: Hi Gordon, * Gordon Ball [2022-10-07 00:10]: * Upload to unstable and see what breaks? * Request an archive rebuild with this version and see what breaks? * File bugs against all likely affected packages with a fixed date for an upload? * Wait until after the freeze? Considering that PyYAML has been issuing a YAMLLoadWarning since version 5.1 (i.e. September 2019), I would expect that many (most?) reverse dependencies have been fixed, and anything that still breaks is probably in dire need of maintenance anyway. Yes, that's a good point. I've done some more investigation of reverse-depends and looking for likely bad patterns with codesearch.d.n, and the set of packages which A) appear to contain plain `yaml.load(f)` or equivalent, where `yaml` is pyyaml and B) actually depend or build-depend on `python3-yaml` is smaller than I feared. This is what I came up with: ``` ansible # confirm - in ansible_collections/community/general/plugins buildbot # confirm - in porttostable ceph # confirm, in civetweb/third_party/duktape, src/test/crimson docker-compose # confirm, in tests dose3 # confirm, in scripts elasticsearch-curator # confirm, in test/unit etm # confirm, in etmTk/data ganeti # confirm, in qa, test/py gnocchi # confirm, in gnocchi/gendoc gr-dab # confirm, in python/app jeepyb # confirm, in cmd/notify_impact knitpy # confirm, in knitpy labgrid # confirm, in remote/coordinator lecm # confirm, in lecm/configuration lirc # confirm, in lirc/database lqa # confirm, in lqa_api/job open-adventure # confirm, in tests owslib # confirm, in ogcapi policyd-rate-limit # confirm, in config, tests python-aptly # confirm, in publisher python-canmatrix # confirm, in canmatrix/formats/yaml python-multipart # confirm, in multipart/tests python-pybedtools # confirm, in test, contrib python-seqcluster # confirm, in install python-tempestconf # confirm, in config_tempest/profile qcat # confirm, in adapters qcengine # confirm, in config refstack-client # confirm, in refstack_client relatorio # confirm, in templates/chart ripe-atlas-tools # confirm, in tools/settings ros-genpy # confirm, in test ros-rosinstall # confirm, in test, src/rosinstall ros-rosinstall-generator # confirm, in test, src/rosinstall_generator spades # confirm, in assembler/src/test/teamcity, assembler/src/projects/mts, assembler/src/spades_pipeline xrstools # confirm, in WIZARD zlmdb # confirm, in _database ``` I don't know if all these codepaths are actually ever reached, or tests ever run, so the packages won't necessarily break (or the breakage is very minor, such as a single community plugin in ansible). I don't guarantee there are no omissions from the list though. There is a significantly longer list of packages which appear to contain a use of yaml.load _somewhere_, but it is usually in some maintainer/release script, or an optional path somewhere, and the package itself doesn't [build-]depend on python3-yaml. I've filed bugs for (most) of the packages listed above (some were dropped on review because they weren't in main, or the affected file appeared to be unused at build time and not installed). I've had a couple of acknowledgements/fixes already. A number of affected package look very sparsely used and/or maintained, and I doubt are likely to be fixed. However, of the packages above, only two have autopkgtests which fail on experimental migration, so I wouldn't be surprised if some are not already broken by other past changes. I propose to wait ~2 weeks until the beginning of November and upload pyyaml 6 to unstable then. pyyaml 6 has now landed in unstable. About half the bugs I filed have been resolved, and there is only one package (ganeti) with autopkgtests blocking migration.
Re: build package xrayutilities - wheel and pip with setuptools
The upstream was right :), their build system is completly broken with
setuptools > 60
I need to fix this.
thanks for your help
Fred
Traceback (most recent call last):
File "/<>/setup.py", line 1137, in
setup_package()
File "/<>/setup.py", line 1133, in setup_package
setup(**setup_kwargs)
File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 87, in
setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line
172, in setup
ok = dist.parse_command_line()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line
474, in parse_command_line
args = self._parse_command_opts(parser, args)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1107, in
_parse_command_opts
nargs = _Distribution._parse_command_opts(self, parser, args)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line
540, in _parse_command_opts
raise DistutilsClassError(
distutils.errors.DistutilsClassError: command class must subclass Command
E: pybuild pybuild:379: clean: plugin distutils failed with: exit code=1:
python3.10 setup.py clean
dh_auto_clean: error: pybuild --clean -i python{version} -p 3.10 returned exit
code 13
make: *** [debian/rules:16: clean] Error 25
dpkg-buildpackage: error: fakeroot debian/rules clean subprocess returned exit
status 2
- Mail d’origine -
De: PICCA Frederic-Emmanuel
À: Scott Kitterman
Cc: debian-python@lists.debian.org
Envoyé: Wed, 02 Nov 2022 09:03:45 +0100 (CET)
Objet: Re: build package xrayutilities - wheel and pip with setuptools
Ok, I understand better what is going on.
in pyproject.toml there is these lines.
[build-system]
requires = [
"wheel",
"setuptools<60.0.0",
"oldest-supported-numpy",
"scipy",
"sphinx",
"nbsphinx",
"silx>=0.10",
"Cython>=0.25"
]
indeed the setuptools < 60, can not be fullfill, So I think that this is why
it try to use pip in order to install the required version of setuptools.
the strack trace seems to config this.
dh clean --buildsystem=pybuild
dh_auto_clean -O--buildsystem=pybuild
install -d
/<>/pyfai-0.21.3\+dfsg1/debian/.debhelper/generated/_source/home
pybuild --clean -i python{version} -p 3.10
I: pybuild base:240: python3.10 setup.py clean
/<>/setup.py:43: DeprecationWarning: The distutils
package is deprecated and slated for removal in Python 3.12. Use setuptools or
check PEP 632 for potential alternatives
from distutils.command.clean import clean as Clean
/usr/lib/python3/dist-packages/_distutils_hack/__init__.py:18: UserWarning:
Distutils was imported before Setuptools, but importing Setuptools also
replaces the `distutils` module in `sys.modules`. This may lead to undesirable
behaviors or errors. To avoid these issues, avoid using distutils directly,
ensure that setuptools is installed in the traditional way (e.g. not an
editable install), and/or make sure that setuptools is always imported before
distutils.
warnings.warn(
/usr/lib/python3/dist-packages/_distutils_hack/__init__.py:33: UserWarning:
Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
INFO: Disabling color, you really want to install colorlog.
INFO:pythran:Disabling color, you really want to install colorlog.
INFO:pyFAI.setup:Use setuptools with cython
INFO:pyFAI.setup:Use setuptools.setup
/usr/lib/python3/dist-packages/setuptools/installer.py:27:
SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements
should be satisfied by a PEP 517 installer.
warnings.warn(
/usr/lib/python3/dist-packages/setuptools/installer.py:60: UserWarning: Unbuilt
egg for pyFAI [unknown version] (/<>)
environment = pkg_resources.Environment()
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None,
status=None)) after connection broken by 'ProxyError('Cannot connect
to proxy.',
NewConnectionError(': Failed to establish a new connection: [Errno 111]
Connection refused'))': /simple/setuptools/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None,
status=None)) after connection broken by 'ProxyError('Cannot connect
to proxy.',
NewConnectionError(': Failed to establish a new connection: [Errno 111]
Connection refused'))': /simple/setuptools/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None,
status=None)) after connection broken by 'ProxyError('Cannot connect
to proxy.',
NewConnectionError(': Failed to establish a new connection: [Errno 111]
Connection refused'))': /simple/setuptools/
WARNING: Retrying (Retry(total=1, connect=
Re: New upstream version of sphinxext-opengraph 0.7.0
On Tue, 1 Nov 2022 12:33:59 -1000 Chiara Marmo wrote: > Dear list, > I have committed to salsa the new version of sphinxext-opengraph > [1]. If someone could have a look, that would be great. > Thanks! Looks like there's a newer upstream release, fixing bugs in 0.7.0? pgp8WgYblHIhF.pgp Description: OpenPGP digital signature
Re: build package xrayutilities - wheel and pip with setuptools
Hi Fred, Hi all other DD, I am the upstream author of pyFAI and fabio, I wish to thank you for taking care of the packaging of those software. I am aware there is an issue with the usage of those software since they use numpy.distutils (deprecated, to be removed in python3.12 and requires setuptools<60). I am working on a different packaging which uses meson-python (same as scipy>=1.9). The migration of fabio is more-or-less ready (see this branch: https://github.com/kif/fabio/tree/meson) and I am just waiting for a new release of meson-python (0.11) to make a new release (without setup.py). Unlike fabio which is ready for release, pyFAI will be released either in December or early next year. On the debian side, I don't know how far is the integration of meson-python ? https://salsa.debian.org/python-team/packages/meson-python To me, it is of crucial importance if one wants an updated version of scipy to be available in debian12. Fabio and pyFAI will follow without foreseeable issue. I am interested in the way the man-page should be produced since this will have to be redone in both those packages. Any advice on this is welcome. Best regards, -- Jérôme Kieffer tel +33 476 882 445
Re: pyyaml 6
On November 2, 2022 8:35:35 AM UTC, Gordon Ball wrote: >On 19/10/2022 22:30, Gordon Ball wrote: >> On 09/10/2022 21:39, Gordon Ball wrote: >>> On 07/10/2022 01:13, Timo Röhling wrote: Hi Gordon, * Gordon Ball [2022-10-07 00:10]: > * Upload to unstable and see what breaks? > * Request an archive rebuild with this version and see what breaks? > * File bugs against all likely affected packages with a fixed date for > an upload? > * Wait until after the freeze? Considering that PyYAML has been issuing a YAMLLoadWarning since version 5.1 (i.e. September 2019), I would expect that many (most?) reverse dependencies have been fixed, and anything that still breaks is probably in dire need of maintenance anyway. >>> >>> Yes, that's a good point. >>> >>> I've done some more investigation of reverse-depends and looking for >>> likely bad patterns with codesearch.d.n, and the set of packages which >>> A) appear to contain plain `yaml.load(f)` or equivalent, where `yaml` is >>> pyyaml and B) actually depend or build-depend on `python3-yaml` is >>> smaller than I feared. This is what I came up with: >>> >>> ``` >>> ansible # confirm - in ansible_collections/community/general/plugins >>> buildbot # confirm - in porttostable >>> ceph # confirm, in civetweb/third_party/duktape, src/test/crimson >>> docker-compose # confirm, in tests >>> dose3 # confirm, in scripts >>> elasticsearch-curator # confirm, in test/unit >>> etm # confirm, in etmTk/data >>> ganeti # confirm, in qa, test/py >>> gnocchi # confirm, in gnocchi/gendoc >>> gr-dab # confirm, in python/app >>> jeepyb # confirm, in cmd/notify_impact >>> knitpy # confirm, in knitpy >>> labgrid # confirm, in remote/coordinator >>> lecm # confirm, in lecm/configuration >>> lirc # confirm, in lirc/database >>> lqa # confirm, in lqa_api/job >>> open-adventure # confirm, in tests >>> owslib # confirm, in ogcapi >>> policyd-rate-limit # confirm, in config, tests >>> python-aptly # confirm, in publisher >>> python-canmatrix # confirm, in canmatrix/formats/yaml >>> python-multipart # confirm, in multipart/tests >>> python-pybedtools # confirm, in test, contrib >>> python-seqcluster # confirm, in install >>> python-tempestconf # confirm, in config_tempest/profile >>> qcat # confirm, in adapters >>> qcengine # confirm, in config >>> refstack-client # confirm, in refstack_client >>> relatorio # confirm, in templates/chart >>> ripe-atlas-tools # confirm, in tools/settings >>> ros-genpy # confirm, in test >>> ros-rosinstall # confirm, in test, src/rosinstall >>> ros-rosinstall-generator # confirm, in test, src/rosinstall_generator >>> spades # confirm, in assembler/src/test/teamcity, >>> assembler/src/projects/mts, assembler/src/spades_pipeline >>> xrstools # confirm, in WIZARD >>> zlmdb # confirm, in _database >>> ``` >>> >>> I don't know if all these codepaths are actually ever reached, or tests >>> ever run, so the packages won't necessarily break (or the breakage is >>> very minor, such as a single community plugin in ansible). I don't >>> guarantee there are no omissions from the list though. >>> >>> There is a significantly longer list of packages which appear to contain >>> a use of yaml.load _somewhere_, but it is usually in some >>> maintainer/release script, or an optional path somewhere, and the >>> package itself doesn't [build-]depend on python3-yaml. >>> >> >> I've filed bugs for (most) of the packages listed above (some were dropped >> on review because they weren't in main, or the affected file appeared to be >> unused at build time and not installed). I've had a couple of >> acknowledgements/fixes already. >> >> A number of affected package look very sparsely used and/or maintained, and >> I doubt are likely to be fixed. However, of the packages above, only two >> have autopkgtests which fail on experimental migration, so I wouldn't be >> surprised if some are not already broken by other past changes. >> >> I propose to wait ~2 weeks until the beginning of November and upload pyyaml >> 6 to unstable then. >> > >pyyaml 6 has now landed in unstable. About half the bugs I filed have been >resolved, and there is only one package (ganeti) with autopkgtests blocking >migration. > There were two more, but I retried the migration reference jobs and they failed (which makes them no longer block since they are no longer a regression). Ganeti can't currently be fixed since it's unbuildable. It's only due to an archive hiccup that it's even in Testing. I was planning on asking the Release Team to ignore it, so pyyaml can transition (worst case it's stuck until ganeti gets removed from Testing in a week and a half). Scott K
Join python team
Hi! I'd like to join the python team and package some libraries (e.g. https://github.com/esphome/aioesphomeapi) and their dependencies for Debian. I'm a DM and my Salsa username is obbardc. I have read the policy at https://salsa.debian.org/python-team/tools/python-modules/blob/master/policy.rst and accept it in its entirety. Thank you for your consideration Christopher Obbard
Re: build package xrayutilities - wheel and pip with setuptools
Hello, just for info. I can confirm that the probleme was in the upstream build system, which was not compatible with the setuptools > 60. After patching the build system, I can confirm that without any change to the rules files. It works out of the box. So there is no issue in pybuild :)), or at least nothing new :) cheers Frederic
Re: New upstream version of sphinxext-opengraph 0.7.0
> > Looks like there's a newer upstream release, fixing bugs in 0.7.0? > Yep. I will package it and commit today. Best Chiara
Re: New upstream version of sphinxext-opengraph 0.7.0
> > > > Looks like there's a newer upstream release, fixing bugs in 0.7.0? > > > > Yep. I will package it and commit today. > > Done. Best, Chiara
