Re: cython 3.x (for Python 3.12)
On 11.12.23 08:09, Matthias Klose wrote: On 10.12.23 21:32, Andrey Rakhmatullin wrote: On Sun, Dec 10, 2023 at 09:30:03PM +0100, Andrey Rakhmatullin wrote: I find that there's also a significant issue with relying on cython3-legacy: it conflicts with cython3, meaning that it will be impossible to simultaneously install packages depending on cython3 and cython3-legacy. Once cython 3.x moves from experimental to unstable to testing and packages start depending on it, this will become a significant issue. I assume that the aim will be for everything to be ported to cython 3.x and for cython3-legacy to be dropped from testing before the trixie freeze? I wonder how many packages actually need a runtime dep on cython. I quickly checked three packages from `reverse-depends cython3` and while python3-pysph probably uses cython to generate code at the run time (not sure), python3-pyzoltan seems to only use it at the build time and python3-epimodels doesn't seem to use it at all. (I may be terribly wrong, I just thought cython is almost always a build time only dep and there are around 30 packages depending on it which was surprising) yes, many setup.py's list that under install_requires. I hope we now identified most of them, and fixed these. At least we were now able to do all the binNMUs. You could package a non-conflicting cython-legacy, however that would require more changes, also how to build it. Graham created a tracker for these, and I filed bug reports: https://release.debian.org/transitions/html/cython-runtime.html https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=cython-rt-dep&users=debian-python@lists.debian.org
Re: pandas 1.5 -> 2.1?
Hi Kingsley, On Sun, Dec 10, 2023 at 12:55:43PM -0800, Kingsley G. Morse Jr. wrote: > Hi Rebecca, Julian and all science minded pythonistas of debian, great and > small! > > I like your correspondence about upgrading from > version 1.5 of pandas to 2.1. > > It's open, scientific and explores the ideal of > proceeding wisely in a matter of public interest. > > My humble thoughts are: > > 1.) Rebecca: *Why* did you write that you'd like > to move forward with the pandas 1.5 -> 2.1 > transition? What's your reason? A thought from me on this: pandas 2.1 has many improvements over pandas 1.5. And increasingly, other packages will be requiring these new features. So why would one not want to move forward with it? > 2.) What may be the advantage of migrating to > version 3.0 of Cython? It is compatible with Python 3.12, whereas the current version of Cython in Debian (0.29.x) is not really. (For example, it has an "import imp" in it, and this breaks with Python 3.12, which has removed this deprecated module.) As Cython 0.29.x is no longer maintained upstream, having been superseded by Cython 3.x after many years of development, our options are to either continue to patch Cython 0.29.x within Debian to keep it working with Python 3.12 or to upgrade to Cython 3.x. As there is also software which now depends on Cython 3.x to build, the former option seems unappealing. (At best, we might wish to keep the cython-legacy package around for building packages which can't yet use Cython 3.x, but that should be a short-term thing, not a long-term one.) > 3.) The following one-liner suggests 44 debian > packages might be affected by the breaks > Rebecca said would be caused by pandas 2.x: > > $ for s in augur cnvkit dyda emperor esda mirtop pymatgen pyranges > python-anndata python-biom-format python-cooler python-nanoget python-skbio > python-ulmo q2-quality-control q2-demux q2-taxa q2-types q2templates > sklearn-pandas ; do apt-cache search "$s" ; done | less This does not seem like a particularly helpful one-liner; it picks up packages such as python3-dyda-pipeline-config which are not in the original list. Instead, you perhaps want to count the number of packages depending on these packages. But what Rebecca is looking at (I think) is how many packages would need fixing by the pandas upgrade. (But it is probably worse than this: I'm guessing these are only the packages which fail to build with pandas 2.x or whose autopkgtest fails with pandas 2.x. But there may well be other breakage caused by the upgrade which is not detectable in this way. That is an issue which will have to be handled by individual packages as they are discovered, and the timing of the pandas upgrade is not related to this problem.) > 4.) The break that worries me the most is > sklearn-pandas, because it seems to me that > sklearn is > > popular and > > fundamental. It seems that sklearn-pandas is abandoned; there were just two commits in 2022, and prior to that was May 2021. There has been no activity since. If someone is willing to patch it for Pandas 2.x, great (perhaps you might help the maintainer to do this?), otherwise it might have to drop out of Debian. Best wishes, Julian
Re: cython 3.x (for Python 3.12)
On Mon, Dec 11, 2023 at 08:09:31AM +0100, Matthias Klose wrote: > [...] > > > > I find that there's also a significant issue with relying on > > > > cython3-legacy: it conflicts with cython3, meaning that it will be > > > > impossible to simultaneously install packages depending on cython3 and > > > > cython3-legacy. Once cython 3.x moves from experimental to unstable > > > > to testing and packages start depending on it, this will become a > > > > significant issue. I assume that the aim will be for everything to be > > > > ported to cython 3.x and for cython3-legacy to be dropped from testing > > > > before the trixie freeze? > [...] > You could package a non-conflicting cython-legacy, however that would > require more changes, also how to build it. I had a very quick look at the packaged cython3-legacy. One could change the name of the module to Cython_legacy or probably better, Cython0, and cython.py to cython0.py, pyximport to pyximport0. Just a handful of changes would be needed, I'm guessing. Then it would be co-installable with Cython 3.x, and any packages that actually need cython3-legacy rather than Cython 3.x would have to be patched to call the legacy version. It's probably easier to just patch them for Cython 3.x ;-) Best wishes, Julian
Re: Bug#1043240: transition: pandas 1.5 -> 2.1
On 12/11/23 08:12, Matthias Klose wrote: On 10.12.23 14:06, Rebecca N. Palmer wrote: I'd like to move forward with the pandas 1.5 -> 2.1 transition reasonably soon. Given that pandas 2.x is *not* required for Python 3.12 (but is required for Cython 3.0), should we wait for the Python 3.12 transition to be done first? These are broken by pandas 2.x and have a possible (but untested) fix in their bug - please test and apply it: dask(?) dials influxdb-python* python-altair python-feather-format python-upsetplot seaborn tqdm* (* = this package is currently also broken for a non-pandas reason, probably Python 3.12, that I don't have a fix for) These are broken by pandas 2.x and have no known-to-me fix: augur cnvkit dyda emperor esda mirtop pymatgen pyranges python-anndata python-biom-format python-cooler python-nanoget python-skbio python-ulmo q2-quality-control q2-demux q2-taxa q2-types q2templates sklearn-pandas Some generic things to try are pandas.util.testing -> pandas.testing, .iteritems() -> .items(), and if one exists, a more recent upstream version. Is this an acceptable amount of breakage or should we continue to wait? Bear in mind that if we wait too long, we may be forced into it by some transition further up the stack (e.g. a future Python or numpy) that breaks pandas 1.x. up to the maintainers. But please wait at least until the current pandas and numpy migrated to testing, e.g. that the autopkg tests of pandas and numpy triggered by python3-defaults pass. Is there a way to see the binNMUs which are still stuck in unstable, and don't migrate? Matthias As a reminder: it's best practice to first upload the new release to Experimental, so we can see what happens with autopkgtest before destroying everything at once... Cheers, Thomas Goirand (zigo)
Re: cython 3.x (for Python 3.12)
On Mon, Dec 11, 2023 at 08:09:31AM +0100, Matthias Klose wrote: > [...] > You could package a non-conflicting cython-legacy, however that would > require more changes, also how to build it. Hi Matthias, Unfortunately, at least some of cython3-legacy doesn't currently work with Python 3.12, and is the primary cause of (at least) #1056531. cython3 provides the pyximport module, and that uses the imp module which has been removed from Python 3.12. Two possible ways forward on this particular bug: - Disable all of the cython tests for this package for the time being, until cython 3.x migrates to testing - this is simple and effective. - Patch cython3-legacy to use importlib rather than imp. This is probably a good thing to do anyway. (It may also be good to do this with cython3 version 0.x currently in testing/unstable until cython 3.x is able to be uploaded to unstable.) Then have my package's autopkgtest depend on cython3-legacy (unless cython3 0.x is also patched). Thoughts? Julian
Re: cython 3.x (for Python 3.12)
On 11.12.23 16:19, Julian Gilbey wrote: On Mon, Dec 11, 2023 at 08:09:31AM +0100, Matthias Klose wrote: [...] You could package a non-conflicting cython-legacy, however that would require more changes, also how to build it. Hi Matthias, Unfortunately, at least some of cython3-legacy doesn't currently work with Python 3.12, and is the primary cause of (at least) #1056531. cython3 provides the pyximport module, and that uses the imp module which has been removed from Python 3.12. Two possible ways forward on this particular bug: - Disable all of the cython tests for this package for the time being, until cython 3.x migrates to testing - this is simple and effective. - Patch cython3-legacy to use importlib rather than imp. This is probably a good thing to do anyway. (It may also be good to do this with cython3 version 0.x currently in testing/unstable until cython 3.x is able to be uploaded to unstable.) Then have my package's autopkgtest depend on cython3-legacy (unless cython3 0.x is also patched). I won't working on this. Have you tried to depend on the python3-zombie-imp instead?
[Help] Re: python-future: FTBFS: dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p 3.11 returned exit code 13
Control: tags -1 help [Bug #1056419 in CC since the issue seems to be caused by python-future] Hi Vincent, I tried to upgrade python-future to the latest upstream version in the hope that this would solve the issue reported in bug #1042244. Unfortunately this is not the case and now with Python3.12 we also have to deal with the removal of imp (which affects bug #1056419). I'd like to ask for help on debian-python list since I'm pretty overworked with other stuff. Please also review my rude patch[1] to hack around a shinx issue. It would be great to have some better solution here. You can find a full build log of the latest upstream version in Salsa CI[2]. Kind regards Andreas. [1] https://salsa.debian.org/python-team/packages/python-future/-/blob/master/debian/patches/hack_around_missing_toint_in_sphinx.patch [2] https://salsa.debian.org/python-team/packages/python-future/-/jobs/5027089 -- http://fam-tille.de
Re: Bug#1056419: [Help] Re: python-future: FTBFS: dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p 3.11 returned exit code 13
Hi Andreas, * Andreas Tille [2023-12-11 16:42]: Control: tags -1 help [Bug #1056419 in CC since the issue seems to be caused by python-future] Hi Vincent, I tried to upgrade python-future to the latest upstream version in the hope that this would solve the issue reported in bug #1042244. Unfortunately this is not the case and now with Python3.12 we also have to deal with the removal of imp (which affects bug #1056419). I'd like to ask for help on debian-python list since I'm pretty overworked with other stuff. Please also review my rude patch[1] to hack around a shinx issue. It would be great to have some better solution here. You can find a full build log of the latest upstream version in Salsa CI[2]. I think the right thing here is to package the new uncertainties version which drops the past import: https://github.com/lebigot/uncertainties/releases/tag/3.1.7 Also we should probably get rid of python-future at some point. Cheers Jochen signature.asc Description: PGP signature
Re: Bug#1043240: transition: pandas 1.5 -> 2.1
On 11.12.23 08:12, Matthias Klose wrote: On 10.12.23 14:06, Rebecca N. Palmer wrote: Is this an acceptable amount of breakage or should we continue to wait? Bear in mind that if we wait too long, we may be forced into it by some transition further up the stack (e.g. a future Python or numpy) that breaks pandas 1.x. up to the maintainers. But please wait at least until the current pandas and numpy migrated to testing, e.g. that the autopkg tests of pandas and numpy triggered by python3-defaults pass. I just nmued pyrle and sorted-nearest, having dependencies on cython3-legacy, letting the pyranges autopkg tests fail. Once this succeeds, pandas should be able to migrate.
Re: cython 3.x (for Python 3.12)
On Mon, Dec 11, 2023 at 04:34:17PM +0100, Matthias Klose wrote: > On 11.12.23 16:19, Julian Gilbey wrote: > > On Mon, Dec 11, 2023 at 08:09:31AM +0100, Matthias Klose wrote: > > > [...] > > > You could package a non-conflicting cython-legacy, however that would > > > require more changes, also how to build it. > > > > Hi Matthias, > > > > Unfortunately, at least some of cython3-legacy doesn't currently work > > with Python 3.12, and is the primary cause of (at least) #1056531. > > cython3 provides the pyximport module, and that uses the imp module > > which has been removed from Python 3.12. > > > > Two possible ways forward on this particular bug: > > > > - Disable all of the cython tests for this package for the time being, > >until cython 3.x migrates to testing - this is simple and effective. > > > > - Patch cython3-legacy to use importlib rather than imp. This is > >probably a good thing to do anyway. (It may also be good to do this > >with cython3 version 0.x currently in testing/unstable until cython > >3.x is able to be uploaded to unstable.) Then have my package's > >autopkgtest depend on cython3-legacy (unless cython3 0.x is also > >patched). > > I won't working on this. Have you tried to depend on the python3-zombie-imp > instead? Excellent - I didn't know about that. Are you OK for me to upload versions of cython and cython-legacy which depend on this to fix the Python 3.12 breakage? Best wishes, Julian
Re: cython 3.x (for Python 3.12)
On 11.12.23 19:55, Julian Gilbey wrote: On Mon, Dec 11, 2023 at 04:34:17PM +0100, Matthias Klose wrote: On 11.12.23 16:19, Julian Gilbey wrote: On Mon, Dec 11, 2023 at 08:09:31AM +0100, Matthias Klose wrote: [...] You could package a non-conflicting cython-legacy, however that would require more changes, also how to build it. Hi Matthias, Unfortunately, at least some of cython3-legacy doesn't currently work with Python 3.12, and is the primary cause of (at least) #1056531. cython3 provides the pyximport module, and that uses the imp module which has been removed from Python 3.12. Two possible ways forward on this particular bug: - Disable all of the cython tests for this package for the time being, until cython 3.x migrates to testing - this is simple and effective. - Patch cython3-legacy to use importlib rather than imp. This is probably a good thing to do anyway. (It may also be good to do this with cython3 version 0.x currently in testing/unstable until cython 3.x is able to be uploaded to unstable.) Then have my package's autopkgtest depend on cython3-legacy (unless cython3 0.x is also patched). I won't working on this. Have you tried to depend on the python3-zombie-imp instead? Excellent - I didn't know about that. Are you OK for me to upload versions of cython and cython-legacy which depend on this to fix the Python 3.12 breakage? not for cython, which won't need that soonish for 3.0.x. and if you have to update the b-d to cython3-legacy, why not add the zombie-imp dependency as well manually for the few packages that need it?
Bug#1058055: galpy: please remove extraneous dependency on python3-future
Source: galpy Version: 1.8.1-2 Severity: important X-Debbugs-Cc: debian-python@lists.debian.org Dear Maintainer, The removal of the python3-future library is being evaluated. It's obsolete & unmaitained upstream. Your package seems not to have required python3-future for a long time. Please remove the hardcoded dependency from debian/control. Greetings, /tmp/galpy $ grep future -r setup.py:# Note for the futureL could now get the actual compiler in the BuildExt class debian/control: python3-future, debian/control: python3-future, debian/changelog: * Add python3-future as build and package dependency /tmp/galpy $ grep past -r HISTORY.txt: examples to the user's clipboard for easy pasting into a Python /tmp/galpy $ -- System Information: Debian Release: trixie/sid APT prefers testing APT policy: (501, 'testing'), (450, 'unstable'), (400, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 6.5.0-5-amd64 (SMP w/2 CPU threads; PREEMPT) Kernel taint flags: TAINT_USER Locale: LANG=fr_BE.UTF-8, LC_CTYPE=fr_BE.UTF-8 (charmap=UTF-8), LANGUAGE=fr_BE:fr Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system)
Bug#1058056: multiqc: please remove extraneous dependency on python3-future
Source: multiqc Severity: normal X-Debbugs-Cc: debian-python@lists.debian.org Dear Maintainers, Your package's setup.py declares an extraneous dependency on old compatibility layer python3-future. > setup.py:"future>0.14.0", But it doesn't need it at all: no import of "past" or "future" libraries. Please report it upstream and in the meantime build a package with this line patched-out. Greetings, Alexandre -- System Information: Debian Release: trixie/sid APT prefers testing APT policy: (501, 'testing'), (450, 'unstable'), (400, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 6.5.0-5-amd64 (SMP w/2 CPU threads; PREEMPT) Kernel taint flags: TAINT_USER Locale: LANG=fr_BE.UTF-8, LC_CTYPE=fr_BE.UTF-8 (charmap=UTF-8), LANGUAGE=fr_BE:fr Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system)
Re: cython 3.x (for Python 3.12)
On Mon, Dec 11, 2023 at 08:06:41PM +0100, Matthias Klose wrote: > > [...] > > Excellent - I didn't know about that. Are you OK for me to upload > > versions of cython and cython-legacy which depend on this to fix the > > Python 3.12 breakage? > > not for cython, which won't need that soonish for 3.0.x. and if you have to > update the b-d to cython3-legacy, why not add the zombie-imp dependency as > well manually for the few packages that need it? The problem is not with other packages importing imp (which need to be fixed in such a case), rather that pyximport (in cython/cython-legacy) imports imp. So it's cython that needs to be patched. I'm wondering what the provisional timescale for cython 3.0.x is? Should I just leave my package with an autopkgtest failure until cython 3.0.x is in unstable or ideally testing? That's why I was thinking of also patching cython in the short term until we are ready for cython 3.0.x to enter unstable. Best wishes, Julian
Bug#1058057: impacket: please remove erroneous extraneous reference to 'future' from setup.py
Source: impacket Severity: important X-Debbugs-Cc: debian-python@lists.debian.org Dear Maitainer, Upstream mistakenly added 'future' to the requirements in setup.py Maybe they tought it was needed to get the "from __future__ import ..." statements working. That would had been "from future import ..." / "from past import ...". Nowadays it means that your package is pulling in python3-future. This library is obsolete and mostly unmaintained and should be removed from Debian. So please patch it out from the build. Greetings, Alexandre -- System Information: Debian Release: trixie/sid APT prefers testing APT policy: (501, 'testing'), (450, 'unstable'), (400, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 6.5.0-5-amd64 (SMP w/2 CPU threads; PREEMPT) Kernel taint flags: TAINT_USER Locale: LANG=fr_BE.UTF-8, LC_CTYPE=fr_BE.UTF-8 (charmap=UTF-8), LANGUAGE=fr_BE:fr Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system)
Re: Bug#1056419: [Help] Re: python-future: FTBFS: dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p 3.11 returned exit code 13
Le lun. 11 déc. 2023 à 17:02, Jochen Sprickerhof a écrit : > I think the right thing here is to package the new uncertainties version > which drops the past import: > > https://github.com/lebigot/uncertainties/releases/tag/3.1.7 +1 > Also we should probably get rid of python-future at some point. I removed it from three games this week-end and filled 6 more bugs since to remove extraneous stale dependency. There are in fact more fake stale dependencies than remaining true ones It takes like 10 minutes to review one package. It's a peaceful life. https://salsa.debian.org/games-team/ardentryst/-/commit/fc6901b0e90b6bb3ec19b23c1c2d458d653b2d4a I will continue this review. The existing bugs can be tagged someway if that helps. Greetings python3-gnocchi 0 python3-mir-eval 1 dioptas 2 python3-bioxtasraw 2 openqa-client3 rocketcea3 graide 5 python3-emperor 5 python3-grapefruit 6 python3-stomper 6 python3-junitparser 7 python3-pyhamtools 8 python3-pyxnat 11 onionbalance 16 python3-graphite216 python3-picopore 16 turing 16 autoradio17 #1054207 python3-biomaj3 24 python3-flask-autoindex 25 python3-scikit-rf26 radon29 osdlyrics30 python3-pyswarms 30 python3-pyocd34 bugwarrior 40 python3-gnocchiclient47 buildbot-worker 63 python3-bibtexparser 71 weechat-matrix 71 gnome-keysign82---> upstream python3-proselint83 python3-cpuset 90 python3-mdp 143 old_div python3-yade 192 c++ python3-plaso212 RM python3-uncertainties262 package new version chirp321 non duplicity10757
Re: Bug#1056419: [Help] Re: python-future: FTBFS: dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p 3.11 returned exit code 13
Hi Alexandre, Am Mon, Dec 11, 2023 at 09:18:16PM +0100 schrieb Alexandre Detiste: > > https://github.com/lebigot/uncertainties/releases/tag/3.1.7 > > +1 OK. > > Also we should probably get rid of python-future at some point. > > I removed it from three games this week-end > and filled 6 more bugs since to remove extraneous stale dependency. > > There are in fact more fake stale dependencies than remaining true ones > It takes like 10 minutes to review one package. > It's a peaceful life. > > https://salsa.debian.org/games-team/ardentryst/-/commit/fc6901b0e90b6bb3ec19b23c1c2d458d653b2d4a > > I will continue this review. I've seen your bug reports on some Debian Med packages and will try to fix these as quickly as possible. Thanks a lot for your invesigation and the bug reports Andreas. -- http://fam-tille.de
