Re: Notes from the DC22 Python Team BoF
On 7/25/22 09:37, Julian Gilbey wrote: On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote: Hey folks, We had a Python Team BoF at DC22 earlier today and I thought relaying the notes we took in gobby here would be a good idea. Thanks for the notes, Louis-Philippe, and sorry I couldn't join you! A few comments -- == python3.11 == python3.11 release has been delayed, from october 2022 to december 2022. [...] My 2 cents' worth is as the 3.9->3.10 transition took several months, and was quite complicated, it is not wise to attempt the 3.10->3.11 before the freeze. We could then potentially go straight to 3.12 a few months after the bookworm freeze rather than going to 3.11 first. And that will probably be quite painful. I agree, also because 3.10 wont be EOL before Bookworm becomes LTS. However, it's quite tempting to upgrade to 3.11 because: - our users will prefer having the latest stable release of Python in our latest release, rather than an old version. - 3.11 has many optimization (it's said to be 25% faster) So if we can at least TRY, it's not a so bad idea... Hopefully, we can take the decision to reverse if needed. Cheers, Thomas Goirand (zigo)
Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)
On Tue, Jul 26, 2022 at 11:50:19AM -0300, Antonio Terceiro wrote: > I think the notes did not capture the consensus correctly. The point was > that it should be possible to automate updating the `Testsuite:` field > to run tests with pybuild-autopkgtest, and that we should probably do > that across team packages with the help of some scripting. This makes more sense, thanks. There seems to be little point running both pybuild-autopkgtest and a manually written debian/tests/* test suite. So would the script only add pybuild-autopkgtest to packages which don't have a manually written debian/tests/* suite? Best wishes, Julian
Re: Notes from the DC22 Python Team BoF
On 2022-07-27 03 h 52, Thomas Goirand wrote: On 7/25/22 09:37, Julian Gilbey wrote: On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote: Hey folks, We had a Python Team BoF at DC22 earlier today and I thought relaying the notes we took in gobby here would be a good idea. Thanks for the notes, Louis-Philippe, and sorry I couldn't join you! A few comments -- == python3.11 == python3.11 release has been delayed, from october 2022 to december 2022. [...] My 2 cents' worth is as the 3.9->3.10 transition took several months, and was quite complicated, it is not wise to attempt the 3.10->3.11 before the freeze. We could then potentially go straight to 3.12 a few months after the bookworm freeze rather than going to 3.11 first. And that will probably be quite painful. I agree, also because 3.10 wont be EOL before Bookworm becomes LTS. As I said during the BoF, I'm also in favor of sticking with 3.10 for bookworm. I think we should aim for a stable Python ecosystem. Using the time we have left to iron out bugs and failures in our packages for 3.10 sounds like a much better decision than trying to fix a ton of RC bugs for 3.11 in time for the final freeze. However, it's quite tempting to upgrade to 3.11 because: - our users will prefer having the latest stable release of Python in our latest release, rather than an old version. - 3.11 has many optimization (it's said to be 25% faster) So if we can at least TRY, it's not a so bad idea... Hopefully, we can take the decision to reverse if needed. AFAIU, doko said it was pretty hard to go back, since that meant recompiling pretty much all the python extensions. I'd much rather we don't waste tons of energy trying to get 3.11 and then deciding it's not working. We know we only have a 2-3 months window of opportunity to make the transition, IMO that's too short. -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ Louis-Philippe Véronneau ⢿⡄⠘⠷⠚⠋ po...@debian.org / veronneau.org ⠈⠳⣄ OpenPGP_0xE1E5457C8BAD4113.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)
On 2022-07-27 04 h 18, Julian Gilbey wrote: There seems to be little point running both pybuild-autopkgtest and a manually written debian/tests/* test suite. So would the script only add pybuild-autopkgtest to packages which don't have a manually written debian/tests/* suite? The way I see it: 1. We should have a Lintian tag for packages not using the new pybuild-autodep8 autopkgtest. It would be even better if this tag would be a pointed hint that identified 'manually' written unit test autopkgtests that could be replaced. This way, you get something like: python-foo source: not-using-pybuil-autodep8 [debian/tests/unittests] for python packages that have old 'manually' written unit test autopkgtests and: python-foo source: not-using-pybuild-autodep8 [no-autopkgtest] for python packages without any autopkgtest. 2. lintian-brush (or something else, but I think lintian-brush is the right tool) would go over these packages to: 2.1 Add the new autodep8 autopkgtests and build the package to see if they pass 2.2 Remove the "manual" unit test autopkgtests if 2.1 succeeds 2.3 Open a bug report if 2.1 fails This way, we can imagine a transition that would be mostly automated. We'd probably have to fix the ones that failed to migrate manually, but it would be much less work :) Cheers, -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ Louis-Philippe Véronneau ⢿⡄⠘⠷⠚⠋ po...@debian.org / veronneau.org ⠈⠳⣄ OpenPGP_signature Description: OpenPGP digital signature
Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)
On Wed, 27 Jul 2022 at 09:18:42 +0100, Julian Gilbey wrote: > There seems to be little point running both pybuild-autopkgtest and a > manually written debian/tests/* test suite. I think it can make sense to have both. d/tests is the right place for an integration test that checks things like "any user-facing executables are correctly in the PATH", which an upstream-oriented test probably cannot assert because you might be installing into ~/.local or similar. Using src:tap.py as an example, pybuild-autopkgtest should presumably replace d/tests/python3 and/or d/tests/python3-with-recommends (which are tests for the library you get from "import tap"), but wouldn't test the tappy(1) CLI entry point, which is tested by d/tests/tappy. smcv
Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)
On Wed, Jul 27, 2022 at 10:26:33AM -0400, Louis-Philippe Véronneau wrote: > The way I see it: > > 1. We should have a Lintian tag for packages not using the new > pybuild-autodep8 autopkgtest. It would be even better if this tag would be a > pointed hint that identified 'manually' written unit test autopkgtests that > could be replaced. > > This way, you get something like: > > python-foo source: not-using-pybuil-autodep8 [debian/tests/unittests] > > for python packages that have old 'manually' written unit test autopkgtests > and: > > python-foo source: not-using-pybuild-autodep8 [no-autopkgtest] > > for python packages without any autopkgtest. > > 2. lintian-brush (or something else, but I think lintian-brush is the right > tool) would go over these packages to: > > 2.1 Add the new autodep8 autopkgtests and build the package to see if they > pass > 2.2 Remove the "manual" unit test autopkgtests if 2.1 succeeds > 2.3 Open a bug report if 2.1 fails I'd be wary about 2.2 and 2.3. I have several packages where I know that an automated test will fail; there are all sorts of weird cases where I've had to write tests manually. I would also be quite cross if manually crafted tests were automatically removed, especially in cases such as Simon mentioned where they do things that that automatically generated test does not do. Another thing I could imagine happening is that the automated test succeeds in a trivial way - it succeeds but doesn't actually test much because of the nature of the package. On the other hand, a bug report saying something like the following seems much more reasonable: "We've tested this package using the automated autopkgtest system and it seems to work by adding the line 'Testsuite: autopkgtest-pkg-pybuild'; please check that the automated tests cover all of the tests of your manually written debian/tests/* and if so, then please remove them. The autopkgtest-pkg-pybuild logs are attached." This would give the maintainer the chance to decide how best to proceed. Best wishes, Julian
Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)
On Wed, Jul 27, 2022 at 07:45:12PM +0100, Julian Gilbey wrote: > On Wed, Jul 27, 2022 at 10:26:33AM -0400, Louis-Philippe Véronneau wrote: > > The way I see it: > > > > 1. We should have a Lintian tag for packages not using the new > > pybuild-autodep8 autopkgtest. It would be even better if this tag would be a > > pointed hint that identified 'manually' written unit test autopkgtests that > > could be replaced. > > > > This way, you get something like: > > > > python-foo source: not-using-pybuil-autodep8 [debian/tests/unittests] > > > > for python packages that have old 'manually' written unit test autopkgtests > > and: > > > > python-foo source: not-using-pybuild-autodep8 [no-autopkgtest] > > > > for python packages without any autopkgtest. > > > > 2. lintian-brush (or something else, but I think lintian-brush is the right > > tool) would go over these packages to: > > > > 2.1 Add the new autodep8 autopkgtests and build the package to see if they > > pass > > 2.2 Remove the "manual" unit test autopkgtests if 2.1 succeeds > > 2.3 Open a bug report if 2.1 fails > > I'd be wary about 2.2 and 2.3. I have several packages where I know > that an automated test will fail; there are all sorts of weird cases > where I've had to write tests manually. I would also be quite cross > if manually crafted tests were automatically removed, especially in > cases such as Simon mentioned where they do things that that > automatically generated test does not do. Another thing I could > imagine happening is that the automated test succeeds in a trivial way > - it succeeds but doesn't actually test much because of the nature of > the package. > > On the other hand, a bug report saying something like the following > seems much more reasonable: "We've tested this package using the > automated autopkgtest system and it seems to work by adding the line > 'Testsuite: autopkgtest-pkg-pybuild'; please check that the automated > tests cover all of the tests of your manually written debian/tests/* > and if so, then please remove them. The autopkgtest-pkg-pybuild logs > are attached." This would give the maintainer the chance to decide > how best to proceed. Here's another alternative to steps 2.1-2.3 based on this: For packages which currently have manually-written autopkgtests: 2.A Try removing debian/tests and adding Testsuite: autopkgtest-pkg-pybuild to debian/control, then building the package and running autopkgtest. 2.B If this works, then submit a bug report to the BTS as I suggested above. 2.C If this does not work, don't do anything more; trust that the maintainer knew what they were doing when they wrote the manual autopkgtests. For packages which don't currently have manually-written autopkgtests: 2.A' Try adding Testsuite: autopkgtest-pkg-pybuild to debian/control, then building the package and running autopkgtest. 2.B' If this works, then either Janitor adds this line to debian/control or submit a bug report to the BTS to recommend this. (But we would not expect Janitor to do step 2.1', so this might have to be a different setup, or maybe the script doing 2.A' could leave a list of packages for Janitor, or something like that.) 2.C' If this does not work, submit a wishlist bug to the BTS to recommend that the maintainer adds some autopkgtest tests, either by making the autodep8 system work or by writing some manual tests. I'd be wary about adding lintian tags for this, though: with so many packages not being able to use the autodep8 system (I vaguely recall someone suggesting that a third of Python packages would not be able to use the system), that's a lot of packages getting false positives. In particular, for the suggested first version of not-using-pybuild-autodep8 tag (which would probably be better named manual-autopkgtest-could-be-pybuild-autodep8), how would lintian go about identifying which packages fall into category 2.B and which into 2.C? The second version of the tag (better named something like no-autopkgtest-could-use-pybuild-autodep8, but that's still not very good) is less problematic. Best wishes, Julian
RFS - reintroduction of the Advene (video annotation) package - ITP #1014535
Hi Advene[1] is a video annotation software (GPL), developed with python, Gtk and Gstreamer. It was present in older Debian releases [2], but got removed some years ago due to major migrations (python2->python3, Gtk2->Gtk3, Gstreamer0.10->Gstreamer1.0). Since then, the application has been updated and is now in good shape to be reintroduced into Debian. The updated Debian package[3], migrated from cdbs to dh_python, has been used/tested for at least 3 years (distributed through a private repository[4]). As per [5]/[6] a ITP bug has been filed: #1014535 It was previously maintained under the Multimedia Team umbrella, but the there seems to be no more interested dev there with time to sponsor this package there, and I was hinted that the Python team may be more appropriate. I am looking for help/sponsorship from the Python Team to help getting the package back in Debian. I am ready to be part of the Python Team (I have read the policy.rst and accept it) and do most of the work if needed, but I would welcome guidance wrt the current practices of python packages. Anyone interested? Happy hacking! Best, Olivier Aubert [1] https://www.advene.org/ [2] https://tracker.debian.org/pkg/advene [3] https://salsa.debian.org/multimedia-team/advene [4] http://advene.org/download.html#linux [5] https://www.debian.org/doc/manuals/developers-reference/pkgs.html#reintroducing-pkgs [6] https://www.debian.org/doc/manuals/developers-reference/pkgs.html#newpackage
