Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
Hey all, Simon McVittie [2014-09-05 16:05 +0100]: > >> It might also be worth noting that the systemd maintainers switched from > >> git-dpm to gbp-pq recently (between 204 and 208, I think), so they > >> obviously didn't think git-dpm was the better option. I don't think anyone in pkg-systemd@ has looked at git-dpm yet. In fact we switched from gitpkg to standard git-buildpackage. gbp-pq is a local development tool which greatly eases patch handling especially for new upstream versions, but as that's not exposed in the official git, but is only a glorified way of maintaining the debian/patches quilt series locally that shouldn't affect other workflows. gitpkg is rather complicated to use and set up, only about 3 people in Debian know how it works properly, and it makes it really hard to track a set of changes against trunk over time (i. e. the equivalent of a quilt series, or stacked git). So I'm not sure where "switched from git-dpm" came from? Martin -- Martin Pitt| http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org) -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140905151824.gh3...@piware.de
Re: Autopkgtest smoke test for Python libraries
Hello all,
Thanks to Matthias for pointing this out to me. I try to reply
in-thread. Please keep CC:ing me as I'm not subscribed.
> The goal is to perform a simple “smoke test” to verify the correct
> installation of the Python libraries from the Debian packages, for all
> relevant Python interpreters.
>
> A secondary goal is to have the actual smoke test code be re-usable,
> possibly incorporated into a standard Debian Python packager toolkit.
Very nice! There's precedent for Perl, Ruby and DKMS packages which
have a fairly standard way to run the upstream test suite. For Python
there are some conventions, like "./setup.py test" or running
nosetests, maybe it's worth experimenting with those after getting the
initial "import works" smoketest in place.
This situation (testing many related packages in a similar way) is
what autodep8 is for. When run in a Debian source package tree, it
checks if it knows something about this type of package, and if so,
synthesizes a debian/tests/control. Have a look at
http://anonscm.debian.org/cgit/collab-maint/autodep8.git/tree/
which has a README, and the "detect" and "generate" scripts in
support/.
> * debian/tests/control
>
> Specifies two tests to run, ‘smoke-python2’ and ‘smoke-python3’.
>
> The dependencies explicitly specify the Python 2 and Python 3 library
> modules respectively. Can this be automated better?
The "generate" script could iterate over all binary packages of the
source and pick out the python-* or the python3-* ones, respectively.
> * debian/tests/smoke-python2, debian/tests/smoke-python3
>
> These are Bash programs that just run a Python module under every
> available Python interpreter.
>
> Running ‘py{,3}versions -i’ gets all the installed Python
> interpreters. Can this fail, e.g. can the set of interpreters be
> different from what interpreters the package was built for?
Yes, it can, and that's IMHO a situation where you *want* the test to
fail, as your package is actually broken (or at least in need of a
binNMU).
> These scripts are very simple, but not simple enough that I was
> comfortable stuffing them directly into the ‘debian/tests/control’
> file. Is that a shortcoming of the Autopkgtest specification?
Not sure what you mean with "shortcoming", you can put arbitrary shell
code into Test-Command:. However, the spirit is that these should be
very short, usually one line. I suggest that they should mostly just
call something like "/usr/share/python-something/run-autopkgtest $py $pkgname"
and you maintain the actual test code centrally in e. g.
python-defaults.
> * debian/tests/smoke_test.py
>
> A module that runs a simple suite of smoke tests: emit the interpreter
> description, fetch the Python distribution of the installed library
> and emit that, import the modules and emit those.
>
> Implemented as a command-line program so that the distribution name,
> and the set of module names, are specified by the caller. This
> hopefully makes ‘smoke_test.py’ reuseable for many Python library
> packages.
>
> Does this module meet your idea of a simple “smoke test” for Python
> library packages? Should it do more?
That seems like an excellent starting point indeed. Making sure that
all modules are importable will catch missing builds for particular
Python versions, regressions in newer Python versions, missing
dependencies, packaging errors, etc. As said above, maybe in the
future we want to try and run upstream test suites, but this will need
a lot more heuristics (starting with additional test deps).
Thanks,
Martin
--
Martin Pitt| http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
signature.asc
Description: PGP signature
Re: Autopkgtest smoke test for Python libraries
Hey Barry, Barry Warsaw [2016-03-01 10:48 -0500]: > I gather that it's supposed to be used by the maintainer in a source package > ($vcs repo) to jumpstart a debian/tests directory? You could use it for that purpose indeed, but that's not really what it is intended for. The idea is that the generic tests apply to all packages of a particular type, so you can change them centrally instead of having to modify and upload hundreds of packages. There is one more thing, though: The test machinery needs to be able to discover that a package has an autodep8 test (without having the source package already available, as getting and checking that is very expensive). So ideally all source packages which want to use that will get a "Testsuite: autopkgtest-pkg-python" header in debian/control, like for example libnet-oauth-perl. However, until these get uploaded we can add some custom code to debci to recognize those packages based on the name and dependencies (that's what we did for bootstrapping the perl and ruby autodep8 tests). > I'm not sure how useful things like "setup.py test" are for DEP-8. In general > I think a package's test suite are best exercised during package build[*]. During package build is of course good, but one of the points of autopkgtest is to run the tests when one of your dependencies change (and thus potentially break you). > For the majority of Python packages, I view DEP-8 as ensuring that the user > experience is what they expect, and that means at a minimum that the packages > import and Do Something Simple. A generic proof of non-broken import can mean > printing the package's version or the module path, and that's something that > autopkg8 could templatize. Anything more than that requires some more > detailed knowledge about what the package does. As I said, running upstream tests works surprisingly well for Perl/Ruby, we had about 80% success rate. But they are a bit more rigid in structure apparently. But indeed, ensuring that your module still imports already says a lot. New dependencies can still break your module in subtle ways, but at least things like new/removed Python versions, linker errors, wrong paths etc. are spotted. Martin -- Martin Pitt| http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org) signature.asc Description: PGP signature
cssutils salsa PR review or permission update
Hello Pythoneers, The other day I worked a bit on the cssutils package (I'm a co-maintainer). But then I noticed that since the salsa migration I can't commit to the official packaging git any more, so I sent a PR instead: https://salsa.debian.org/python-team/modules/cssutils/merge_requests/1/commits Does the Python modules team receive notifications about these? I don't mind much working through PRs, or perhaps it's possible to add me as a committer for cssutils on the salsa repo? I just wouldn't like to upload a new version and leave the VCS out of date. Thanks for any suggestions, Martin
