On Fri, Aug 08, 2025 at 10:51:21AM +0800, Tianyu Chen wrote:
Yesterday I was packaging python-pywebpush[1]. To enable autopkgtest, I used `Testsuite: autopkgtest-pkg-pybuild' in d/control. But when autopkgtest stage, pytest didn't discover any tests. weepingclown and I dug into the problem and found tests in pywebpush/tests are not copied to the build directory; instead, pybuild only copies pyproject.toml, pytest.ini, test, and tests directories. pybuild(1) documented that "you can list them in debian/pybuild.testfiles file".By creating d/pybuild.testfiles and adding "pywebpush/tests" to the file, autopkgtest works. But reviewing the build log, we found all tests are running twice in dh_auto_test. The log shows: tests/test_webpush.py pywebpush/tests/test_webpush.py I asked on #debian-python IRC yesterday, and currently using `export PYBUILD_TEST_ARGS=--ignore pywebpush/tests' to avoid being tested twice. I also did some searching on codesearch.debian.net, and I found that some other packages are either tested twice or doing similar things to avoid being tested twice. Following are three attempts to ensure tested only once: src:isodate export PYBUILD_BEFORE_TEST=cp -r {dir}/tests {build_dir}/ export PYBUILD_TEST_ARGS=tests/test_*.py src:python-cymem export PYBUILD_NAME = cymem export PYBUILD_TEST_ARGS = tests src:python-ewokscore ifeq ($(PYBUILD_AUTOPKGTEST),1) export PYBUILD_TEST_ARGS := {dir}/src/ewokscore endif Are there better solutions to this problem?
These are all ultimately workarounds for https://bugs.debian.org/947800. I've generally found that there's unfortunately no one size that fits all: which option is best depends on the exact details of the package and how tolerant it is of its tests living in a slightly different location from the one it expects. For example, some tests do relative imports so they'll fail if you effectively move foo/tests/ to tests/, while some tests have expectations about their current working directory relative to the test suite.
With pytest it's often (not always) easiest to tell it to run tests from a non-installed path, as in python-cymem and python-ewokscore above. Manually copying the tests into the expected subdirectory using PYBUILD_BEFORE_TEST and PYBUILD_AFTER_TEST works too, and that's usually what I end up doing with other test frameworks, but it's cumbersome enough that I usually only do this when I have to.
In your case I would probably drop debian/pybuild.testfiles and do `export PYBUILD_TEST_ARGS = {dir}/pywebpush/tests` in debian/rules, although it does produce mildly uglier test output, and to some extent this is a matter of taste.
Ideally I think we'd be able to write something like `pywebpush/tests => pywebpush/tests` in debian/pybuild.testfiles, in the style of dh-exec. Then most of these hacks would be unnecessary.
-- Colin Watson (he/him) [cjwat...@debian.org]
