On Sun, Sep 08, 2024 at 03:43:33PM +0200, Stefano Rivera wrote: > [...] > We've implemented the feature in Python 3.12, unittest's runner now > exits with return value 5 if no tests were discovered, like pytest does. > https://github.com/python/cpython/issues/113661 > [...] > > I built 6440 packages build-depending on dh-python in one way or > another. 1483 failed, and 1124 of them say "NO TESTS RAN" in the logs.
My guess is that most of these 1124 have no tests at all, rather than having a misconfigured setup. A unittest is the pybuild default test framework, unittest is used and fails to find any tests, hence all of these failures. > To fix these build failures, package maintainers would have these > options: > > 1. Get the build to run some unit tests (assuming they exist), > 2. override_dh_auto_test with something noop, > 3. export PYBUILD_DISABLE=test, > 4. We could make this failure opt-in in dh-python. Maybe via an explicit > --test-unittest option that selects the unittest runner. If you don't > explicitly select this runner, you'd get an attempt to run tests by > with unittest, and no failure if no tests are found. I like option 4 for the above reason. But implementing this would mean that all of the packages that currently *do* use unittest (intentionally, but without having to code it explicitly as it's the default) would suddenly not have any tests running until they proactively add --test-unittest or set PYBUILD_TEST_UNITTEST = 1 or similar. This seems like an unfortunate consequence. Is there a way of looking at the logs of the packages that passed the build to identify which ones successfully passed tests using unittest? There is also the issue of packages that use unittest (as the default) via autopkgtest-pkg-pybuild but override dh_auto_test during the build, though that will be much rarer. Best wishes, Julian
