Hi Ephraim, On Thu, Oct 05 2023, Efraim Flashner wrote:
> I propose changing the cargo-build-system to have '#:tests? #f' by > default At first sight, it appears improper to turn off tests because they fail. Please allow me to remind everyone that build-time tests cover only a small proportion of problems actually encountered by users. Most packaging errors, like improper permissions or the installation of components in a wrong path, usually go undetected. One of Debian's solutions to that realization is autopkgtest [1] which allows maintainers to specify a test bed that tests the *installed* versions and not the versions in the build trees. A common strategy for Debian maintainers is to convert the build-time tests to an autopkgtest suite. That way, folks get the benefit of both. Unfortunately, the setup of test beds is complicated in Debian, as the installation of the packages being tested has to take place in containers. In Guix, package installations are decoupled from the running system. Guix would make that process a lot easier, faster and more reliable! In summary, I believe that #:test? should be turned off for all build systems. Guix should instead test installed versions like Debian's autopkgtest. It would be an extra burden on contributors because such a 'test-installed phase would require more attention. It may be worthwhile, however, because than packages could be built without testing them---as Ephraim would like to do here. In addition, pre-built substitutes could be tested by consumers on their own systems. The substitutes could even be tested before they become part of any Guix profile. For Debian's QA tool Lintian, which I maintained for several years, the speed-up in the development process was remarkable. As a Perl script, the build toook seven minutes, while the build-time tests took seven hours. The builds were initiated with each commit in Salsa (in an online runner sponsored by a large company). Lintian was extreme because thousands of tests replicated much of what happens daily in Debian. The extreme duration of build-time the tests also took up further resources in downstream distributions. The tests ran for each backport and for each derivative, such as Ubuntu. For Guix, which relies on frequent rebuilds, the speed benefit could be remarkable. Substitutes could become available for testing in perhaps half the time. That being said, old habits die hard. The attachment to build-time tests is formidable. The people who maintained Lintian after me enabled them again. Kind regards Felix [1] https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst
