Hi Josselin, On Mon, Mar 13, 2023 at 3:21 PM Josselin Poiret <d...@jpoiret.xyz> wrote: > > I would really like for tests to move out of build phases > > That would require a huge change to Guix though
Fortunately, there is a great precedent for that work in Debian. It is called autopkgtest. [1] One package I maintained there, called Lintian, had a comprehensive test suite that mimicked large parts of Debian including building packages. When I moved the tests out of the build process, it shortened the build time from six hours to seven minutes. That was on Gitlab with external runners provided by a donor. The key to understanding Debian's autopkgtest process is that, while the test suites are distributed together with the sources, they actually test the *installed* versions of the software (which are loaded separately). That crucial distinction catches important user-facing errors that were previously undetectable, such as installation in the wrong path. We already have many good starting points in Guix. Some truly great work has been done. [2] There are also two SRFIs [3][4] although I am personally biased toward TAP (the "Test-Anything Protocol") [5] and already used that in one of my Guile modules. [6] Either way, such testing efforts will make our distribution even better, especially as we support additional architectures. I would be eager to cooperate if there is an extra seat at the table. Kind regards, Felix [1] https://people.debian.org/~eriberto/README.package-tests.html [2] https://luis-felipe.gitlab.io/guile-proba/manual/ [3] https://srfi.schemers.org/srfi-64/srfi-64.html [4] https://srfi.schemers.org/srfi-78/srfi-78.html [5] https://codeberg.org/ft/guile-tap [6] https://codeberg.org/lechner/guile-naptcha/src/branch/history/scm/test/naptcha/pseudo-random.scm
