Pjotr Prins <pjotr.publi...@thebird.nl> skribis: > I am *not* suggesting we stop testing and stop writing tests. They are > extremely important for integration (thought we could do with a lot > less and more focussed integration tests - ref Hickey). What I am > writing is that we don't have to rerun tests for everyone *once* they > succeed *somewhere*. If you have a successful reproducible build and > tests on a platform there is really no point in rerunning tests > everywhere for the exact same setup. It is a nice property of our FP > approach. Proof that it is not necessary is the fact that we > distribute substitute binaries without running tests there. What I am > proposing in essence is 'substitute tests'.
Understood. > If tests are so important to rerun: tell me why we are not running > tests when substituting binaries? Because you have a substitute if and only those tests already passed somewhere. This is exactly the property we’re interested in, right? That is why I was suggesting putting effort in improving substitute delivery rather than trying to come up with special mechanisms. >> Ricardo Wurmus <rek...@elephly.net> skribis: >> >> > An idea that came up on #guix several months ago was to separate the >> > building of packages from testing. Testing would be a continuation of >> > the build, like grafts could be envisioned as a continuation of the >> > build. >> >> I agree it would be nice, but I think there’s a significant technical >> issue: test suites usually expect to run from the build tree. > > What I understand is that Nix already does something like this. they > have split testing out to allow for network access. Do you have pointers to that? All I’m aware of is the ‘doCheck’ variable that is unset (i.e., false) by default: https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/setup.sh#L1192 Ludo’.
