On 2022-09-14, Philip Hands wrote: > I suggest adding a 'nocheck' variation, that sets DEB_BUILD_OPTIONS=nocheck > during the build,
Sounds reasonable! > and enabling it by default. Less sure... > The reason for doing so is that one could imagine that a package produces > differing results depending upon whether the tests were run or not Indeed! > but also > (given that the tests will have passed during the normal build) the tests > failing during the varied build seems unlikely to be identifying faults that > are > worth fixing, and so is just a waste of cycles. How do you know weather the bugs it is identifying are worth fixing? It could also identify non-deterministic failures, or failures triggered by specific build environment configurations... > This idea is prompted by `busybox` where the tests fail in the varied > scenario, > despite the fact that the package is reproducible. > Here they are failing: > > https://salsa.debian.org/installer-team/busybox/-/jobs/3227197 > > (among other things, du produces weird results when the `fileordering` > variation is active, claiming the 1MB directoy is 2MB so the tests fail, so > the varied package is not produced, so we don't get to see that it was > reproducible: > > https://salsa.debian.org/installer-team/busybox/-/blob/master/testsuite/du/du-m-works > ) Consistantly? Then, maybe the test needs to be improved? > I found a couple of ways of making the issue go away: > > 1) disabling the 'fileordering' variation, thus: > > > https://salsa.debian.org/installer-team/busybox/-/commit/17387890c73388e1f56a6ae9fbc79783095b4e86 > > https://salsa.debian.org/installer-team/busybox/-/jobs/3233259 > > 2) telling the package to skip the tests when doing the variations: > > > https://salsa.debian.org/installer-team/busybox/-/commit/5260442e8ceea220fa36bdda169978d15108f781 > > which is setting this in the salsa-ci.yml: > SALSA_CI_REPROTEST_ARGS: > --variations=environment.variables+=DEB_BUILD_OPTIONS=nocheck > > https://salsa.debian.org/installer-team/busybox/-/jobs/3235476 > > > Option 2) is what I'm suggesting making into a default variation. > > If nothing else it will speed up testing of packages with extensive test > suits. I think it's a valuable feature, but I'm not entirely sure weather it should be default or not... live well, vagrant
signature.asc
Description: PGP signature
_______________________________________________ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
