On Mon, Jan 16, 2017 at 05:45:19PM -0800, Russ Allbery wrote: > > Well, maybe what it's excessively aggressive or questionable is to run > > the tests at build time and making the package build as a whole to fail > > when any test fails. > > *blink*. > > I'm quite surprised that you would advocate not failing a build if tests > fail during the package build? I think that would be an awful way to > proceed. My packages have test suites for a reason. I do not want > packages to appear to successfully build if their tests are failing. That > may mean that the resulting binaries are nonfunctional or even dangerous.
Not exactly. I'm not advocating not failing a build if tests fail as a general rule. In this context, I refer specifically to flaky tests. What I call questionable is keeping a flaky test making the build to fail when the test fails so much that it's clearly a wrongly designed test. Or, alternatively, if the test fail a lot and it's correctly designed, it is questionable not to consider the bug as RC. Thanks.
