On Wed, Jan 18, 2017 at 1:08 AM, Russ Allbery wrote: > Santiago Vila writes: >> In this context, I refer specifically to flaky tests. What I call >> questionable is keeping a flaky test making the build to fail when the >> test fails so much that it's clearly a wrongly designed test. > > Oh, sure, I'm in favor of disabling flaky tests if we can't fix them. My > experience is usually more that I'm leaving them on *because* I'm trying > to fix them and can't reproduce locally, or I think I've fixed it (but > actually haven't). > > Some upstream test suites also make it a little difficult to disable a > single test without carrying a patch. (Hm, including mine....)
I would expect most upstream test frameworks support marking tests as flaky, which usually means they always get run and results printed but their outcome never causes a build failure. -- bye, pabs https://wiki.debian.org/PaulWise
