On Wed, 11 Oct 2017, Martin Sebor wrote: > I don't have a strong opinion on the definition of a Regression > in this context but I would very much like to see status changes > highlighted in the test results to indicate that something that
There are lots of things that are useful *if* you have someone actively reviewing them for every test run (maybe several times a day) and alerting people / filing bugs in Bugzilla if there are problems. Some of those things, however, are likely to have too many false positives for a display people can quickly look at to see if the build is red or green, or for automatically telling people their patch broke something. If we can clean up results for each system the bot runs tests on - XFAILing and filing bugs in Bugzilla for failures where there isn't a reasonably simple and obvious fix - we can make green mean "no FAILs or ERRORs" (remembering the possibility that with very broken testing, sometimes an ERROR might only be in the .log not the .sum). Other differences (such as PASS -> UNSUPPORTED) can then be reviewed manually by someone who takes responsibility for doing so, resulting in bugs being filed if appropriate, without affecting the basic red/green status. (Variants such as green meaning "no FAILs or ERRORs, except for failing guality tests where there should be no regressions" are possible as well, for cases like that where PASS/FAIL status depends on non-GCC components and meaningfully selective XFAILing is hard.) -- Joseph S. Myers jos...@codesourcery.com
