On Wed, 20 Sep 2017, Segher Boessenkool wrote: > > - buildbot can notify people if the build fails or if there's a test > > regression. Notification can be sent to IRC and email for example. What > > would people prefer to have as the settings for notifications? > > Just try it! IRC is most useful I think, at least for now. But try > whatever seems useful, if there are too many complaints you can always > turn it off again ;-)
We have the gcc-regression list for automatic regression notifications (but as per my previous message, regression notifications are much more useful if someone actually does the analysis, promptly, to identify the cause and possibly a fix). My glibc bots only detect testsuite regressions that change the exit status of "make check" from successful to unsuccessful (and regressions that break any part of the build, etc.). That works for glibc, where the vast bulk of configurations have clean compilation-only testsuite results. It won't work for GCC - you need to detect changes in the results of individual tests (or new tests which start out as FAIL and may not strictly be regressions but should still be fixed). Ideally the expected baseline *would* be zero FAILs, but we're a long way off that. -- Joseph S. Myers jos...@codesourcery.com
