On 20/09/17 19:14, Joseph Myers wrote: > On Wed, 20 Sep 2017, Paulo Matos wrote: > >> - buildbot can notify people if the build fails or if there's a test >> regression. Notification can be sent to IRC and email for example. What >> would people prefer to have as the settings for notifications? > > It's very useful if someone reviews regressions manually and files bugs / > notifies authors of patches that seem likely to have caused them / fixes > them if straightforward. However, that can take a lot of time (especially > if you're building a large number of configurations, and ideally there > would be at least compilation testing for every target architecture > supported by GCC if enough machine resources are available). (I do that > for my glibc bot, which does compilation-only testing for many > configurations, covering all supported glibc ABIs except Hurd - the > summaries of results go to libc-testresults, but the detailed logs that > show e.g. which test failed or failed to build aren't public; each build > cycle for the mainline bot produces about 3 GB of logs, which get deleted > after the following build cycle.) >
I totally agree that only if people get involved in checking if there were regressions and keeping an eye on what's going on are things going to improve. The framework can help a lot here by notifying the right people and the mailing list if something gets broken or if there are regressions but once the notification is sent someone certainly needs to pick it up. I believe that once the framework is there and if it's reliable and user friendly and does not force people to check the UI every day, instead notifying people only when things go wrong, then it will force people to take notice and do something about breakages. At the moment, there are no issues with regards to logs sizes but we are starting small with a single worker. Once we have more we'll have to revisit this issue. -- Paulo Matos
