On 07/25/2017 07:22 AM, Sergei Trofimovich wrote: > 2. Q: How to make arch testing faster and easier? > > A: - KEYWORDREQ/STABLEREQ bugs not marked as "runtime testing > required" will be automatically tested and keyworded. > > [handwave] automated tinderbox setup would help a lot > to now upfront what fails to built, fails tests.
I've had similar thoughts about this and have already been working on some tooling for this. We would need to establish exactly what criteria must be met for an automated test to be considered as successful. Here's a sample report that my tool produces: https://dev.gentoo.org/~kensington/tinderbox/sys-apps/dbus/dbus-1.6.18-r1/df017e14-bd68-47e2-9738-554e7ba1cf10.html In this case, would it be enough that it builds and tests pass? What about the QA issues? Do we need someone to review them to determine if they should block stabilisation, or if they're even a regression or not?
