On wto, 2017-07-25 at 22:59 +1000, Michael Palimaka wrote: > On 07/25/2017 07:22 AM, Sergei Trofimovich wrote: > > 2. Q: How to make arch testing faster and easier? > > > > A: - KEYWORDREQ/STABLEREQ bugs not marked as "runtime testing > > required" will be automatically tested and keyworded. > > > > [handwave] automated tinderbox setup would help a lot > > to now upfront what fails to built, fails tests. > > I've had similar thoughts about this and have already been working on > some tooling for this. > > We would need to establish exactly what criteria must be met for an > automated test to be considered as successful. > > Here's a sample report that my tool produces: > https://dev.gentoo.org/~kensington/tinderbox/sys-apps/dbus/dbus-1.6.18-r1/df017e14-bd68-47e2-9738-554e7ba1cf10.html > > In this case, would it be enough that it builds and tests pass? What > about the QA issues? Do we need someone to review them to determine if > they should block stabilisation, or if they're even a regression or not? >
There are different kinds of QA issues and they have different significance. You can ignore some of them, some others are more important. GLEP 65 defines a 'eqatag' function that the checks can use to provide machine-readable results for the checks. You should work with that instead of parsing the verbose messages. -- Best regards, Michał Górny
signature.asc
Description: This is a digitally signed message part
