Acked-by: Dylan Baker <[email protected]>
On Tue, Mar 03, 2015 at 02:47:59PM +0000, Jose Fonseca wrote: > I recently tried the junit backend's ability to ignore expected > failures/crashes and found it a godsend -- instead of having to look as > test graph results periodically, I can just tell jenkins to email me > when things go south. > > The only drawback is that by reporting the expected issues as passing it > makes it too easy to forget about them and misinterpret the pass-rates. > So this change modifies the junit backend to report the expected issues > as skipped, making it more obvious when looking at the test graphs that > these tests are not really passing, and that whatever functionality they > target is not being fully covered. > > This change also makes use of the junit `message` attribute to explain > the reason of the skip. (In fact, we could consider using the `message` > attribute on other kind of failures to inform the piglit result, instead > of using the non-standard `type`.) > --- > framework/backends/junit.py | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/framework/backends/junit.py b/framework/backends/junit.py > index 82f9c29..53b6086 100644 > --- a/framework/backends/junit.py > +++ b/framework/backends/junit.py > @@ -129,17 +129,19 @@ class JUnitBackend(FileBackend): > # Add relevant result value, if the result is pass then it > doesn't > # need one of these statuses > if data['result'] == 'skip': > - etree.SubElement(element, 'skipped') > + res = etree.SubElement(element, 'skipped') > > elif data['result'] in ['warn', 'fail', 'dmesg-warn', > 'dmesg-fail']: > if expected_result == "failure": > err.text += "\n\nWARN: passing test as an expected > failure" > + res = etree.SubElement(element, 'skipped', > message='expected failure') > else: > res = etree.SubElement(element, 'failure') > > elif data['result'] == 'crash': > if expected_result == "error": > err.text += "\n\nWARN: passing test as an expected crash" > + res = etree.SubElement(element, 'skipped', > message='expected crash') > else: > res = etree.SubElement(element, 'error') > > -- > 2.1.0 >
signature.asc
Description: Digital signature
_______________________________________________ Piglit mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/piglit
