Piglit silently ignored crashes in subtests. It's impossible to know what subtest crashed. Theoretically it might even be possible that a test crashes after the last call to piglit_report_subtest_result and thus no subtest crashed. Though the odds of that happening are probably pretty long.
If a test with subtests crashes, this commit adds an additional subtest named "unknown" with result "crash". The issue that subsequent subtests are not run is not touched upon by this commit. This is more of a work-around. A proper fix would modify the C subtest framework to print a list of subtests it intends to run at the start of the test. This would enable the python subtest framework to know which subtest crashed. But that's a lot of work (it would require modifying more than 100 tests) and this is better than nothing. See also: https://bugs.freedesktop.org/show_bug.cgi?id=74642 --- framework/results.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/framework/results.py b/framework/results.py index 99dd3735b..786bd6b4d 100644 --- a/framework/results.py +++ b/framework/results.py @@ -231,7 +231,13 @@ class TestResult(object): # Set special instances if 'subtests' in dict_: - inst.subtests = Subtests.from_dict(dict_['subtests']) + subtests = dict_['subtests'] + if getattr(inst, 'result', status.NOTRUN) == status.CRASH: + # A test with subtests crashed. + # It's impossible to know what subtest (if any) + # crashed and how many subtests were not run. + subtests.update({'unknown': 'crash', '__type__': 'Subtests'}) + inst.subtests = Subtests.from_dict(subtests) if 'time' in dict_: inst.time = TimeAttribute.from_dict(dict_['time']) -- 2.15.1 _______________________________________________ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit
