Shouldn't the test's result just be included into the "max" no matter what?
On Mon, May 9, 2016 at 5:46 PM, Dylan Baker <[email protected]> wrote: > In piglit when a test generates subtests we treat the "worst" subtest as > the test status, and in most cases this works as expected. There is at > least one case where this is not correct, and that is the case of crash. > > There are a couple of reasons that crash should not be masked. One is > that it is generated by the framework when the test binary hits an > assert or segfaults, or any number of similar cases. The second is that > it may mean that all of the subtests did not run, as such we don't want > the status to be masked by the "worst" subtest which would be fail at > worst. > > Signed-off-by: Dylan Baker <[email protected]> > --- > > framework/results.py | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/framework/results.py b/framework/results.py > index 469abeb..2095d90 100644 > --- a/framework/results.py > +++ b/framework/results.py > @@ -173,10 +173,13 @@ class TestResult(object): > """Return the result of the test. > > If there are subtests return the "worst" value of those subtests. If > - there are not return the stored value of the test. > + there are not return the stored value of the test. There is an > + exception to this rule, and that's if the status is crash; since this > + status is set by the framework, and can be generated even when some > or > + all unit tests pass. > > """ > - if self.subtests: > + if self.subtests and self.__result != status.CRASH: > return max(six.itervalues(self.subtests)) > return self.__result > > -- > 2.8.2 > > _______________________________________________ > Piglit mailing list > [email protected] > https://lists.freedesktop.org/mailman/listinfo/piglit _______________________________________________ Piglit mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/piglit
