Quoting Ilia Mirkin (2016-05-10 14:32:59) > On Tue, May 10, 2016 at 5:32 PM, Dylan Baker <[email protected]> wrote: > > Quoting Ilia Mirkin (2016-05-10 06:41:33) > >> Shouldn't the test's result just be included into the "max" no matter what? > >> > > > > Well, the problem is that a lot of tests were retrofitted with subtests, > > and now the overall status is bunk, and for new tests we generally don't > > add an overall result when there's subtests. > > It's surprising if the status of a test says fail but piglit-summary > reports it as pass. In the case where one of the subtests fails but > the overall status is pass, this would still say fail, no? > > What situation are you protecting against by only incorporating the > overall status on crash?
I have this gut feeling that says skip or notrun might play havok here. But I guess I should look test it and see if that's the case. The other thing I guess is that there's only about 100 tests (or, c/c++ files, really) using subtests. So maybe it would just be easier to audit them to make sure that they do the right thing and just add the overall result to the max call.
signature.asc
Description: signature
_______________________________________________ Piglit mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/piglit
