On Wed, Nov 25, 2015 at 02:21:18PM -0500, Rob Clark wrote: > On Wed, Nov 25, 2015 at 1:58 PM, Dylan Baker <baker.dyla...@gmail.com> wrote: > > On Wed, Nov 25, 2015 at 11:32:48AM -0500, Rob Clark wrote: > >> Complete hack, but maybe we want to make something like this an optional > >> way that piglit-summary dumps out results. > >> > >> Basically it groups all the results according to transition (ie. 'pass > >> -> fail' or 'fail -> fail -> pass', etc., and then for each group dumps > >> out the test environment and cmdline. > >> > >> This gives me something I can easily cut/paste to rerun. For example, > >> a common use-case for me while debugging some fix/feature/etc on the > >> driver side, is to diff results between a baseline run and most recent > >> run. And as I debug/fix regressions on driver side, I tend to first > >> want to re-run the set of tests that had gone pass->fail. The old way > >> involved './piglit-summary.py -d baseline latest | grep "pass fail"' > >> then finding the results.json (which is slightly more annoying because > >> of the whole s/@/\// thing) and cut/paste the cmdline. A somewhat time > >> consuming and annoying way to do things. > > > > Just FYI, you dont ned to worry about the 's!/!@!g' thing, -t and -x > > handle that automagically ;) > > hmm, well usually for the 'rerun things that regressed to see if I > fixed it' pass I'm not using full piglit run w/ -t/-x but just fishing > out the cmdlines to run them one by one.. > > for example, frequently I re-run them one by one w/ before/after mesa > + cmdstream logging and diff the cmdstream.. > > >> > >> There is still the slight problem of how to escape special chars in > >> piglit cmdline. Seriously, cmdline args like "*Lod" are a horrible > >> idea. > >> --- > >> fwiw, example output: http://hastebin.com/raw/pezotoyoje > >> > >> framework/results.py | 11 +++++++++++ > >> framework/summary/console_.py | 33 ++++++++++++++++++++++++++++++--- > >> 2 files changed, 41 insertions(+), 3 deletions(-) > >> > >> diff --git a/framework/results.py b/framework/results.py > >> index eeffcb7..fa43cf6 100644 > >> --- a/framework/results.py > >> +++ b/framework/results.py > >> @@ -308,6 +308,17 @@ class TestrunResult(object): > >> except KeyError: > >> raise e > >> > >> + def get_result_object(self, key): > >> + """Similar to get_result() but returns result object""" > >> + try: > >> + return self.tests[key] > >> + except KeyError as e: > >> + name, test = grouptools.splitname(key) > >> + try: > >> + return self.tests[name] > >> + except KeyError: > >> + raise e > >> + > >> def calculate_group_totals(self): > >> """Calculate the number of pases, fails, etc at each level.""" > >> for name, result in self.tests.iteritems(): > >> diff --git a/framework/summary/console_.py b/framework/summary/console_.py > >> index d219498..ebc7adb 100644 > >> --- a/framework/summary/console_.py > >> +++ b/framework/summary/console_.py > >> @@ -89,10 +89,37 @@ def _print_summary(results): > >> > >> def _print_result(results, list_): > >> """Takes a list of test names to print and prints the name and > >> result.""" > >> + # Setup a hashtable mapping transition (ie. 'pass -> fail' to list of > >> + # result objects for test that followed that transition (ie. first > >> + # result is 'pass' and second result is 'fail'). This could of course > >> + # mean transition strings like 'pass -> fail -> pass' if there were > >> + # three sets of results. I guess the normal use case would be to > >> + # compare two sets of results. > >> + # > >> + # Note that we just keep the last result object, but it is expected > >> + # that command/environment are the same across piglit runs. > >> + groups = {} > >> for test in sorted(list_): > >> - print("{test}: {statuses}".format( > >> - test=grouptools.format(test), > >> - statuses=' '.join(str(r) for r in results.get_result(test)))) > >> + transition = None > >> + last = None > >> + for each in results.results: > >> + status = str(each.get_result(test)) > >> + if transition is None: > >> + transition = status > >> + else: > >> + transition = ' -> '.join([transition, status]) > >> + last = each > >> + result = last.get_result_object(test) > >> + if not transition in groups: > >> + groups[transition] = [] > >> + groups[transition].append(result) > >> + > >> + # And now print out results grouped by transition. > >> + for transition, resultlist in groups.iteritems(): > >> + print(transition + ':') > >> + for result in resultlist: > >> + print(result.environment + ' ' + result.command) > >> + print('') > >> > >> > >> def console(results, mode): > >> -- > >> 2.5.0 > >> > >> _______________________________________________ > >> Piglit mailing list > >> Piglit@lists.freedesktop.org > >> http://lists.freedesktop.org/mailman/listinfo/piglit > > > > Hi Rob, > > > > Somewhere I have a patch that automates this, you point it at two > > results and then give it a set of statuses and it reruns the tests that > > match. Something like: > > > > piglit rerurn res1 res2 new pass fail,warn,crash > > > > And it gives you a new result with just those changes. Is that what > > you're looking for? > > Interesting.. > > The big problem is the time it takes to slurp in previous run results > is non-trivial.. although I did have this dream-idea of a feature of a > sort of curses based results browser where I could filter on (for > example) pass->fail or pass->*, and then re-run the selected set (and > then iterate that as I debug/fix things on driver side without exiting > result browser, so without having to re-load results at each step) > > That said, a small modification of what you have to dump out a list of > cmdlines+env would accomplish more or less the same thing in a less > flashy way ;-) > > BR, > -R
That sounds more like what you want is an extension to the piglit-print-commands.py script, which takes a profile and prints all of the command line arguments for each test in that profile. Dylan
signature.asc
Description: PGP signature
_______________________________________________ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
