Yes, the output of dotest.py goes through LitTestCommand parse. The parser is matching for "XPASS", but dotest output is using "UNEXPECTED SUCCESS". :)
Thanks, Ying On Tue, Oct 20, 2015 at 6:34 PM, Todd Fiala <todd.fi...@gmail.com> wrote: > Hi Ying, > > Our dotest.py lldb test results go through that lit test parser system? I > see XPASS happen frequently (and in fact is my whole reason for starting a > thread on getting rid of flakey tests, or making them run enough times so > that their output can be a useful signal rather than useless). According > to this script, an XPASS would be listed as failure. I'm not seeing us > treat XPASS as failures AFAICT. > > Are we just saying that our gtests get processed by that? > > -Todd > > On Tue, Oct 20, 2015 at 4:51 PM, Ying Chen via lldb-dev < > lldb-dev@lists.llvm.org> wrote: > >> Hi Zachary, >> >> The big unknown here is how to make the buildbots understand unit test >>> failures and trigger a failure when ninja check-lldb-unit fails. >>> >> >> There're two conditions buildbot will identity a test step as failure. >> One is that the command has non-zero return code. >> The other is that there're failing codes in stdout message. (Refer to >> LitTestCommand::evaluateCommand in this file >> <https://github.com/llvm-mirror/zorg/blob/master/zorg/buildbot/commands/LitTestCommand.py> >> .) >> Failing codes are defined as: >> failingCodes = set(['FAIL', 'XPASS', 'KPASS', 'UNRESOLVED', >> 'TIMEOUT']) >> >> So if the failures are print out as '^FAIL: (.*) \(.*\)', buildbot will >> understand it's failing even if ninja check-lldb-unit returns 0. >> Or we could add some logic to the above file to handle the output of unit >> test. >> >> Thanks, >> Ying >> >> _______________________________________________ >> lldb-dev mailing list >> lldb-dev@lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >> >> > > > -- > -Todd >
_______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
