Hi Ying, Our dotest.py lldb test results go through that lit test parser system? I see XPASS happen frequently (and in fact is my whole reason for starting a thread on getting rid of flakey tests, or making them run enough times so that their output can be a useful signal rather than useless). According to this script, an XPASS would be listed as failure. I'm not seeing us treat XPASS as failures AFAICT.
Are we just saying that our gtests get processed by that? -Todd On Tue, Oct 20, 2015 at 4:51 PM, Ying Chen via lldb-dev < lldb-dev@lists.llvm.org> wrote: > Hi Zachary, > > The big unknown here is how to make the buildbots understand unit test >> failures and trigger a failure when ninja check-lldb-unit fails. >> > > There're two conditions buildbot will identity a test step as failure. > One is that the command has non-zero return code. > The other is that there're failing codes in stdout message. (Refer to > LitTestCommand::evaluateCommand in this file > <https://github.com/llvm-mirror/zorg/blob/master/zorg/buildbot/commands/LitTestCommand.py> > .) > Failing codes are defined as: > failingCodes = set(['FAIL', 'XPASS', 'KPASS', 'UNRESOLVED', 'TIMEOUT']) > > So if the failures are print out as '^FAIL: (.*) \(.*\)', buildbot will > understand it's failing even if ninja check-lldb-unit returns 0. > Or we could add some logic to the above file to handle the output of unit > test. > > Thanks, > Ying > > _______________________________________________ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > > -- -Todd
_______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
