Oh haha okay. :-) Thanks for explaining, Ying!
-Todd On Wed, Oct 21, 2015 at 10:01 AM, Ying Chen <chy...@google.com> wrote: > Yes, the output of dotest.py goes through LitTestCommand parse. > The parser is matching for "XPASS", but dotest output is using "UNEXPECTED > SUCCESS". :) > > Thanks, > Ying > > On Tue, Oct 20, 2015 at 6:34 PM, Todd Fiala <todd.fi...@gmail.com> wrote: > >> Hi Ying, >> >> Our dotest.py lldb test results go through that lit test parser system? >> I see XPASS happen frequently (and in fact is my whole reason for starting >> a thread on getting rid of flakey tests, or making them run enough times so >> that their output can be a useful signal rather than useless). According >> to this script, an XPASS would be listed as failure. I'm not seeing us >> treat XPASS as failures AFAICT. >> >> Are we just saying that our gtests get processed by that? >> >> -Todd >> >> On Tue, Oct 20, 2015 at 4:51 PM, Ying Chen via lldb-dev < >> lldb-dev@lists.llvm.org> wrote: >> >>> Hi Zachary, >>> >>> The big unknown here is how to make the buildbots understand unit test >>>> failures and trigger a failure when ninja check-lldb-unit fails. >>>> >>> >>> There're two conditions buildbot will identity a test step as failure. >>> One is that the command has non-zero return code. >>> The other is that there're failing codes in stdout message. (Refer to >>> LitTestCommand::evaluateCommand in this file >>> <https://github.com/llvm-mirror/zorg/blob/master/zorg/buildbot/commands/LitTestCommand.py> >>> .) >>> Failing codes are defined as: >>> failingCodes = set(['FAIL', 'XPASS', 'KPASS', 'UNRESOLVED', >>> 'TIMEOUT']) >>> >>> So if the failures are print out as '^FAIL: (.*) \(.*\)', buildbot will >>> understand it's failing even if ninja check-lldb-unit returns 0. >>> Or we could add some logic to the above file to handle the output of >>> unit test. >>> >>> Thanks, >>> Ying >>> >>> _______________________________________________ >>> lldb-dev mailing list >>> lldb-dev@lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >>> >>> >> >> >> -- >> -Todd >> > > -- -Todd
_______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
