[JIRA] [xunit] (JENKINS-18098) xunit does not recognize failed tests anymore when there is a delay between testing and processing of test result files (because of coverage post processing in my case)

[kuypers.d...@googlemail.com (JIRA)]

Dirk Kuypers created JENKINS-18098 xunit does not recognize failed tests anymore when there is a delay between testing and processing of test result files (because of coverage post processing in my case) Issue Type: Bug Assignee: Gregory Boissinot Components: xunit Created: 27/May/13 1:31 PM Description: (I posted this question on the user's mailinglist before) I recently started to use Opencover with my MSTest unit tests to get code coverage results. Since then the xunit plugin does not recognize errors as a Failure anymore. The TRX file is parsed and you can see at the summary page that there are tests failed and since when, but the job is counted as success and no email is triggered. Could it be that there is some delay now introduced inbetween writing the MSTest results file and processing by xunit that causes this behaviour? When I rerun the tests without coverage after processing the coverage results everything works like it should. What I can see from the last versions of xunit plugin and a short look into the source code I think there should be a mechanism that if the test result file is older than 3 seconds there should be an error which can be ignored if configured. But even when the error because of an outdated file is ignored (because I make sure that the workspace is always cleaned after the running the tests) the Jenkins test job should fail if one or more of the unit tests fail. I paste a log and if more info is needed like job config xml etc I will be happy to provide it. 18:44:02 D:\WS\CommonFunctionsTests\ContestContinuous\contest_sw>"C:\Program Files\OpenCover\OpenCover.Console.exe" -target:"C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\Tools\..\IDE\MSTEST.EXE" -output:coverage.xml -targetargs:"/testcontainer:\WS\CommonFunctionsTests\ContestContinuous\contest_sw\Bin\Debug\CommonFunctionsTests.dll /testsettings:\WS\CommonFunctionsTests\ContestContinuous\contest_sw\localtestrun.testsettings /resultsfile:\WS\CommonFunctionsTests\ContestContinuous\contest_sw\TestResults\CommonFunctionsTests\CommonFunctionsTests.trx" -mergebyhash -targetdir:"Bin\Debug" -filter:+[CommonFunctions*]* 18:44:03 Executing: C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTEST.EXE 18:44:06 Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1 18:44:06 Copyright (c) Microsoft Corporation. All rights reserved. 18:44:06 18:44:06 Loading D:\WS\CommonFunctionsTests\ContestContinuous\contest_sw\localtestrun.testsettings... 18:44:07 Loading D:\WS\CommonFunctionsTests\ContestContinuous\contest_sw\Bin\Debug\CommonFunctionsTests.dll... 18:44:08 Starting execution... 18:44:18 18:44:18 Results Top Level Tests 18:44:18 ------- --------------- 18:44:18 Passed CommonFunctionsTest.BufferedTraceListenerTest.BufferedTraceListenerConstructorTest [... Passes removed ...] 18:44:31 Passed CommonFunctionsTest.CommonFunctionsTest.ResizeTwoDimensionalArrayTest2 18:44:31 Failed CommonFunctionsTest.ContestConfigurationParametersTest.ContestConfigurationParametersFileTest 18:44:31 Passed CommonFunctionsTest.ContestSecurityStreamTest.ContestSecurityStreamConstructorTest [... Passes removed ...] 18:44:59 Passed TestcaseInformationTest.TestcaseXmlElementTest.TestcaseXmlElementConstructorTest2 18:45:00 216/217 test(s) Passed, 1 Failed 18:45:00 18:45:00 Summary 18:45:00 ------- 18:45:00 Test Run Failed. 18:45:00 Passed 216 18:45:00 Failed 1 18:45:00 ----------- 18:45:00 Total 217 18:45:00 Results file: D:\WS\CommonFunctionsTests\ContestContinuous\contest_sw\TestResults\CommonFunctionsTests\CommonFunctionsTests.trx 18:45:00 Test Settings: localtestrun 18:45:00 Committing... 18:45:00 Visited Classes 70 of 114 (61.4) 18:45:00 Visited Methods 588 of 1049 (56.05) 18:45:00 Visited Points 5763 of 12043 (47.85) 18:45:00 Visited Branches 1629 of 4073 (40.00) 18:45:00 18:45:00 ==== Alternative Results (includes all methods including those without corresponding source) ==== 18:45:00 Alternative Visited Classes 70 of 127 (55.12) 18:45:00 Alternative Visited Methods 676 of 1266 (53.4) 18:45:00 18:45:00 D:\WS\CommonFunctionsTests\ContestContinuous\contest_sw>"C:\Program Files\ReportGenerator_1.8.1.0.\bin\ReportGenerator.exe" -reports:coverage.xml -targetdir:coverage 18:45:01 Loading report 'D:\WS\CommonFunctionsTests\ContestContinuous\contest_sw\coverage.xml' 18:45:01 Preprocessing report 18:45:02 Indexing classes in directory 'c:\Snapshots\ContestContinuous\contest_sw\CommonFunctions\CommonFunctions\' 18:45:03 Added coverage information of 86/90 auto properties to module 'CommonFunctions' 18:45:03 Initiating parser for OpenCover 18:45:03 Current Assembly: CommonFunctions 18:45:03 Current Assembly: CommonFunctionsTests 18:45:03 Analyzing 89 classes 18:45:03 Creating report 1/89 (Assembly: CommonFunctionsTests, Class: CommonFunctionsTest.CommonFunctionsTest) [....] 18:45:04 Creating report 84/89 (Assembly: CommonFunctions, Class: TSTraceOrigin) 18:45:04 Creating summary 18:45:04 Report generation took 3 seconds 18:45:04 Archiving artifacts 18:45:04 [htmlpublisher] Archiving HTML reports... 18:45:04 [htmlpublisher] Archiving at BUILD level D:\WS\CommonFunctionsTests\ContestContinuous\contest_sw\coverage to D:\Jenkins\jobs\CommonFunctionsTests\builds\2013-05-24_18-43-55\htmlreports\Coverage_Report 18:45:06 [xUnit] [INFO] - Starting to record. 18:45:06 [xUnit] [INFO] - Processing MSTest-Version N/A (default) 18:45:06 [xUnit] [INFO] - [MSTest-Version N/A (default)] - 1 test report file(s) were found with the pattern 'ContestContinuous\contest_sw\TestResults\CommonFunctionsTests\CommonFunctionsTests.trx' relative to 'D:\WS\CommonFunctionsTests' for the testing framework 'MSTest-Version N/A (default)'. 18:45:06 [xUnit] [INFO] - Check 'Failed Tests' threshold. 18:45:06 [xUnit] [INFO] - Check 'Skipped Tests' threshold. 18:45:06 [xUnit] [INFO] - Setting the build status to SUCCESS 18:45:06 [xUnit] [INFO] - Stopping recording. 18:45:06 Recording fingerprints 18:45:06 No emails were triggered. 18:45:06 18:45:06 Deleting project workspace... 18:45:18 done 18:45:18 18:45:18 Notifying upstream projects of job completion 18:45:18 Finished: SUCCESS Environment: xunit 1.58, Jenkins 1.515 Project: Jenkins Priority: Major Reporter: Dirk Kuypers

This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira

--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.