Dear Wiki user, You have subscribed to a wiki page or wiki category on "Ant Wiki" for change notification.
The following page has been changed by SteveLoughran: http://wiki.apache.org/ant/Proposals/EnhancedTestReports The comment on the change is: some ideas ------------------------------------------------------------------------------ 1. Scales well to lots of JUnit tests on a single process 1. Integrates with CI servers 1. Produced by lots of tools : <junit>, antunit, maven surefire, testng - 1. Consumed by: <junitreport>, CruiseControl, Luntbuild, Bamboo, Hudson + 1. Consumed by: <junitreport>, CruiseControl, Luntbuild, Bamboo, Hudson, IntelliJ TeamCity, AntHill 1. Includes log output, JVM properties, test names 1. Reasonable HTML security (we sanitise all output but test names in the HTML) @@ -60, +60 @@ Add the names of people actively participating - * Ant team (ant-dev) + * Ant team (ant-dev) (SteveLoughran to do the prototyping) - * SmartFrog team (SteveLoughran) + * SmartFrog team (SteveLoughran) - * TestNG: AlexandruPopescu + * TestNG: AlexandruPopescu - * Maven SureFire: DanFabulich, BrettPorter + * Maven SureFire: DanFabulich, BrettPorter == Plan and Timetable == @@ -77, +77 @@ 1. add junit4 test runner. Let TestNG, SmartFrog do their prototyping 1. ship! + == Ideas == + + === Improved Logging information === + + * Retain the current stdout+stderr logs, but also allow people to bind to custom log4j/commons-logging/java.util.logging back ends that grab the raw events and log them as structured XML events (host,process,thread,level,timestamp,text). These would be renderable at different levels + + === Improved Host information === + + As well as logging the jvm properties, grab other bits of system state + * wall time and time zone + * Ant-determined OS (using standard names) + * environment variables on java5+ + The risk here is that we could massively increase XML file size by duplicating all this stuff for every test. We may only need this per test suite. + + == Improved Fault Information === + + Java faults should be grabbed and their (recusrsive) stack traces extracted into something that can be zoomed in on + + <throwable classname="java.lang.Throwable"> + <message>text here</message> + <stack> + <stackrow>org.example.something.Classlet:53</stackrow> + <stackrow>org.example.sp2.Main:36</stackrow> + </stack> + <nested> + <throwable classname="java.remote.RmiException"> + ... + </throwable> + </nested> + </throwable> + + This would be better placed for display on demand operations; other postmortem activities. We should still render the fault in the existing format for IDEs to handle (e.g. IDEA's "analyze stack trace" feature) + + == Open ended failure categories == + + success and skipped are fixed, but there are many forms of failure (error, failure, timeout, out of performance bounds, and others that depend on the framework itself) Can we enumerate all possible categories for failure, or could we make it extensible? + + == Add partial-failure, not-yet-finished as test outcomes == + + * not-yet-finished is for tests in progress; makes sense if you are doing a live view of the system on a continuously updated web site. + * partial-failure is when you run tests on lots of machines in the style of grid unit, and some tests fail on some of the hosts, but not others. These are the troublesome tests, as they are hardest to replicate. It's an interesting out come as it is not one you can infer from a single test -its the result of aggregating the same test over different machines + + == Extend the viewer == + + * for every test, test case, show the description. Should descriptions be plain text only? + * list skipped tests + * list tests by tag + * list tests by execution time + * list tests that are partial failures, unfinished + * for tests that are published to a web site, integrate with searching (allow a search box in the HTML) + * list failing tests by exception name + * For live output, provide an ongoing status page that updates regularly. + * allow test results to include links to images, videos that are then embedded inline. Your test runs, on youtube! + --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
