Dear Wiki user, You have subscribed to a wiki page or wiki category on "Ant Wiki" for change notification.
The following page has been changed by SteveLoughran: http://wiki.apache.org/ant/Proposals/EnhancedTestReports The comment on the change is: charting ------------------------------------------------------------------------------ SteveLoughran: True. But I'm thinking about better XSL support. Also the standard exception dumps don't scale well to deeply chained exceptions; gets hard to see what is going on. - == Open ended failure categories == + === Open ended failure categories === success and skipped are fixed, but there are many forms of failure (error, failure, timeout, out of performance bounds, and others that depend on the framework itself) Can we enumerate all possible categories for failure, or could we make it extensible? @@ -148, +148 @@ SteveLoughran: I like failure itself to be categorised. So every state can be a pass state, a warning or a failure. Warnings could include "passing, but took 50% longer than usual". Failures can include "tests passed but outside allowed time" "tests passed but memory consumption was over the limit", as well as simple "some assertion failed". Maybe we decouple result name from outcome, with a limited set of outcomes, but an unlimited set of outcome categories. - == Add partial-failure, not-yet-finished as test outcomes == + === Add partial-failure, not-yet-finished as test outcomes === * not-yet-finished is for tests in progress; makes sense if you are doing a live view of the system on a continuously updated web site. * partial-failure is when you run tests on lots of machines in the style of grid unit, and some tests fail on some of the hosts, but not others. These are the troublesome tests, as they are hardest to replicate. It's an interesting out come as it is not one you can infer from a single test -its the result of aggregating the same test over different machines - == Extend the viewer == + === Extend the viewer === * for every test, test case, show the description. Should descriptions be plain text only? Needs to support UTF8 at least. * list skipped tests @@ -165, +165 @@ * For live output, provide an ongoing status page that updates regularly. * allow test results to include links to images, videos that are then embedded inline. Your test runs, on youtube! - == Publish under atom == + === Publish under atom === * Allow an individual test to be served as a single Atom entry. This may require xmlns games. Perhaps we could define an optional xmlns for test results; if served through atom this would be required. - == test results as wire format == + === test results as wire format === Could we use a test result as a wire format for exchanging the outcome of a single test between processes/hosts? Rather than just something for XSL and other reporting tools to handle. @@ -184, +184 @@ This may be something for things other than Ant to consider. - == Add support for artifacts == + === Add support for artifacts === * Formalise the the idea that every test run/test case can include artifacts off the results. * artifacts to include inline SVG content and linked images @@ -192, +192 @@ * viewers/atom feeds could embed SVG content and images, other content would be turned into links. * relative links are best for publishing reports that are HTML viewed - == Integration with Issue Trackers == + === Integration with Issue Trackers === What would this look like? Perhaps and element with URLs to related issues? + + + === Generate Google Charts === + + The Google [http://code.google.com/apis/chart/ Charts] API dynamically creates a PNG image from an appropriate URL. We could use an XSL transform to create a result image showing + + * A pie chart of outcomes + * history charting + + History analysis really needs full history logging and is something that some CI tools already do; its an analysis service, not something the XML/XSL files can do. == Examples of existing tests == --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
