On Thu, 2015-04-02 at 11:02 -0700, John Stultz wrote: > On Thu, Apr 2, 2015 at 3:18 AM, Prarit Bhargava <pra...@redhat.com> wrote: > > On 03/26/2015 01:33 PM, Tyler Baker wrote: > >> I realize this may be a good amount of work, so I'd like to help out. > >> Perhaps working John to convert his timer tests to use TAP output > >> would be a good starting point? > > > > John, I could probably do that for you. I'm always willing to give it a > > shot. > > I took a quick look into it, since I'm definitely interested in > improving output formatting, but man, TAP is a fairly ugly output > format if you ask me. > > It only has binary "ok" or "not ok" (why not "fail", or something else > that's exclusively grep-able, I don't know). So I'm not sure if cases > where functionality is unsupported should be a pass or fail. > > Most problematically: It seems to want enumeration in the test output > (so test 2 needs to print: "ok 2 Test complete") which means either > there needs to be a wrapper that does the TAP output knowing which > test of N its currently running, or the test number needs to be > submitted as an runtime argument to the test, and the test then has to > add it to its output line.
Yeah TAP is horrible, for the reasons you describe. I *think* in practice most tools will handle just "ok" / "not ok" without the tests being numbered, but I don't know that for sure. For the powerpc tests I'm using "subunit v1" [1], which is basically just: - "^success: <test name>" - "^failure: <test name>" See: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/powerpc/subunit.h So I can do eg: $ cd tools/testing/selftests/powerpc $ make run_tests 2>&1 | subunit-1to2 | subunit-stats --no-passthrough Total tests: 35 Passed tests: 31 Failed tests: 0 Skipped tests: 4 Seen tags: git_version:v4.0-rc7-0-gf22e6e8 But unfortunately TAP has a lot more traction with tools. It wouldn't be too hard to convert the subunit stream into TAP I think, but for some reason no one seems to have written that. Maybe we should, I just haven't had time to do it. cheers [1]: https://github.com/testing-cabal/subunit/blob/master/README#L343 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
