On Thu, 17 Jan 2013, Stefano Lattarini wrote:
(I know future is to use TAP log protocol, but that's not an excuse
to break working package tests.)
Actually, TAP makes sense in a lot of situations, but not always,
an maybe not even most of the time. First, several TAP tests in
a single script are usually run serially (most TAP-producing libraries
out there are not multi-threaded), and on fast and many-cores machines
this can be a real performance killer if you implement your testsuite
in just a handful of TAP scripts. In addition, TAP is overkill for
many testing scenarios, not worth the trouble. The good thing of the
parallel Automake harness is that it allows you to freely mix "classic"
test scripts and TAP based test script, allowing you to use the best
of both worlds (Automake' own test suite makes an heavy use of this
capability).
My own experience with TAP is that the per-test overhead was reduced
so that the tests actually ran considerably faster (particularly under
Microsoft Windows). If your test software runs really slow, then the
experience may be different. There is a lot of overhead in the
parallel-tests framework.
For parallel builds, just make sure that you have enough independent
TAP test scripts to keep the system busy when running parallel tests.
Only very CPU-bound single-threaded tests will be sped-up much by the
parallel-tests framework. If your software is already threaded, then
parallel-tests may lengthen the test time due to creating too many
threads on the system.
Of course, TAP does not solve the backward-compatibility problem.
Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
