Hello, * NightStrike wrote on Mon, May 18, 2009 at 09:41:27PM CEST: > What's the difference between using this: > http://www.gnu.org/software/autoconf/manual/autoconf.html#Using-Autotest > > and this: > http://www.gnu.org/software/automake/manual/automake.html#Tests > > ?
Autotest is a generated portable shell script, similar to configure, that you can, ideally, distribute stand-alone, and built with a set of m4 macros like configure.ac is (but a slightly different set). Automake TESTS support, as of version 1.11, consists of two different test drivers that run tests from within a makefile. The old one is pretty minimalistic in test suite functionality, the new one (selected with the `parallel-tests' option) is mostly backward-compatible, but also offers some more and slightly changed functionality. It still operates as a set of rules within a makefile. DejaGNU (which you didn't mention) is a separate package that provides yet another, quite elaborate, test suite framework. It is quite powerful, but may be overkill for simpler uses. YMMV. > Which is better to use? Good question. The general answer is "it depends", I would say, > Which will be maintained, It is intended that all of them be maintained. > and more future-proof? I cannot answer that well, really. I do hope though that future-proofness should not be a deciding point. > Why is there duplication between two products which are very > inter-related? Because these test suite frameworks grew separately, and with slightly different purposes, and slightly different strengths and weaknesses. For example, DejaGNU works very well with complex setups where a program (say, a compiler) needs to be run on many different files, as in the GCC test suite. Being Tcl, is way more efficient than a bunch of shell scripts with lots of forks. OTOH, Autotest and Automake TESTS have less software requirements. Autotest has quite some overhead due to aiming to be in portable shell. The test structure is a bit more rigid when compared to Automake TESTS, but also initialization and cleanup handling of test group directories is all built-in and requires practically no developer intervention. At the moment, there is experimental support for parallel execution on some systems. Automake TESTS has little initialization and cleanup handling, that typically needs to be open-coded by the developer. The parallel-tests driver supports efficient parallel test execution through `make' level parallelism, and it supports test wrappers based on file name extension and such. It is not as shell-centered as Autotest. All frameworks have some level of summary reports. It really depends upon the kind of testing that you would like to do, and I would choose different of these frameworks based on the task at hand. I've also omitted lots of features in this message. Cheers, Ralf
