* Baurzhan Ismagulov wrote on Wed, Feb 24, 2010 at 11:11:28PM CET:
> On Wed, Feb 24, 2010 at 08:49:26PM +0100, Ralf Wildenhues wrote:
> > Dunno if you've heard of the parallel-tests option
> ...
> > It also allows you to sort of rerun only tests that are out of date
>
> Wow! Does it work with DejaGNU?
No; it is a replacement for the simple TESTS = ... driver. The
DejaGNU handling in Automake is really separate from that.
> Which automake version is required?
1.11, but you really want 1.11.1 for a small bugfix in the driver, and a
security fix in the automake-generated dist* rules.
> > (once you have formulated dependencies for not only test programs, but
> > also test results)
>
> Can't automake do the former automatically?
Yes.
> Could you give an example use case for the latter?
Say I have lots of tests whose outcome (of course) depends on the test
source and some library being tested, but also on some parameter file.
I would like test compilation to be done as lazily as possible, because
linking take long (think C++ template libraries), and the edit-compile-
test cycle should be as quick as possible.
For completeness, the example also has baz.test which is a shell script.
Say, foo.test and bar.test both are compiled programs and both read
foobar.dat, and when I change the latter, I'd like to see just those
tests being rerun that read the file.
Now, I could achieve that with
make check TESTS='foo.test bar.test'
but for that I'd have to remember which test depends on foobar.dat.
Instead, the Makefile.am has the information, and I just run
make check RECHECK_LOGS=
to only rerun those tests whose results are not up to date. To see the
difference, try something like
make check # run everything once
touch foobar.dat # edit ...
make check RECHECK_LOGS= # baz.test is not rerun
touch foo.c # edit ...
make check RECHECK_LOGS= # bar.test, baz.test are not rerun
or things like
make clean
make -j check # bar.test maybe run before baz.test is linked
make recheck # rerun only failed (FAIL, XPASS) tests
Another example case for dependencies that I haven't shown here is when
some tests need to be run before others, because the latter use data
produced by the former.
Cheers,
Ralf
cat >configure.ac <<\EOF
AC_INIT([parallel-tests example package], [1])
AM_INIT_AUTOMAKE([foreign])
AC_PROG_CC
AC_PROG_RANLIB
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
EOF
cat >Makefile.am <<\EOF
# Enable the parallel-tests driver (could also be done in AM_INIT_AUTOMAKE)
# and colored output. The former is new in 1.11
AUTOMAKE_OPTIONS = 1.11 parallel-tests color-tests
# Some library being tested
LDADD = libfoo.a
noinst_LIBRARIES = libfoo.a
# Test suffixes: Foo.test will be turned into Foo.log by 'make check'
# (rather than Foo.test.log). Specifying suffixes is necessary when
# we want to specifying additional dependencies
TEST_EXTENSIONS = .test
# These are the tests to be run
TESTS = foo.test bar.test baz.test
# Compiled tests go into EXTRA_PROGRAMS not check_PROGRAMS for lazy updating
EXTRA_PROGRAMS = foo.test bar.test
# EXTRA_PROGRAMS need to be cleaned manually, see automake.info(Uniform)
CLEANFILES = $(EXTRA_PROGRAMS)
# Extra data files and scripts we need to distribute
EXTRA_DIST = foobar.dat baz.test
# The tests foo.test and bar.test also read the data file foobar.dat,
# so they should be rerun when that file changes
foo.log: foobar.dat
bar.log: foobar.dat
EOF
touch libfoo.c foobar.dat
cat >foo.c <<\EOF
/* source for foo.test */
int main () { return 0; }
EOF
cat >bar.c <<\EOF
/* source for bar.test */
int main () { return 0; }
EOF
cat >baz.test <<\EOF
#! /bin/sh
exit 0
EOF
autoreconf -vi
./configure
make check # observe baz.test failure
make recheck # only baz.test gets rerun
chmod +x ./baz.test
make recheck # better now
