[ <http://thread.gmane.org/gmane.comp.sysutils.automake.general/10749> ]
Hello Alejandro, Martin, * aaragon wrote on Fri, May 29, 2009 at 05:44:57AM CEST: > > I found a Makefile.am file on the internet to run tests under a tests > folder. The code looks like follows: [ *snip*, looks an awful lot like <http://www.bioinf.uni-freiburg.de/~mmann/HowTo/automake.html> ] Martin, a while ago I went over your Automake page and noted some issues with it, and proposed a newer version that uses Automake 1.11 features. 1.11 has been released now, would you be so nice to update your web page to reflect these fixes? A few nits inline below, and part of my recommendation follows at the end. It will also help you, Alejandro, avoid the pitfall of letting code be compiled by GNU make implicit rules (which uses different variable names and thus does not pick up the include paths). > check: all You can use check-local; and check will depend upon all-am anyway (the part of 'all' that is updated in this directory), maybe that is sufficient for your needs. > @echo "Running tests" > @make test -s -C $(TSTDIR) Non-GNU make doesn't support -C, and you should use $(MAKE) so that parallel make and make -k and make -n work; you can use cd $(TSTDIR) && $(MAKE) $(AM_MAKEFLAGS) -s test but if I were you, I'd just let the automake-provided check rule do its work in the toplevel directory. > TESTSOURCE = $(wildcard test*.cpp) GNU make-specific. > TESTOBJS=$(TESTSOURCE:%.cpp=%.o) Likewise. More instances below. > # additional linker and compiler flags > LDFLAGS = -L$(top_srcdir)/yafeq -lyafeq $(LDFLAGS) For in-tree libtool libraries, please always specify them as relative paths to the .la file; also, don't stick them in LDFLAGS; thus, use LDADD = $(top_srcdir)/yafeq/libyafeq.la (or even LIBS). > aara...@~/Documents/workspace/yafeq/tests$mt > Compiling IntegrationRules.cpp OK > BD Software STL Message Decryptor v3.10 for gcc 2/3/4 > In file included from testIntegrationRules.cpp:1: > ../yafeq/quadrature.hpp:14:36: error: cpputils/map_adaptor.hpp: No such file > or > directory > > > Now, the file cpputils/map_adaptor.hpp does exist, and I actually have a > test for detecting the cpputils library. I guess the CPPFLAGS that includes > the correct path is not being passed to this piece of code written in the > Makefile.am. What follows is my suggestion that I wrote to Martin about the tests thing. I haven't tested it much, so if you have issues with them, please speak up. Cheers, Ralf For the tests example, I actually have a more radical suggestion: replace it completely with a setup based on automake support for simple tests using the TESTS variable. Since I am lazy, and we have added several new features to the upcoming Automake 1.11 release which can be of help here, I will further suggest to require that next version, and use the 'parallel-tests' test driver which has several new features. In the following, I will also use some more Automake 1.11 features like AM_DEFAULT_SOURCE_EXT. (The stuff can more or less also be done with 1.10 features, but it will be uglier, and, as already noted, I am lazy. :-) However, for portability, you will not be able to use wildcards. So you will need to mention each test once. BTW, in your example is a line of the form | LDFLAGS = $(LDFLAGS) If this really were in your code, then it would provoke an endless recursion, and an error from 'make' if referenced anywhere. | # additional linker and compiler flags | CXXFLAGS = -I$(top_srcdir)/src $(CXXFLAGS) You should not override CXXFLAGS: that variable is reserved for the user. Automake provides AM_CXXFLAGS for that case. However, include paths are for the preprocessor, so AM_CPPFLAGS should be used here. With the tests/Makefile.am and the test driver files below, there is no need to modify toplevel Makefile.am, and you can run the testsuite with make check or even make -j3 check if you so prefer. :-) A word of caution: this code is completely untested. Another word of caution: you need to run make generate-verified-files once before you can run "make dist". If you do not like that kind of setup, please speak up. (The dependencies are not completely precise yet.) You can update individual .verified files with something like make generate-verified-files TESTS='test1 test2' Please note that 'diff -w' is not portable, at least not POSIX. I simply used diff in the driver, but you may want to make that configurable or so. Cheers, Ralf --- tests/diff-driver --- #! /bin/sh # Compare output of test programs with expected output. # Usage: diff-driver [--create] TEST-PROGRAM # # TEST-PROGRAM is the program to run. # With --create, a .verified file is created. # Without it, a .lastout file is created and compared # to an existing .verified file; the diff is produced on stdout. # $srcdir should contain the path to the source directory. # # .lastout files are created in the build tree; the TEST-PROGRAM # may or may not live in the source tree; an existing .verified # file may live in the source tree (if it came with the distribution) # or the build tree, but is updated in the build tree. if test -z "$srcdir"; then echo "$0: \`\$srcdir' is not set" >&2 exit 1 fi create=no for arg do case $arg in --create) create=yes ;; [!-]*) break ;; esac shift done test_program=$1 test_program_noext=${test_program%$EXEEXT} case $test_program_noext in $srcdir/*) srcdirstrip=`echo "$srcdir" | sed 's|.|.|g'` test_stem=`echo "$test_program_noext" | sed "s|^$srcdirstrip/||"` ;; *) test_stem=$test_program_noext ;; esac output_file=$test_stem.lastout verified_file=$test_stem.verified if test "$create" = no && test ! -f "$verified_file"; then verified_file=$srcdir/$verified_file if test ! -f "$verified_file"; then echo "$0: cannot find .verified file \`$verified_file'" >&2 exit 1 fi fi if test "$create" = yes; then $test_program > $verified_file exit $? else $test_program > $output_file || exit $? diff $verified_file $output_file exit $? fi --- tests/Makefile.am --- # We use the new parallel-tests driver from Automake 1.11 # For nice and pretty output, we will be using color, too. AUTOMAKE_OPTIONS = 1.10c parallel-tests color-tests # Each program `foo' is compiled from `foo.cc', not `foo.c'. AM_DEFAULT_SOURCE_EXT = .cc # Add test programs here. # If their sources do not end in .cc, or are made up of more than one # file, then you need to provide a testX_SOURCES variable, too. TESTEXECS = test1 test2 test3 ... # All test executables are to be run by `make check'. TESTS = $(TESTEXECS) # Let automake be aware that all test executables are programs to be # compiled (and $(EXEEXT) appended etc) using the normal PROGRAMS # machinery. We use EXTRA_ instead of check_, because the # parallel-tests driver will cause the programs to be built as normal # prerequisites just in time. EXTRA_PROGRAMS = $(TESTEXECS) AM_CPPFLAGS = -I$(top_srcdir)/src # Tests are run with the diff-driver LOG_COMPILER = ./diff-driver # Run `make generate-verified-files' to create .verified files. # This just runs all tests and passes --create to the diff-driver. generate-verified-files: $(MAKE) $(AM_MAKEFLAGS) AM_LOG_FLAGS=--create check # add verified output files to the distribution EXTRA_DIST = $(TESTEXECS:=.verified) MOSTLYCLEANFILES = $(TESTEXECS:=.lastout) MAINTAINERCLEANFILES = $(TESTEXECS:=.verified)
