Hello Gary,
you wanted a way to test old Autoconf and Automake with a git Libtool
tree:
* Gary V. Vaughan wrote on Sun, Aug 22, 2010 at 05:41:52PM CEST:
> On 22 Aug 2010, at 22:30, Ralf Wildenhues wrote:
> > * Gary V. Vaughan wrote on Sun, Aug 22, 2010 at 05:24:05PM CEST:
> >> Of course, that doesn't test the patch series you haven't
> >> pushed yet...
> >
> > Just add 'git checkout origin/parallel-tests'.
>
> I meant that if I want to check that my next round of m4sh patches
> don't break with older Autotools before I push, this script doesn't
> help, since it wants a fresh libtool checkout. The script would
> be more useful if it can be run inside a dirty working directory.
>
> >> And running it as an Autotest inside the existing tree would
> >> get us better coverage, and obviate the need for a separate
> >> checkout of the libtool tree entirely.
> >
> > Sure. But look at the existing sort-of-recursive tests. They are
> > tricky. I am waiting for the day where we add infinite recursion in
> > some corner case and have developers complain.
>
> Actually, I've not looked at the details of those recursive tests,
> so I must admit that I don't know how tricky they are. I just
> assumed that we could reuse that pattern here.
>
> > The net is abundant now, I think such test helper scripts are a real
> > easy way to get more coverage, but I don't think it is necessary to
> > run them as part of each testsuite run, because they are very expensive.
>
> A very good point. Maybe we should add a 'maintainer-test' target or
> similar, which executes this and other expensive test-helper scripts
> in addition to the regular testsuite(s). And it's also worth considering
> migrating our other recursive tests to helper scripts too in that case.
I haven't done that yet; but in the end, that might actually be a better
idea than what I hacked up for now:
This patch checks whether $AUTOCONF and $AUTOMAKE point to the oldest
versions we aim to support, otherwise downloads repective release
tarballs, builds and installs them temporarily below a subdirectory in
the test group, then runs those parts of the new testsuite recursively
which use autoconf. The old testsuite is not dealt with.
While debugging the patch, I did run into an endless recursion or two.
The test is fragile in that it might need updating when we add more
environment variables that influence general test behavior.
At this point, I'm not actually asking to commit this now. Consider it
a test balloon that you could use in your git tree to test things. :-)
Cheers,
Ralf
>From e9930b5cf137500b2c08ceb3e3819dfbc6404be0 Mon Sep 17 00:00:00 2001
From: Ralf Wildenhues <ralf.wildenh...@gmx.de>
Date: Sun, 12 Sep 2010 12:45:44 +0200
Subject: [PATCH] Try out new testsuite with old Autoconf and Automake versions.
* tests/old-autotools.at (old autoconf and automake): New file,
new test.
* Makefile.am (TESTSUITE_AT): Update.
* doc/libtool.texi (Test descriptions): Document new Autotest
keywords `expensive' and `net-access'.
* HACKING: Likewise; also document `interactive'.
Suggested by Gary V. Vaughan.
Signed-off-by: Ralf Wildenhues <ralf.wildenh...@gmx.de>
---
ChangeLog | 11 +++++++++++
HACKING | 7 +++++--
Makefile.am | 1 +
doc/libtool.texi | 9 +++++++++
4 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 4584882..d6fb995 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2010-09-12 Ralf Wildenhues <ralf.wildenh...@gmx.de>
+
+ Try out new testsuite with old Autoconf and Automake versions.
+ * tests/old-autotools.at (old autoconf and automake): New file,
+ new test.
+ * Makefile.am (TESTSUITE_AT): Update.
+ * doc/libtool.texi (Test descriptions): Document new Autotest
+ keywords `expensive' and `net-access'.
+ * HACKING: Likewise; also document `interactive'.
+ Suggested by Gary V. Vaughan.
+
2010-09-12 Jürgen Reuter <juergen.reu...@physik.uni-freiburg.de> (tiny
change)
Ralf Wildenhues <ralf.wildenh...@gmx.de>
diff --git a/HACKING b/HACKING
index b462c98..277a011 100644
--- a/HACKING
+++ b/HACKING
@@ -65,11 +65,14 @@ and is not part of a release distribution.
* The new Autotest testsuite uses keywords to denote test features:
autoconf needs Autoconf
automake needs Automake
+ expensive very expensive test (only enabled if the environment
+ variable RUN_EXPENSIVE_TESTS is set)
+ interactive might cause popup windows from the system
libltdl exercises the `libltdl' library
libtool exercises the `libtool' script
libtoolize exercises the `libtoolize' script
- recursive runs the suite recursively, with a modified
- `libtool' script and with `-k libtool'
+ net-access might download files from the GNU website
+ recursive runs the suite recursively
CXX F77 FC GCJ exercises a language other than C
diff --git a/Makefile.am b/Makefile.am
index dcd0876..ff8a225 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -499,6 +499,7 @@ TESTSUITE_AT = tests/testsuite.at \
tests/stresstest.at \
tests/cmdline_wrap.at \
tests/pic_flag.at \
+ tests/old-autotools.at \
tests/darwin.at \
tests/deplibs-mingw.at \
tests/sysroot.at
diff --git a/doc/libtool.texi b/doc/libtool.texi
index 9314612..91f98e8 100644
--- a/doc/libtool.texi
+++ b/doc/libtool.texi
@@ -5463,6 +5463,11 @@ by the test group. The tests are typically skipped if
the program is
not installed. The @samp{automake} keyword may also denote use of the
@command{aclocal} program.
+...@item expensive
+This test group is very expensive, and skipped unless the environment
+variable @env{RUN_EXPENSIVE_TESTS} is also set. Do not do this unless
+you know what you are doing!
+
@item interactive
This test group may require user interaction on some systems. Typically,
this means closing a popup window about a DLL load error on Windows.
@@ -5475,6 +5480,10 @@ Denote that the @file{libltdl} library is exercised by
the test group.
Denote that the @command{libtool} or @command{libtoolize} scripts are
exercised by the test group, respectively.
+...@item net-access
+This test group might access the internet to download files from GNU
+servers.
+
@item recursive
Denote that this test group may recursively re-invoke the test suite
itself, with changed settings and maybe a changed @command{libtool}
--
1.7.2.1.222.g9988
