Re: use of make in AM_SANITY_CHECK (was: improved timestamp resolution test)

2024-06-16 Thread Karl Berry
make(1) in AM_SANITY_CHECK seems to be a logic error, since the user may want to build with a different $MAKE, You're right. Crap. It never ends. In practice it probably doesn't matter, though. Although in theory one can imagine that "make" succeeds while $MAKE fails, resulting in a fals

Re: use of make in AM_SANITY_CHECK (was: improved timestamp resolution test)

2024-06-15 Thread Jacob Bachmeyer
ced a logic error in the current code. When _AM_FILESYSTEM_TIMESTAMP_RESOLUTION was introduced in commit 720a1153134b833de9298927a432b4ea266216fb, it did not use make. Commit 23e69f6e6d29b0f9aa5aa3aab2464b3cf38a59bf introduced the use of make in that test to work around a limitation on MacOS, but

Re: End of life dates vs regression test matrix

2024-06-14 Thread Nick Bowler
2.0" never materialized (probably won't) and the depcomp script today still has the code to handle SGI compilers (Sadly, I don't currently have an SGI/IRIX setup in my collection to test this). But even if it were removed automatic dependency tracking is not an important feature: pac

Re: End of life dates vs regression test matrix

2024-06-14 Thread Dan Kegel
Oh, ok, perhaps I was confused by the note in automake 1.13.1's NEWS file (iirc), which said Support for IRIX and the SGI C/C++ compilers will be removed in Automake 1.14: they have seen their last release in 2006, and SGI is expected to retire support from them in December 2013 Did that not

Re: End of life dates vs regression test matrix

2024-06-14 Thread Bob Friesenhahn
Does automake have a policy on when to stop supporting a CPU, operating system, or compiler? Automake does not have specific support for anything so this concept is utterly alien to it. Libtool does have specific support for certain targets/tools, and Automake may optionally make use of i

Re: End of life dates vs regression test matrix

2024-06-13 Thread Jacob Bachmeyer
Dan Kegel wrote: Does automake have a policy on when to stop supporting a CPU, operating system, or compiler? I am pondering the size of the matrix of supported operating systems, cpus, and compilers, and wonder where a policy like "Automake drops support 20 years after the release of a CPU, ope

End of life dates vs regression test matrix

2024-06-13 Thread Dan Kegel
Does automake have a policy on when to stop supporting a CPU, operating system, or compiler? I am pondering the size of the matrix of supported operating systems, cpus, and compilers, and wonder where a policy like "Automake drops support 20 years after the release of a CPU, operating system, or c

Re: improved timestamp resolution test (was: 1.16.90 regression: configure now takes 7 seconds to start)

2024-06-13 Thread Karl Berry
Jacob, [*sigh*] You said it. About this whole thing. I rather wish this bright idea had never come to pass. It has delayed the release by months. Oh well. Still, could we use make(1) for *all* of the testing and not use `ls -t` I guess it is technically possible, but I somehow feel dou

Re: improved timestamp resolution test (was: 1.16.90 regression: configure now takes 7 seconds to start)

2024-06-12 Thread Jacob Bachmeyer
This guess can be one step too fast, if the shorter delay just # happened to span a clock tick boundary. am_resolution_guess=`make -f conftest.mk conftest.ts0 | tail -1` case $am_resolution_guess in *9) i=no for am_try_res in $am_try_resolutions; do if test x$i = xyes; then am_reso

system info in test-suite.log

2024-06-07 Thread Bruno Haible
Hi, The 1.16.90 NEWS file states: - test-suite.log now contains basic system information, and the console message about bug reporting on failure has a bit more detail. (bug#68746) 1) This system information is the naked output of some OS commands: (uname -a | awk '

no easy way to generate a test-suite.log without skipped tests

2024-06-07 Thread Bruno Haible
Hi, In my projects that use Automake and the "parallel" test driver, often I have 1000 unit tests, of which 100-150 are skipped and 1-5 fail. When analyzing the "make check" results, the generated test-suite.log is more handy to use than the 1000 different .log files. Howe

Re: small patch to make a test work with upcoming autoconf

2024-06-02 Thread Jim Meyering
I wrote: > One of autoconf's diagnostics changed a couple of months ago, > and an automake test depended on the precise wording of the > now-changed diagnostic. This change (will push shortly) > makes the automake test work with both the old and the new wording: ... > - grep

small patch to make a test work with upcoming autoconf

2024-06-02 Thread Jim Meyering
One of autoconf's diagnostics changed a couple of months ago, and an automake test depended on the precise wording of the now-changed diagnostic. This change (will push shortly) makes the automake test work with both the old and the new wording: tests: work also with v2.72-9-g7a6347d1 and

Re: purpose of line ".. contents:: :depth: 2" in test-suite.log?

2024-01-28 Thread Russ Allbery
Karl Berry writes: > Does anyone know what this line in automake-generated test-suite.log > files is used by or intended for? > .. contents:: :depth: 2 It's a Sphinx reStructuredText directive that would insert a table of contents that goes two levels deep on headings. I ca

purpose of line ".. contents:: :depth: 2" in test-suite.log?

2024-01-28 Thread Karl Berry
Does anyone know what this line in automake-generated test-suite.log files is used by or intended for? . contents:: :depth: 2 It appears to be machine-parsable, but I can't find any reference to it in the code or the doc. There doesn't seem to be anything that outputs :depth: 3 (or any

Re: [platform-testers] automake-1.16j pretest available: please test

2024-01-05 Thread Frederic Berat
or issue found during my mass rebuild against dependent packages on Fedora-rawhide. LGTM. > We intend for automake 1.17 to be released soon, essentially with only > bug fixes for whatever is found in this pretest. So please do test if at > all possible. > > The main change from 1.1

automake-1.16j pretest available: please test

2023-12-29 Thread Karl Berry
found in this pretest. So please do test if at all possible. The main change from 1.16i was another attempt at making the "subsecond mtime" support reliable and backward compatible. That is, to work with autoconf-2.72 (which supports subsecond mtimes) and earlier versions (which don't)

1.16j: test release incoming

2023-12-27 Thread Jim Meyering
Preparing to make the next test release... This failed: n=50; ( make bootstrap && make -j$n check keep_testdirs=yes && make maintainer-check && make -j$n distcheck && make -j$n distcheck AM_TESTSUITE_MAKE="make -j$n" && make -j$n check-no-t

Re: [platform-testers] automake-1.16i pretest available: please test

2023-12-19 Thread Frederic Berat
On Mon, Dec 18, 2023 at 10:32 PM Karl Berry wrote: > We are pleased to announce the GNU Automake 1.16i development snapshot. > We intend for automake 1.17 to be released soon, essentially with only > bug fixes for whatever is found in this pretest. So please do test if at >

automake-1.16i pretest available: please test

2023-12-18 Thread Karl Berry
We are pleased to announce the GNU Automake 1.16i development snapshot. We intend for automake 1.17 to be released soon, essentially with only bug fixes for whatever is found in this pretest. So please do test if at all possible. It's our primary goal to preserve compatibility. If this relea

Re: rhel8 test failure confirmation?

2023-12-05 Thread Jacob Bachmeyer
Zack Weinberg wrote: On Mon, Dec 4, 2023, at 7:14 PM, Jacob Bachmeyer wrote: Zack Weinberg wrote: [snip everything addressed in the other thread] Yes, there was a bit of confusion here; not only is the FileUtils module synchronized between autom4te and automake Thanks for rem

Re: rhel8 test failure confirmation?

2023-12-05 Thread Zack Weinberg
On Mon, Dec 4, 2023, at 7:14 PM, Jacob Bachmeyer wrote: > Zack Weinberg wrote: [snip everything addressed in the other thread] > Yes, there was a bit of confusion here; not only is the FileUtils > module synchronized between autom4te and automake Thanks for reminding me that I need to make sure al

Re: rhel8 test failure confirmation?

2023-12-04 Thread Jacob Bachmeyer
sion output. The implementation is just BEGIN { our $subsecond_timestamps = 0; eval { require Time::HiRes; import Time::HiRes qw(stat); $subsecond_timestamps = 1; } } Jacob, can you confirm that's an accurate test, given all the things you said earlier about wa

Re: rhel8 test failure confirmation?

2023-12-04 Thread Karl Berry
thing more convenient than the long multi-month thread on the automake discussion list (which I pointed to from that bug). Jacob, can you confirm that's an accurate test, +1 for Jacob's confirmation being welcome :).

Re: rhel8 test failure confirmation?

2023-12-04 Thread Zack Weinberg
ond-resolution timestamps *without* loss of precision due to conversion to an NV (aka IEEE double) we could, in the future, look for that first. The implementation is just BEGIN { our $subsecond_timestamps = 0; eval { require Time::HiRes; import Time::HiRes qw(stat); $subsec

Re: rhel8 test failure confirmation?

2023-12-03 Thread Jacob Bachmeyer
inks. I lean towards Jacob's view that automake --version | grep HiRes will suffice. Not having a new option seems simpler/better in terms of later understanding, too. --thanks, karl. P.S. As for case vs. grep, personally I find a simple if...grep easier to comprehend/test/debug than a cas

Re: rhel8 test failure confirmation?

2023-12-03 Thread Karl Berry
upthread somewhere Karl (iirc) threw out a bikeshed idea like --has=. Pretty sure it wasn't me :).

Re: rhel8 test failure confirmation?

2023-12-03 Thread Karl Berry
ds Jacob's view that automake --version | grep HiRes will suffice. Not having a new option seems simpler/better in terms of later understanding, too. --thanks, karl. P.S. As for case vs. grep, personally I find a simple if...grep easier to comprehend/test/debug than a case statement. (Especi

Re: rhel8 test failure confirmation?

2023-12-02 Thread Jacob Bachmeyer
Mike Frysinger wrote: On 02 Dec 2023 17:07, Jacob Bachmeyer wrote: Mike Frysinger wrote: On 06 Apr 2023 21:29, Jacob Bachmeyer wrote: Karl Berry wrote: jb> a more thorough test would locate the autom4te script and grep it for the perllibdir that

Re: rhel8 test failure confirmation?

2023-12-02 Thread Zack Weinberg
On Sat, Dec 2, 2023, at 8:58 PM, Jacob Bachmeyer wrote: > Zack Weinberg wrote: >> Either way is no problem from my end, but it would be more work >> for automake (parsing --version output, instead of just checking the >> exit status of autom4te --assert-high-resolution-timestamp-support) >> Karl, d

Re: rhel8 test failure confirmation?

2023-12-02 Thread Jacob Bachmeyer
Zack Weinberg wrote: On Sat, Dec 2, 2023, at 7:33 PM, Jacob Bachmeyer wrote: Zack Weinberg wrote: Would it help if we added a command line option to autom4te that made it report whether it thought it could use high resolution timestamps? Versions of autom4te that didn't recognize this o

Re: rhel8 test failure confirmation?

2023-12-02 Thread Mike Frysinger
On 02 Dec 2023 17:07, Jacob Bachmeyer wrote: > Mike Frysinger wrote: > > On 06 Apr 2023 21:29, Jacob Bachmeyer wrote: > >> Karl Berry wrote: > >>> jb> a more thorough test would locate the autom4te script and grep it > >>> for the perl

Re: rhel8 test failure confirmation?

2023-12-02 Thread Mike Frysinger
eUtils and check if that also loaded Time::HiRes. > >> > >> The problem with that turned out to be that Time::HiRes got loaded from > >> other system modules, resulting in the test thinking that autom4te used > >> it when that wasn't actually the case. That&

Re: rhel8 test failure confirmation?

2023-12-02 Thread Mike Frysinger
e::HiRes. > > > > The problem with that turned out to be that Time::HiRes got loaded from > > other system modules, resulting in the test thinking that autom4te used > > it when that wasn't actually the case. That's what happened in practice > > with your patch. >

Re: rhel8 test failure confirmation?

2023-12-02 Thread Zack Weinberg
On Sat, Dec 2, 2023, at 7:33 PM, Jacob Bachmeyer wrote: > Zack Weinberg wrote: >> Would it help if we added a command line option to autom4te that made >> it report whether it thought it could use high resolution timestamps? >> Versions of autom4te that didn't recognize this option should be >> con

Re: rhel8 test failure confirmation?

2023-12-02 Thread Jacob Bachmeyer
Time::HiRes got loaded from other system modules, resulting in the test thinking that autom4te used it when that wasn't actually the case. That's what happened in practice with your patch. Would it help if we added a command line option to autom4te that made it report whether it

Re: rhel8 test failure confirmation?

2023-12-02 Thread Zack Weinberg
Time::HiRes got loaded from > other system modules, resulting in the test thinking that autom4te used > it when that wasn't actually the case. That's what happened in practice > with your patch. Would it help if we added a command line option to autom4te that made it report wh

Re: rhel8 test failure confirmation?

2023-12-02 Thread Karl Berry
The best way to check if high-resolution timestamps are available to autom4te is to have perl load Autom4te::FileUtils and check if that also loaded Time::HiRes. The problem with that turned out to be that Time::HiRes got loaded from other system modules, resulting in the test

Re: rhel8 test failure confirmation?

2023-12-02 Thread Jacob Bachmeyer
Mike Frysinger wrote: On 06 Apr 2023 21:29, Jacob Bachmeyer wrote: Karl Berry wrote: jb> a more thorough test would locate the autom4te script and grep it for the perllibdir that was substituted when autoconf was configured. I guess that would work. Challe

Re: rhel8 test failure confirmation?

2023-12-02 Thread Zack Weinberg
On Sat, Dec 2, 2023, at 6:58 AM, Mike Frysinger wrote: > On 06 Apr 2023 21:29, Jacob Bachmeyer wrote: >> Karl Berry wrote: >> > jb> a more thorough test would locate the autom4te script and grep it >> > for the perllibdir that was substituted when au

Re: rhel8 test failure confirmation?

2023-12-02 Thread Mike Frysinger
On 06 Apr 2023 21:29, Jacob Bachmeyer wrote: > Karl Berry wrote: > > jb> a more thorough test would locate the autom4te script and grep it > > for the perllibdir that was substituted when autoconf was > > configured. > > > > I guess that would work

Re: faster tests [was: rhel8 test failure confirmation?]

2023-04-18 Thread Bogdan
x27;t work in all cases, but at least it works here. # A trick to make the test run muuuch faster, by avoiding repeated # runs of aclocal (one order of magnitude improvement in speed!). echo 'AC_INIT(x,0) AM_INIT_AUTOMAKE' > configure.ac Alternatively, I w

Re: faster tests [was: rhel8 test failure confirmation?]

2023-04-17 Thread Karl Berry
usion than it's worth... I agree. Not worth the complications. t/backcompat-acout.sh: 35 -> 24s That seems to me like the only one that might be worth applying the patch for. Quite a bit more savings than anything else in the list. >> # A trick to make the test run mu

Re: faster tests [was: rhel8 test failure confirmation?]

2023-04-09 Thread Bogdan
Bogdan , Tue Apr 04 2023 21:51:52 GMT+0200 (Central European Summer Time) Karl Berry , Mon Apr 03 2023 02:08:22 GMT+0200 (Central European Summer Time) # A trick to make the test run muuuch faster, by avoiding repeated # runs of aclocal (one order of magnitude improvement in speed

Re: rhel8 test failure confirmation?

2023-04-08 Thread Karl Berry
Are you sure about that? Yes. Well, as sure as I can be. I don't see any $(...) constructs in Automake's *.m4 now, and this is certainly deliberate. We discussed shell portability many times over the decades of Automake development. I can't quickly find where it's documented, if anywhere, but

Re: rhel8 test failure confirmation?

2023-04-07 Thread Jacob Bachmeyer
m4te]). [...] Would you be up for tweaking the check to use such least-common-denominator shell stuff? Let's try: AC_PATH_PROG([AUTOM4TE], [autom4te]) if test x$autom4te_perllibdir = x; then autom4te_perllibdir=`sed -n \ '/autom4te_perllibdir/{s/^.*|| //;s/;$//;s/^.//;s/

Re: rhel8 test failure confirmation?

2023-04-07 Thread Karl Berry
Hi Jacob, The guess was the two most probable locations: /usr/share/autoconf and /usr/local/share/autoconf. Wouldn't have worked on my own system :). Challenge accepted. Thanks! if $PERL -I${autom4te_perllibdir:-$(sed -n \ '/autom4te_perllibdir/{s/^.*|| //

Re: rhel8 test failure confirmation?

2023-04-06 Thread Jacob Bachmeyer
Karl Berry wrote: jb> The test also guesses the location of autoconf's Perl libraries; I'm skeptical that any "guessing" of library locations would be reliable enough. The guess was the two most probable locations: /usr/share/autoconf and /usr/local/share/auto

Re: rhel8 test failure confirmation?

2023-04-05 Thread Karl Berry
jb> The test also guesses the location of autoconf's Perl libraries; I'm skeptical that any "guessing" of library locations would be reliable enough. jb> a more thorough test would locate the autom4te script and grep it for the perllibdir that was sub

Re: rhel8 test failure confirmation?

2023-04-04 Thread Jacob Bachmeyer
be some other simple solution. Generally, "exit 0" reports success to the shell and any other exit value is taken as false. I am unsure if POSIX actually requires that, however. As I understand, this could even be used to actually call the sub which checks the timestamps, so we

Re: faster tests [was: rhel8 test failure confirmation?]

2023-04-04 Thread Karl Berry
Is there a way to time individual tests I don't know. Maybe one of the more experienced automakers here can advise. Makes me wonder about having the test infrastructure start each test by running date to get the start time into the log file. Does anyone see a problem with doing that?

Re: rhel8 test failure confirmation?

2023-04-04 Thread Zack Weinberg
On Tue, Apr 4, 2023, at 3:51 PM, Bogdan wrote: > Nice. The 0 and 1 may not be portable to each OS in the Universe > (see EXIT_SUCCESS and EXIT_FAILURE in exit(3)), but should be > good/portable enough for our goals. Or maybe some other simple solution. ISO C guarantees that exit(0) has the sam

Re: rhel8 test failure confirmation?

2023-04-04 Thread Bogdan
mple solution. As I understand, this could even be used to actually call the sub which checks the timestamps, so we'd have a read-to-use test. Only a matter of where to put it... Is there some code that runs *before* all tests that could set some environment variable passed to the

Re: rhel8 test failure confirmation?

2023-04-04 Thread Bogdan
But this loop would require calling autom4te the number of times needed to establish the result, which could make just this testing longer than the gain... In fact, I wonder about autom4te providing an explicit way to test whether it's been patched, because I suspect that a functionality t

Re: faster tests [was: rhel8 test failure confirmation?]

2023-04-04 Thread Bogdan
Karl Berry , Mon Apr 03 2023 02:08:22 GMT+0200 (Central European Summer Time) # A trick to make the test run muuuch faster, by avoiding repeated # runs of aclocal (one order of magnitude improvement in speed!). echo 'AC_INIT(x,0) AM_INIT_AUTOMAKE' > configure.ac H

Re: rhel8 test failure confirmation?

2023-04-02 Thread Jacob Bachmeyer
nstalled its package.) [...] It seems to me that using autoconf -f or similar is papering over the problem, so that the tests are no longer testing the normal behavior. Which does not sound desirable. The Automake testsuite is supposed to test Automake, not Autoconf, so working aroun

faster tests [was: rhel8 test failure confirmation?]

2023-04-02 Thread Karl Berry
# A trick to make the test run muuuch faster, by avoiding repeated # runs of aclocal (one order of magnitude improvement in speed!). echo 'AC_INIT(x,0) AM_INIT_AUTOMAKE' > configure.ac Hadn't noticed this before. Maybe you could see what tests are currently taking

Re: rhel8 test failure confirmation?

2023-04-02 Thread Karl Berry
about autom4te providing an explicit way to test whether it's been patched, because I suspect that a functionality test (run autom4te a bunch of times on a bunch of newly-created files) would be highly susceptible to wrong results. Checking the autom4te --version number might be the easiest and

Re: rhel8 test failure confirmation? [PATCH for problem affecting Automake testsuite]

2023-04-01 Thread Paul Eggert
Thanks for reporting that. I installed the attached slightly different patch into Autoconf on Savannah. This patch also changes one other instance of file timestamp comparison from < to <=. Something like this should appear in the next Autoconf release.From 713d9822bbfb2923115065efaefed34a0113f8

Re: rhel8 test failure confirmation? [PATCH for problem affecting Automake testsuite]

2023-04-01 Thread Bogdan
Jacob Bachmeyer , Sat Apr 01 2023 04:54:22 GMT+0200 (Central European Summer Time) A quick introduction to the situation for the Autoconf list: The Automake maintainers have encountered a bizarre issue with sporadic random test failures, seemingly due to "disk writes not taking effect

Re: rhel8 test failure confirmation? [PATCH for problem affecting Automake testsuite]

2023-03-31 Thread Jacob Bachmeyer
A quick introduction to the situation for the Autoconf list: The Automake maintainers have encountered a bizarre issue with sporadic random test failures, seemingly due to "disk writes not taking effect" (as Karl Berry mentioned when starting the thread). Bogdan appears to have

Re: rhel8 test failure confirmation?

2023-03-31 Thread Bogdan
on the systems that are failing now. That's next on my list. [...]  Another tip, maybe: cache again. When I compare which files are newer than the only trace file I get in the failing 'backcompat2' test ('autom4te.cache/traces.0'), I see that 'configure.ac'

Re: rhel8 test failure confirmation?

2023-03-06 Thread Frederic Berat
. Whether it creates the problem or simply reveals it is yet unclear. With this patch I could reproduce multiple failures relatively easily (rarely more than one run). Once reverted, the only test that fails from time to time on my side is "subobj.sh" (twice out of ten trials). The amount of

Re: rhel8 test failure confirmation?

2023-03-06 Thread Frederic Berat
uot;. Whether it creates the > problem or simply reveals it is yet unclear. With this patch I could > reproduce multiple failures relatively easily (rarely more than one run). > Once reverted, the only test that fails from time to time on my side is > "subobj.sh" (twice out

Re: rhel8 test failure confirmation?

2023-03-05 Thread Bogdan
Another tip, maybe: cache again. When I compare which files are newer than the only trace file I get in the failing 'backcompat2' test ('autom4te.cache/traces.0'), I see that 'configure.ac' is older than this file in the succeeding run, but it's newer in the fail

Re: rhel8 test failure confirmation?

2023-03-03 Thread Karl Berry
Note that 'config.h' is older (4 seconds) than './configure', which shouldn't be the case as it should get updated with new values. Indeed. That is the same sort of thing as I was observing with nodef. But what (at any level) could be causing that to happen? Files just aren't getting upda

Re: rhel8 test failure confirmation?

2023-03-03 Thread Bogdan
27;debug' in the form of the 'stat' command and: -- [...] --   Note that 'config.h' is older (4 seconds) than './configure', which shouldn't be the case as it should get updated with new values.   What's funny, when I run just

Re: rhel8 test failure confirmation?

2023-03-03 Thread Bogdan
--  Note that 'config.h' is older (4 seconds) than './configure', which shouldn't be the case as it should get updated with new values.  What's funny, when I run just the failing test alone, it works 10 times in a row. A "Schroedinger bug&quo

Re: rhel8 test failure confirmation?

2023-03-03 Thread Bogdan
27;s not. I run tests sequentially and also get failures. Randomly in time, but always in the same test (and just one). In the one I've looked into most, t/nodef.sh, it seems to be about disk writes not taking effect. That test runs the usual autotool sequence twice, both times writing to th

Re: rhel8 test failure confirmation?

2023-03-02 Thread Karl Berry
guess it's possible, though I've tried make check in quite a few different environments by now (latest tools, distro tools, minimal env, maximal env, different kinds of filesystems, etc.), and the make check failures have persisted in all cases. Also, I'd expect an environment issue to ca

Re: rhel8 test failure confirmation?

2023-03-02 Thread Frederic Berat
Hello, For the record, I made a first run of testing on a rhel 8 system. While building the trunk directly led to check failures, rebuilding the RPM in a mock environment didn't. I'll likely spend more time next week to perform more testing. It may simply be an environment problem: I noticed the

Re: rhel8 test failure confirmation?

2023-03-01 Thread Karl Berry
FAIL: t/remake-aclocal-version-mismatch.sh Thanks for trying it, Nick. I'm glad it's not just me. And I sure wonder wtf is going on :(. -k

Re: rhel8 test failure confirmation?

2023-03-01 Thread Nick Bowler
== Testsuite summary for GNU Automake 1.16i # TOTAL: 2935 # PASS: 2808 # SKIP: 87 # XFAIL: 39 # FAIL: 1 # XPASS: 0 # ERROR: 0

rhel8 test failure confirmation?

2023-03-01 Thread Karl Berry
but I don't think it's directly related to parallelism. In the one I've looked into most, t/nodef.sh, it seems to be about disk writes not taking effect. That test runs the usual autotool sequence twice, both times writing to the same file named `output' and then grepping it, e.g

Re: Help with parallel test harness

2023-01-08 Thread Kip Warner
On Sun, 2023-01-08 at 14:23 -0800, Dan Mahoney wrote: > That makes sense Here's the fun, I can't find an easy way to say: > > (literally 159 tests).log : test-setup.log What you might want to do is list the 159 tests in a multiline variable defined in a seperate Automake fr

Re: Help with parallel test harness

2023-01-08 Thread Dan Mahoney
> On Jan 8, 2023, at 10:55 AM, Kip Warner wrote: > > On Sun, 2023-01-08 at 04:31 -0800, Dan Mahoney wrote: >> Hello, > > Hey Dan, > >> I’m one of the maintainers of OpenDKIM, dealing with an interesting >> program where make -j is breaking some of our

Re: Help with parallel test harness

2023-01-08 Thread Kip Warner
On Sun, 2023-01-08 at 04:31 -0800, Dan Mahoney wrote: > Hello, Hey Dan, > I’m one of the maintainers of OpenDKIM, dealing with an interesting > program where make -j is breaking some of our test programs.  > Specifically, we have tests that *must* run first to generate keys, >

Help with parallel test harness

2023-01-08 Thread Dan Mahoney
Hello, I’m one of the maintainers of OpenDKIM, dealing with an interesting program where make -j is breaking some of our test programs. Specifically, we have tests that *must* run first to generate keys, and *must* run last to do cleanup tasks. Is there an easy way with automake to force

Re: Build and test failures with Autoconf 2.70

2020-12-29 Thread Zack Weinberg
Karl Berry wrote: > Zack Weinberg wrote: >> They all appear to be cases of autoconf and/or aclocal >> getting run when the test suite does not expect them to be run. I am >> stumped as to why > > In short: because the 2.70 autom4te decided not to update configure. >

Re: Build and test failures with Autoconf 2.70

2020-12-19 Thread Karl Berry
They all appear to be cases of autoconf and/or aclocal getting run when the test suite does not expect them to be run. I am stumped as to why In short: because the 2.70 autom4te decided not to update configure. I don't know why not, but I have a guess: maybe the new autom4te ig

Re: Build and test failures with Autoconf 2.70

2020-12-15 Thread Karl Berry
FAIL: t/distcheck-missing-m4.sh FAIL: t/distcheck-outdated-m4.sh FAIL: t/libobj-basic.sh FAIL: t/remake-not-after-make-dist.sh FAIL: t/vala-non-recursive-setup.sh Whoops, I see the vala test is the only one you didn't also get, the rest are duplicates. Sorry for the noise.

Re: Build and test failures with Autoconf 2.70

2020-12-15 Thread Karl Berry
The attached patch fixes two of the problems: Thanks, I pushed that. There are still four testsuite failures on my dev box, After applying your patch, running make -j12 check against autoconf 2.70, I get those failures and several more (as in your case, they don't appear with 2.69): FA

Re: Stopping unit test on failed partial ordering dependency

2019-04-24 Thread Bob Friesenhahn
. The tests which fail still need to be reported as failed. One reason why I like the TAP testing framework is that a test script has execution context across multiple tests (avoiding repeated stop/start), which improves control over the test, and executes more efficiently. You can start

Re: Stopping unit test on failed partial ordering dependency

2019-04-23 Thread Kip Warner
On Tue, 2019-04-23 at 10:25 -0500, Bob Friesenhahn wrote: > On Mon, 22 Apr 2019, Kip Warner wrote: > > How can I solve this problem? > > By using the TAP test framework you could organize your related > tests into ordered scripts which assure that test > construction/destru

Re: Stopping unit test on failed partial ordering dependency

2019-04-23 Thread Kip Warner
On Mon, 2019-04-22 at 22:33 -0600, John Calcote wrote: > Try using a stamp file only created on success. To do this you’ll > need to wrap your test calls in a script that creates the stamp file > on success. Hey John, Wrapping the script in another wrapper is possible, but I'd pr

Re: Stopping unit test on failed partial ordering dependency

2019-04-23 Thread Bob Friesenhahn
On Mon, 22 Apr 2019, Kip Warner wrote: How can I solve this problem? By using the TAP test framework you could organize your related tests into ordered scripts which assure that test construction/destruction is orderly even if some tests fail. This approach may lose parallelism if there

Re: Stopping unit test on failed partial ordering dependency

2019-04-22 Thread John Calcote
On Mon, Apr 22, 2019 at 10:12 PM Kip Warner wrote: > How can I solve this problem? Try using a stamp file only created on success. To do this you’ll need to wrap your test calls in a script that creates the stamp file on success. John >

Stopping unit test on failed partial ordering dependency

2019-04-22 Thread Kip Warner
aring explicit make prerequisites between the generated log files. This works fine, even when using the parallel test harness. But only as long as all the tests pass. One problem I am having is if test B is dependent on test A to complete successfully before it can run, but A fails. B will run anyways

Re: automatically showing test-suite.log on failure?

2018-09-30 Thread Karl Berry
ctly one line of code: am/check.am:373:test x"$$VERBOSE" = x || cat $(TEST_SUITE_LOG); (All the other occurrences are about Automake's V= verbosity feature, I believe.) But since in my project (TeX Live) we have always used the regular driver, I am happy :). --best, karl.

Re: automatically showing test-suite.log on failure?

2018-09-30 Thread Bob Friesenhahn
On Sat, 29 Sep 2018, Karl Berry wrote: I might be missing something, but I get that behavior in my automatic builds by calling 'make check VERBOSE=1'. Yes! Thank you!! This does not appear to have any effect when using the TAP framework. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx

Re: automatically showing test-suite.log on failure?

2018-09-29 Thread Karl Berry
I might be missing something, but I get that behavior in my automatic builds by calling 'make check VERBOSE=1'. Yes! Thank you!!

Re: automatically showing test-suite.log on failure?

2018-09-28 Thread Peter Johansson
Hi Karl, On 9/13/2018 7:53 AM, Karl Berry wrote: After make check runs, if there were any failures, I'm wishing for a way to have automake to automatically show the relevant test-suite.log. I might be missing something, but I get that behavior in my automatic builds by calling '

Re: automatically showing test-suite.log on failure?

2018-09-21 Thread Eric Blake
to do in the canonical test-driver script. So, any chance of adding it as a standard feature? Any reasonable way of enabling it would be fine, e.g., a flag that can be added to [AM_]LOG_DRIVER_FLAGS. Take care since some test logs could be megabytes in size. I want to put a point in that often

Re: automatically showing test-suite.log on failure?

2018-09-21 Thread Bob Proulx
Bob Friesenhahn wrote: > Karl Berry wrote: > > However, this seems like it would be fairly commonly useful and easy > > enough to do in the canonical test-driver script. So, any chance of > > adding it as a standard feature? Any reasonable way of enabling it would > >

Re: automatically showing test-suite.log on failure?

2018-09-12 Thread Bob Friesenhahn
On Wed, 12 Sep 2018, Karl Berry wrote: However, this seems like it would be fairly commonly useful and easy enough to do in the canonical test-driver script. So, any chance of adding it as a standard feature? Any reasonable way of enabling it would be fine, e.g., a flag that can be added to

automatically showing test-suite.log on failure?

2018-09-12 Thread Karl Berry
After make check runs, if there were any failures, I'm wishing for a way to have automake to automatically show the relevant test-suite.log. The post at https://stackoverflow.com/questions/20961959 suggests that the only way to do this is to modify the test-driver script. Is that correct?

Re: [GSOC] Proposal for "Modularize Automake to improve the test-suite performance"

2018-03-25 Thread Matthias Paulmier
Hello and sorry for the late response, I didn't expect to have so much work to do for school this week end. I have uploaded an updated version of the proposal for this subject. I tried to take into consideration your suggestions. > The most important part in this proposal should be the descripti

Re: [GSOC] Proposal for "Modularize Automake to improve the test-suite performance"

2018-03-23 Thread Mathieu Lirzin
Hello, Matthias Paulmier writes: > I am candidating for the project "Modularize Automake to improve the > test-suite performance" > > As you may have noticed, this is my second proposal for the GSoC to the > Automake project. You may have also noticed that 2 other pe

[GSOC] Proposal for "Modularize Automake to improve the test-suite performance"

2018-03-23 Thread Matthias Paulmier
Hello, I am candidating for the project "Modularize Automake to improve the test-suite performance" As you may have noticed, this is my second proposal for the GSoC to the Automake project. You may have also noticed that 2 other persons are interested in this other project. This is

Re: test programs in C and C++

2018-03-11 Thread Vincent Torri
fficulties > to make it work properly. > > The current automake setup is as follows > > - /Makefile.am: for the C code compilation and the C test programs > > - /e-antic/test/Makefile.am: for the C++ test programs > > First of all, I would be happier with only one Ma

  1   2   3   4   5   6   >