[I'm sending this on behalf of Yann without his corporate-imposed
footer. You may see it in future msgs. --k...@gnu.org]
Generating the tests wrapper template currently depends on the
dependencies needed for the frozen lib/m4sugar/m4sh.m4f. However,
it also requires that frozen sugar to actually be generated, or the
build will fail if it is not (or at least if it is in the process of
being generated):
[...]
/bin/mkdir -p lib/m4sugar
lang=`echo 'lib/m4sugar/m4sh' | sed 's,.*/,,'` && if test $lang =
autoconf; then lang=autoconf-without-aclocal-m4; else :; fi &&
autom4te_perllibdir='.'/lib AUTOM4TE_CFG='lib/autom4te.cfg'
bin/autom4te -B ''lib -B '.'/lib --language=$lang --freeze
--output=lib/m4sugar/m4sh.m4f
autom4te_perllibdir='.'/lib AUTOM4TE_CFG='lib/autom4te.cfg'
bin/autom4te -B ''lib -B '.'/lib --language=M4sh ./tests/wrapper.as -o
tests/wrapper.in
/usr/bin/m4:lib/m4sugar/m4sh.m4f:627: premature end of frozen file
autom4te: error: /usr/bin/m4 failed with exit status: 1
make[3]: *** [Makefile:2227: tests/wrapper.in] Error 1
So, in the trace above, lib/m4sugar/m4sh.m4f is only 627 lines when it
gets called. But when later inspected, it is actually 2046 lines. This
means that the file is not yet fully generated while it is being used.
Change the dependency to actually depend on the generated file, rather
than on its dependencies.
This kind of failure has been noticed on highly parallel builds on very
loaded machines, and is quite difficult to reproduce.
Here's a little script that shows that lib/m4sugar/m4sh.m4f and
tests/wrapper.in are generated out-of-order without this patch:
#!/bin/sh
git clean -dX --force
./bootstrap
./configure
make -j2048
(
stat -c '%.9Y %n' lib/m4sugar/m4sh.m4f
stat -c '%.9W %n' tests/wrapper.in
) |sort -n
Here, about 70% of the time, wrapper.in is birthed before m4sh.m4f is
last modified. Not sure why that does not cause the failure I see, but
I suspect that m4 uses the unfrozen file if the frozen one does not
exist, and only use the frozen file if it exists; because of scheduling,
either the frozen file does not yet exists at the point where m4 would
use it to generate the wrapper, or it already exists and is fully
written to _by the time it is parsed_. As that file is relatively small,
the window where it is only partially generated by the time it is
parsed, is very small and very difficult to hit...
Signed-off-by: Yann E. MORIN <yann.mo...@orange.com>
---
To give an idea of how difficult it was to reproduce: our CI has been
running about 20 builds a day for the past three years, but the failure
happened only once in the past year; I also do like 10 builds a day
locally, and it only eventually happened yesterday.
---
tests/local.mk | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/local.mk b/tests/local.mk
index 5c45d9ee..6a2c06a8 100644
--- a/tests/local.mk
+++ b/tests/local.mk
@@ -63,7 +63,7 @@ CLEANFILES += \
tests/wrapper.in \
$(wrappers)
-tests/wrapper.in: $(srcdir)/tests/wrapper.as $(m4sh_m4f_dependencies)
+tests/wrapper.in: $(srcdir)/tests/wrapper.as lib/m4sugar/m4sh.m4f
$(MY_AUTOM4TE) --language=M4sh $(srcdir)/tests/wrapper.as -o $@
edit_wrapper = sed \
--
2.34.1
