$(wildcard ...*/) and symlinks
GNU Make 4.4.0.91 Run from an otherwise empty directory: % cat Makefile all: prepare echo $(wildcard */) echo $(wildcard l*/) echo $(wildcard */*/) prepare: mkdir -p d/d touch f d/f ln -sf f l ln -sf f d/l % make -s d/ l d/d/ d/l % So "*/" ignores the symlink to a file as expected, but "l*/" does return it, likewise "*/*/" which doesn't make much sense to me. Maybe related to the issues discussed on https://stackoverflow.com/questions/66271761/gnu-make-wildcard-function-and-terminating-slashes-on-directory-names The workaround given there, adding a dot (and removing it from the results), seems to help, i.e.: $(patsubst %/., %, $(wildcard ...*/.))
Re: VPATH question
On Fri, 2023-02-24 at 08:44 +0100, Gisle Vanem wrote: > So I want to create a './lib' directory in > 'Current Directory', but gnumake won't do it > since there is a 'lib' under 'apps'. Output: > gnumake: Nothing to be done for 'default'. > > Why doesn't it understand the meaning of '.'? It does. Make adds the prerequisite to every VPATH entry. Your VPATH entry is "apps" and the prerequisite is "./lib", and so the combination is "apps/./lib" which is a perfectly legal pathname, that is found. > It also drops the '.' in the --debug' output. This is due to a special rule in GNU make that allows "." to be ignored for the purposes of target matching. It's a little obscure, but it's not related to the issue you're having with VPATH. Even if it kept the "./", "apps/./lib" would still exist and be matched. > Besides I feel it's not very helpful explaining why > it won't do "Nothing..". E.g. if I make a typo: >default all: $(GEN_DIRS2) > > it's the same bleeding message. Well, I'm not sure it's useful for make to generate one message for situations where a target is not rebuilt because all of its N prerequisites are up to date and N > 0, and a different message for the special case where N == 0. > Adding a '--debug=w', > would be nice with a 'why not' message. If you use "-d" you'll get the info you want: No implicit rule found for 'default'. Considering target file 'lib'. Finished prerequisites of target file 'lib'. No need to remake target 'lib'; using VPATH name 'apps/lib'. Finished prerequisites of target file 'default'. > Only adding 'MAKEFLAGS += --warn-undefined-variables' is somewhat > helpful. You should feel free to do that, if your makefiles are written such that it doesn't give spurious warnings. Make has silently expanded empty variables to the empty string for 40 years and changing that as the default behavior is not an option.
Re: Regression on Cygwin: Problems with parallel make in 4.4
On Thu, 2023-02-23 at 16:17 -0500, Ken Brown wrote: > I'm attaching that script so you can see exactly how "make" is > invoked in a subshell. I'm also attaching my build log up to the > point of the warning and the Makefile in the ft-build directory in > which the warning occurred. I've previously sent you the top-level > Makefile. > > Let me know if you need anything else. The ft-build/Makefile is not the interesting one: by the time make is invoked here it already has the bad arguments. The makefile we want is the one that invoked this make, with the bad arguments; from your build: make[3]: Entering directory '/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2' rm -rf ft-build /usr/bin/mkdir -p ft-build cd ft-build && \ CC='gcc' CONFIG_SITE=/dev/null CONFIG_SHELL='/bin/sh' \ /bin/sh /home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2/../../../libs/freetype2/freetype-src/configure \ --disable-shared \ --without-bzip2 \ --without-brotli \ --without-harfbuzz \ --without-png \ --without-zlib \ --prefix=/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2/ft-install \ --libdir=/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2 \ --includedir=/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2 so, the makefile we want is the makefile in freetype2 that contains the recipe that invokes the make command in the ft-build subdirectory.
Re: $(wildcard ...*/) and symlinks
On Fri, Feb 24, 2023 at 6:55 AM Frank Heckenbach wrote: ... > So "*/" ignores the symlink to a file as expected, but "l*/" does > return it, likewise "*/*/" which doesn't make much sense to me. There is a known bug in glibc glob impl. $ ls makefile $ cat makefile all:; $(info $(wildcard hell*/)) $ touch hello $ ls hell*/ ls: cannot access 'hell*/': No such file or directory $ make hello make: 'all' is up to date. The current impl of glob handles trailing slashes correctly on certain filesystems and fails on others. See https://lists.gnu.org/archive/html/bug-gnulib/2019-11/msg6.html regards, Dmitry
Re: Regression on Cygwin: Problems with parallel make in 4.4
On 2/24/2023 9:54 AM, Paul Smith wrote:
On Thu, 2023-02-23 at 16:17 -0500, Ken Brown wrote:
I'm attaching that script so you can see exactly how "make" is
invoked in a subshell. I'm also attaching my build log up to the
point of the warning and the Makefile in the ft-build directory in
which the warning occurred. I've previously sent you the top-level
Makefile.
Let me know if you need anything else.
The ft-build/Makefile is not the interesting one: by the time make is
invoked here it already has the bad arguments.
The makefile we want is the one that invoked this make, with the bad
arguments; from your build:
make[3]: Entering directory
'/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2'
rm -rf ft-build
/usr/bin/mkdir -p ft-build
cd ft-build && \
CC='gcc' CONFIG_SITE=/dev/null CONFIG_SHELL='/bin/sh' \
/bin/sh
/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2/../../../libs/freetype2/freetype-src/configure
\
--disable-shared \
--without-bzip2 \
--without-brotli \
--without-harfbuzz \
--without-png \
--without-zlib \
--prefix=/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2/ft-install \
--libdir=/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2 \
--includedir=/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2
so, the makefile we want is the makefile in freetype2 that contains the
recipe that invokes the make command in the ft-build subdirectory.
OK, I hope I'm attaching the right one now.
I thought it might also help for me to send a complete recipe for
reproducing the problem rather than just sending bits and pieces. The
recipe is adapted from
https://www.tug.org/texlive/build.html
1. git clone https://github.com/TeX-Live/texlive-source.git
2. cd texlive-source
3. TL_MAKE_FLAGS="-j13" ./Build
The build is then done in texlive-source/Work, which also contains a log
file "build.log". On my system the jobserver warning appears after
about 10 minutes and is in line 2074 of build.log. I'm using make
4.4.0.91 patched with commit 8b8cc3a825b0e9e176db5d90084c814e3744a300.
Ken
# Makefile.in generated by automake 1.16.5 from Makefile.am.
# Makefile. Generated from Makefile.in by configure.
# Copyright (C) 1994-2021 Free Software Foundation, Inc.
# This Makefile.in is free software; the Free Software Foundation
# gives unlimited permission to copy and/or distribute it,
# with or without modifications, as long as this notice is preserved.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY, to the extent permitted by law; without
# even the implied warranty of MERCHANTABILITY or FITNESS FOR A
# PARTICULAR PURPOSE.
VPATH = ../../../libs/freetype2
am__is_gnu_make = { \
if test -z '$(MAKELEVEL)'; then \
false; \
elif test -n '$(MAKE_HOST)'; then \
true; \
elif test -n '$(MAKE_VERSION)' && test -n '$(CURDIR)'; then \
true; \
else \
false; \
fi; \
}
am__make_running_with_option = \
case $${target_option-} in \
?) ;; \
*) echo "am__make_running_with_option: internal error: invalid" \
"target option '$${target_option-}' specified" >&2; \
exit 1;; \
esac; \
has_opt=no; \
sane_makeflags=$$MAKEFLAGS; \
if $(am__is_gnu_make); then \
sane_makeflags=$$MFLAGS; \
else \
case $$MAKEFLAGS in \
*\\[\ \ ]*) \
bs=\\; \
sane_makeflags=`printf '%s\n' "$$MAKEFLAGS" \
| sed "s/$$bs$$bs[$$bs $$bs ]*//g"`;; \
esac; \
fi; \
skip_next=no; \
strip_trailopt () \
{ \
flg=`printf '%s\n' "$$flg" | sed "s/$$1.*$$//"`; \
}; \
for flg in $$sane_makeflags; do \
test $$skip_next = yes && { skip_next=no; continue; }; \
case $$flg in \
*=*|--*) continue;; \
-*I) strip_trailopt 'I'; skip_next=yes;; \
-*I?*) strip_trailopt 'I';; \
-*O) strip_trailopt 'O'; skip_next=yes;; \
-*O?*) strip_trailopt 'O';; \
-*l) strip_trailopt 'l'; skip_next=yes;; \
-*l?*) strip_trailopt 'l';; \
-[dEDm]) skip_next=yes;; \
-[JT]) skip_next=yes;; \
esac; \
case $$flg in \
*$$target_option*) has_opt=yes; break;; \
esac; \
done; \
test $$has_opt = yes
am__make_dryrun = (target_option=n; $(am__make_running_with_option))
am__make_keepgoing = (target_option=k; $(am__make_running_with_option))
pkgdatadir = $(datadir)/freetype--tex-live-
pkgincludedir = $(includedir)/freetype--tex-live-
pkglibdir = $(libdir)/freetype--tex-live-
pkglibexecdir = $(libexecdir)/freetype--tex-live-
am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd
install_sh_DATA = $(install_sh) -c -m 644
install_sh_PROGRAM = $(install_sh) -c
install_sh_SCRIPT = $(install_sh) -c
INSTALL_HEADER = $(INSTALL_DATA)
transform = $(program_transform_name)
NORMAL_INSTALL = :
PRE_INSTALL = :
POST_INSTALL = :
NORMAL_UNINSTALL = :
PRE_UNINSTALL = :
POST_UNINSTALL = :
subdir = .
ACLOCAL_M4 = $(top_srcdir)/ac
Re: Regression on Cygwin: Problems with parallel make in 4.4
On Thu, 2023-02-23 at 16:17 -0500, Ken Brown wrote:
> Sorry, I spoke too soon. I just went back to my original use case,
> in which I used the TeX Live "Build" script instead of directly
> invoking make, and I again got the jobserver warning.
Thanks for the info.
TL;DR: this is not an error in make :)
This warning is happening because the freetype2 configure script itself
is invoking make.
So the freetype2/Makefile contains this rule:
ft-config:
rm -rf ft-build
$(MKDIR_P) ft-build
cd ft-build && \
CC='$(CC)' CONFIG_SITE=/dev/null CONFIG_SHELL='$(SHELL)' \
$(SHELL) $(abs_srcdir)/$(FREETYPE_TREE)/configure \
$(config_args) --disable-shared \
--without-bzip2 \
--without-brotli \
--without-harfbuzz \
--without-png \
--without-zlib \
--prefix=$(abs_builddir)/ft-install \
--libdir=$(abs_builddir) \
--includedir=$(abs_builddir)
echo timestamp >ft-config
This rule does not appear to invoke make (doesn't contain $(MAKE) or
${MAKE}) and it does not use a "+" prefix, so make believes that this
recipe is not a recursive make invocation, and it disables the
jobserver before invoking this recipe.
But we can see from the output that the recursive make invocation
happens in the configure script:
/usr/bin/mkdir -p ft-build
cd ft-build && \
CC='gcc' CONFIG_SITE=/dev/null CONFIG_SHELL='/bin/sh' \
/bin/sh
/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2/../../../libs/freetype2/freetype-src/configure
\
--disable-shared \
--without-bzip2 \
--without-brotli \
--without-harfbuzz \
--without-png \
--without-zlib \
--prefix=/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2/ft-install \
--libdir=/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2 \
--includedir=/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2
Copying documentation assets
cp: cannot stat
'/home/kbrown/src/texlive/test.x86_64/libs/freetype2/freetype-src/docs/markdown':
No such file or directory
Copying `modules.cfg'
Generating `Makefile'
make[4]: Entering directory
'/home/kbrown/src/texlive/test.x86_64/Work/libs/freetype2/ft-build'
make[4]: not recursive -2,-2
make[4]: warning: jobserver unavailable: using -j1. Add '+' to parent make
rule.
So, if you go get the freetype2/configure script and investigate its
contents, you'll find this:
CFG=$CFG $MAKE setup unix
This invokes make, and it's THIS invocation of make which is printing
that message, and the message is absolutely correct: the jobserver WAS
de-activated because the "recipe" (the thing that invoked configure)
was not a recursive invocation.
You have a number of options here:
- You can ignore the message since it doesn't make any difference.
- You can add a "+" prefix to the configure script in the ft-config
target's recipe in the freetype2/Makefile to explicitly tell make that
this recipe is a recursive make.
You can add a spurious reference to "$(MAKE)", like:
ft-config:
rm -rf ft-build
$(MKDIR_P) ft-build
cd ft-build && : '$(MAKE)' && \
^^
CC='$(CC)' CONFIG_SITE=/dev/null CONFIG_SHELL='$(SHELL)' \
$(SHELL) $(abs_srcdir)/$(FREETYPE_TREE)/configure \
...
echo timestamp >ft-config
to implicitly tell make that this recipe is a recursive make.
- You can remove all the MAKEFLAGS before you invoke configure:
ft-config:
rm -rf ft-build
$(MKDIR_P) ft-build
cd ft-build && : '$(MAKE)' && \
CC='$(CC)' CONFIG_SITE=/dev/null CONFIG_SHELL='$(SHELL)' \
MAKEFLAGS= $(SHELL) $(abs_srcdir)/$(FREETYPE_TREE)/configure \
^^
...
echo timestamp >ft-config
- You could also file a bug against freetype2 and suggest that they
themselves unset MAKEFLAGS in their configure script, since it's very
unusual for a configure script to invoke make:
CFG=$CFG MAKEFLAGS= $MAKE setup unix
This issue is one of the reasons we switched to named pipes: in that
mode there's no need to pass resources (like open file descriptors) to
sub-makes, and so we don't have to disable them. We can just assume
that only a sub-make will parse the MAKEFLAGS and open the named pipe.
Re: VPATH question
Paul Smith wrote: If you use "-d" you'll get the info you want: No implicit rule found for 'default'. Considering target file 'lib'. Finished prerequisites of target file 'lib'. No need to remake target 'lib'; using VPATH name 'apps/lib'. Finished prerequisites of target file 'default'. Only adding 'MAKEFLAGS += --warn-undefined-variables' is somewhat helpful. You should feel free to do that, if your makefiles are written such that it doesn't give spurious warnings. Make has silently expanded empty variables to the empty string for 40 years and changing that as the default behavior is not an option. Ok thanks for the explanation. But I tend to use 'vpath %.c dir' more and more. It seems a lot faster too. -- --gv
Re: Regression on Cygwin: Problems with parallel make in 4.4
On 2/24/2023 11:29 AM, Paul Smith wrote: On Thu, 2023-02-23 at 16:17 -0500, Ken Brown wrote: Sorry, I spoke too soon. I just went back to my original use case, in which I used the TeX Live "Build" script instead of directly invoking make, and I again got the jobserver warning. Thanks for the info. TL;DR: this is not an error in make :) That's a relief. Thanks for the detailed explanation. Ken
Re: VPATH question
On Fri, 2023-02-24 at 17:40 +0100, Gisle Vanem wrote: > Paul Smith wrote: > > If you use "-d" you'll get the info you want: > > > > No implicit rule found for 'default'. > > Considering target file 'lib'. > > Finished prerequisites of target file 'lib'. > > No need to remake target 'lib'; using VPATH name 'apps/lib'. > > Finished prerequisites of target file 'default'. > > > > > Only adding 'MAKEFLAGS += --warn-undefined-variables' is somewhat > > > helpful. > > > > You should feel free to do that, if your makefiles are written such > > that it doesn't give spurious warnings. Make has silently expanded > > empty variables to the empty string for 40 years and changing that > > as the default behavior is not an option. > > Ok thanks for the explanation. But I tend to use 'vpath %.c dir' > more and more. It seems a lot faster too. In my opinion it's just not a good idea to use directories as prerequisites in makefile rules, period, with only very specific, limited exceptions. But, I realize that I'm in the distinct minority in this opinion and that all the "hot new makefiles" tend to do this. I'll stick to my old fuddy-duddy opinions however. Keep those directories off my lawn! As for "it seems a lot faster", I have no comment since you don't say what it seems faster than. If you mean that "vpath %.c dir" is faster than "VPATH = dir", then that's not surprising since the latter searches "dir" for every single prerequisite while the former only searches for prerequisites that match the pattern. If you mean it's faster than listing the full pathname explicitly as a prerequisite, then I can't believe that's the case. Cheers!
