bitset: Fix conflict with 'free-posix' module

[Bruno Haible]

The continuous integration uncovered this problem: A testdir with modules
'bitset' and 'free-posix' fails to build.

In file included from ../../gllib/bitset.h:31:0,
 from ../../gllib/bitset.c:22:
../../gllib/bitset.c: In function ‘bitset_free’:
../../gllib/bitset/base.h:200:18: error: ‘const struct bitset_vtable’ has no 
member named ‘rpl_free’
  ((SRC)->b.vtable->free ? (SRC)->b.vtable->free (SRC) :(void)0)
  ^
../../gllib/bitset.c:173:7: note: in expansion of macro ‘BITSET_FREE_’
   BITSET_FREE_ (bset);
   ^
../../gllib/bitset/base.h:200:42: error: ‘const struct bitset_vtable’ has no 
member named ‘rpl_free’
  ((SRC)->b.vtable->free ? (SRC)->b.vtable->free (SRC) :(void)0)
  ^
../../gllib/bitset.c:173:7: note: in expansion of macro ‘BITSET_FREE_’
   BITSET_FREE_ (bset);
   ^
../../gllib/bitset.c: In function ‘bitset_obstack_free’:
../../gllib/bitset/base.h:200:18: error: ‘const struct bitset_vtable’ has no 
member named ‘rpl_free’
  ((SRC)->b.vtable->free ? (SRC)->b.vtable->free (SRC) :(void)0)
  ^
../../gllib/bitset.c:184:5: note: in expansion of macro ‘BITSET_FREE_’
 BITSET_FREE_ (bset);
 ^
../../gllib/bitset/base.h:200:42: error: ‘const struct bitset_vtable’ has no 
member named ‘rpl_free’
  ((SRC)->b.vtable->free ? (SRC)->b.vtable->free (SRC) :(void)0)
  ^
../../gllib/bitset.c:184:5: note: in expansion of macro ‘BITSET_FREE_’
 BITSET_FREE_ (bset);
 ^

This patch fixes it.


2020-12-26  Bruno Haible  

bitset: Fix conflict with 'free-posix' module.
* lib/bitset/base.h: Include .

diff --git a/lib/bitset/base.h b/lib/bitset/base.h
index fe447c2..ad674ef 100644
--- a/lib/bitset/base.h
+++ b/lib/bitset/base.h
@@ -24,6 +24,7 @@
 #include 
 #include 
 #include 
+#include  /* because Gnulib's  may '#define free ...' 
*/
 #include  /* ffsl */
 
 #include "attribute.h"

stack: Fix typo in comments

[Bruno Haible]

A trivial typo.


2020-12-26  Bruno Haible  

stack: Fix typo in comments.
* lib/stack.h: Fix typo in comments.

diff --git a/lib/stack.h b/lib/stack.h
index 5d80390..7f19d26 100644
--- a/lib/stack.h
+++ b/lib/stack.h
@@ -53,8 +53,8 @@
After including this file, these names will be undefined.
 
Before including this file, you also need to include:
- #include ""
- #include ""
+ #include 
+ #include 
  #include "assure.h"
  #include "xalloc.h"
 */

new modules execve, execvpe, execvp, execv, execl, execle, execlp

[Bruno Haible]

Eli Zaretskii wrote in
:

  ... found that the implementation of the exec* family did
  the following: it started the process that runs the child program, and
  then immediately exited without waiting for the child to finish.
  ...
  The solution was to reimplement execvp in a way that didn't exit ...
  until the child program finished.

Indeed, that is a major problem with these exec* implementations: The
parent process cannot know the exit code if the exec'd process runs
asynchronously.

Another major problem is that the Windows _spawnv* and CreateProcess
functions split the arguments when they contain spaces or special
characters.

All of these problems are fixed with these new Gnulib modules.


2020-12-26  Bruno Haible  

execlp: Add tests.
* tests/test-execlp-main.c: New file.
* tests/test-execlp.sh: New file.
* modules/execlp-tests: New file.

execlp: New module.
* lib/execlp.c: New file.
* m4/execlp.m4: New file.
* modules/execlp: New file.
* doc/posix-functions/execlp.texi: Mention more Windows problems and the
new module.

execle: Add tests.
* tests/test-execle-main.c: New file.
* tests/test-execle.sh: New file.
* modules/execle-tests: New file.

execle: New module.
* lib/execle.c: New file.
* m4/execle.m4: New file.
* modules/execle: New file.
* doc/posix-functions/execle.texi: Mention more Windows problems and the
new module.

execl: Add tests.
* tests/test-execl-main.c: New file.
* tests/test-execl.sh: New file.
* modules/execl-tests: New file.

execl: New module.
* lib/execl.c: New file.
* m4/execl.m4: New file.
* modules/execl: New file.
* doc/posix-functions/execl.texi: Mention more Windows problems and the
new module.

execv: Add tests.
* tests/test-execv-main.c: New file.
* tests/test-execv.sh: New file.
* modules/execv-tests: New file.

execv: New module.
* lib/execv.c: New file.
* m4/execv.m4: New file.
* modules/execv: New file.
* doc/posix-functions/execv.texi: Mention more Windows problems and the
new module.

execvp: Add tests.
* tests/test-execvp-main.c: New file.
* tests/test-execvp.sh: New file.
* modules/execvp-tests: New file.

execvp: New module.
* lib/execvp.c: New file.
* m4/execvp.m4: New file.
* modules/execvp: New file.
* doc/posix-functions/execvp.texi: Mention more Windows problems and the
new module.

execvpe: Add tests.
* tests/test-execvpe-main.c: New file.
* tests/test-execvpe.sh: New file.
* modules/execvpe-tests: New file.

execvpe: New module.
* lib/execvpe.c: New file.
* m4/execvpe.m4: New file.
* modules/execvpe: New file.
* doc/glibc-functions/execvpe.texi: Mention the Windows problems and the
new module.

execve: Add tests.
* tests/test-exec-child.c: New file.
* tests/test-execve-main.c: New file.
* tests/test-execve.sh: New file.
* modules/execve-tests: New file.

execve: New module.
* lib/execve.c: New file.
* m4/execve.m4: New file.
* modules/execve: New file.
* doc/posix-functions/execve.texi: Mention more Windows problems and the
new module.

execve, execvpe, execvp, execv, execl, execle, execlp: Prepare modules.
* lib/unistd.in.h (execl, execle, execlp, execv, execve, execvp,
execvpe): Add declarations for the new modules.
* m4/unistd_h.m4 (gl_UNISTD_H): Test whether execl, execle, execlp,
execv, execve, execvp, execvpe are declared.
(gl_UNISTD_H_DEFAULTS): Initialize GNULIB_EXEC*, HAVE_EXECVPE,
REPLACE_EXEC*.
* modules/unistd (Makefile.am): Substitute GNULIB_EXEC*, HAVE_EXECVPE,
REPLACE_EXEC*.
* tests/test-unistd-c++.cc: Check the signature of execl, execle,
execlp, execv, execve, execvp, execvpe.

>From ba3aeabdd5317c3e21d1da54577235a054ce60f3 Mon Sep 17 00:00:00 2001
From: Bruno Haible 
Date: Sat, 26 Dec 2020 14:09:29 +0100
Subject: [PATCH 01/15] execve, execvpe, execvp, execv, execl, execle, execlp:
 Prepare modules.

* lib/unistd.in.h (execl, execle, execlp, execv, execve, execvp,
execvpe): Add declarations for the new modules.
* m4/unistd_h.m4 (gl_UNISTD_H): Test whether execl, execle, execlp,
execv, execve, execvp, execvpe are declared.
(gl_UNISTD_H_DEFAULTS): Initialize GNULIB_EXEC*, HAVE_EXECVPE,
REPLACE_EXEC*.
* modules/unistd (Makefile.am): Substitute GNULIB_EXEC*, HAVE_EXECVPE,
REPLACE_EXEC*.
* tests/test-unistd-c++.cc: Check the signature of execl, execle,
execlp, execv, execve, execvp, execvpe.
---
 ChangeLog|  14 +

cmp/diff

[Simon Josefsson via Gnulib discussion list]

Hi!  I have noticed self-check failures of libidn on ArchLinux:

https://gitlab.com/libidn/libidn/-/jobs/932649011

The gnulib self-tests that fail are: test-binary-io.sh test-perror.sh
test-init.sh

https://gitlab.com/libidn/libidn/-/jobs/932649011/artifacts/file/libidn-1.36a/b/gltests/test-binary-io.sh.log
https://gitlab.com/libidn/libidn/-/jobs/932649011/artifacts/file/libidn-1.36a/b/gltests/test-perror.sh.log
https://gitlab.com/libidn/libidn/-/jobs/932649011/artifacts/file/libidn-1.36a/b/gltests/test-init.sh.log

Essentialy it boils down to:

../../gltests/test-binary-io.sh: line 8: cmp: command not found
test-init.sh: failed test: err not empty: ../../gltests/init.sh: line 648: cmp: 
command not found
../../gltests/test-perror.sh: line 14: diff: command not found

I was a little surprised ArchLinux do not have cmp/diff by default, but
I was surprised that nothing except these gnulib tests failed.

I am reading gnulib's DEPENDENCIES which says:

  * The comparison utilities 'cmp' and 'diff'.
+ Mandatory. Using the platform's native utilities gives good portability
  exposure, but you can also use GNU diffutils.
+ Homepage:
  https://www.gnu.org/software/diffutils/

I read the DEPENDENCIES file a bit more careful after noticing that it
has several mandatory dependencies that I find questionable, and I
noticed this in the beginning:

  The following packages are needed by maintainers for using
  'gnulib-tool'.

It doesn't say the packages are required for users building gnulib-based
packages.  I can live with having diffutils installed as a maintainer.
Was this subtle distinction intentional?

I sympathize with ArchLinux's decision to be minimal, so my preference
is to make things work when it appears to be so simple.

What do you think about rewriting these self-tests to avoid cmp/diff?

Does anyone know a good portable shell-based idiom for comparing files
instead of using cmp/diff?

I'm posting this instead of just fixing the self-tests because I have a
suspicion this has been discussed, and maybe it is time to revisit it.

/Simon


signature.asc
Description: PGP signature

Re: cmp/diff

[Bruno Haible]

Hi Simon,

> Essentialy it boils down to:
> 
> ../../gltests/test-binary-io.sh: line 8: cmp: command not found
> test-init.sh: failed test: err not empty: ../../gltests/init.sh: line 648: 
> cmp: command not found
> ../../gltests/test-perror.sh: line 14: diff: command not found

'cmp' and 'diff' are among the programs that a 'configure' script and a Makefile
can assume to be present, see the GNU Coding Standards
.

> What do you think about rewriting these self-tests to avoid cmp/diff?

There is no adequate replacement. It would be possible to use 'md5sum'
instead of 'cmp'; but since 'md5sum' is not in the above list from the
GNU Coding Standards, this is not a useful direction.

Gnulib makes sure to pave over differences between different 'diff'
implementations; this was already a major headache. See tests/init.sh lines
589..669.

> https://gitlab.com/libidn/libidn/-/jobs/932649011

The Gitlab jobs typically start by installing packages into the build
environment. Yours has

  $ pacman -Sy --noconfirm make gcc

I guess that adding 'diffutils' to this package list will solve the problem.

> I read the DEPENDENCIES file a bit more careful after noticing that it
> has several mandatory dependencies that I find questionable, and I
> noticed this in the beginning:
> 
>   The following packages are needed by maintainers for using
>   'gnulib-tool'.
> 
> It doesn't say the packages are required for users building gnulib-based
> packages.  I can live with having diffutils installed as a maintainer.
> Was this subtle distinction intentional?

Yes, of course, we make a distinction between the packages needed for
creating the tarballs and the packages needed to build+install the tarballs.
The GNU Coding Standards matters for the latter.

Bruno

Re: cmp/diff

[Simon Josefsson via Gnulib discussion list]

Bruno Haible  writes:

> Hi Simon,
>
>> Essentialy it boils down to:
>> 
>> ../../gltests/test-binary-io.sh: line 8: cmp: command not found
>> test-init.sh: failed test: err not empty: ../../gltests/init.sh:
>> line 648: cmp: command not found
>> ../../gltests/test-perror.sh: line 14: diff: command not found
>
> 'cmp' and 'diff' are among the programs that a 'configure' script and a 
> Makefile
> can assume to be present, see the GNU Coding Standards
> .

Hi Bruno.  Thanks for the pointer!  I believe this resolves my concern.

It would be nice to reduce that set of tools.  Is there any automated
purpose for diff other than a fancy cmp?

/Simon


signature.asc
Description: PGP signature

Re: cmp/diff

[Bruno Haible]

Hi Simon,

> > 'cmp' and 'diff' are among the programs that a 'configure' script and a 
> > Makefile
> > can assume to be present, see the GNU Coding Standards
> > .
> 
> Hi Bruno.  Thanks for the pointer!  I believe this resolves my concern.
> 
> It would be nice to reduce that set of tools.

Well, both 'cmp' and 'diff' are part of POSIX:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/cmp.html
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/diff.html

They have been listed at this place in the GNU Coding Standards since
at least 2004.

But, of course, you are free to write to
https://lists.gnu.org/mailman/listinfo/bug-standards

> Is there any automated purpose for diff other than a fancy cmp?

Makefile rules are written both for automated execution and for the
developers of a package. While for the automated execution of a
unit test, "cmp expected.out actual.out" is sufficient, for a developer,
"diff expected.out actual.out" is definitely nicer. Especially if the
log of the unit test is created on a machine on which the developer has
no direct access (such as a contributor's machine or a build farm),
"diff expected.out actual.out" has information that is essential for
debugging.

Bruno

Re: cmp/diff

[Paul Eggert]

On 12/26/20 10:49 AM, Bruno Haible wrote:

It would be nice to reduce that set of tools.

Well, both 'cmp' and 'diff' are part of POSIX:


Even awk is on that list, and it's much fancier than cmp or diff. And I 
wouldn't support removing awk either, as it's a handy tool in Makefiles 
when the shell isn't up to a text-processing task.


Looking at that list now, the only obvious tool to remove is egrep, 
since 'grep -E' subsumes it nowadays.

Re: cmp/diff

[Paul Smith]

On Sat, 2020-12-26 at 19:49 +0100, Bruno Haible wrote:
> Makefile rules are written both for automated execution and for the
> developers of a package. While for the automated execution of a unit
> test, "cmp expected.out actual.out" is sufficient, for a developer,
> "diff expected.out actual.out" is definitely nicer.

Agreed.

It could be possible for the configure script to check for diff and if
it's not provided, replace it with cmp in situations where we only need
to care about equality for correct processing (which is most places).

A message could be shown to the user saying that without diff it will
be hard for maintainers to assist if the configure fails.

However, that seems like a lot of effort for not much gain (for the GNU
project) so I'd suggest that if someone wanted something like that they
would need to provide the implementation.

gnulib-tool --conditional-dependencies --test problem

[Paul Eggert]

When I run this command:

./gnulib-tool --conditional-dependencies --test regex

on Fedora 33, it fails as shown in the attached file. The problem occurs 
because --test brings in binary-io for some reason, but doesn't bring in 
the fcntl-h module that binary-io depends on.


A simple workaround would be to have gnulib-tool's --test option disable 
the conditionalization of dependencies, or to have gnulib-tool diagnose 
and fail any attempt to combine the two options (which it already does 
for --test and --gnu-make). But is there a better way?
69-penguin $ ./gnulib-tool --conditional-dependencies --test regex
Module list with included dependencies (indented):
absolute-header
accept
accept-tests
alloca
alloca-opt
alloca-opt-tests
arpa_inet
arpa_inet-tests
attribute
binary-io
binary-io-tests
bind
bind-tests
btowc
btowc-tests
builtin-expect
c-ctype
c-ctype-tests
c-strcase
c-strcase-tests
c99
cloexec
cloexec-tests
close
close-tests
connect
connect-tests
ctype
ctype-tests
dup2
dup2-tests
environ
environ-tests
errno
errno-tests
extensions
extern-inline
fcntl
fcntl-h
fcntl-h-tests
fcntl-tests
fd-hook
fdopen
fdopen-tests
fgetc-tests
filename
flexmember
fputc-tests
fread-tests
fstat
fstat-tests
ftruncate
ftruncate-tests
fwrite-tests
getcwd-lgpl
getcwd-lgpl-tests
getdtablesize
getdtablesize-tests
gettext-h
gettimeofday
gettimeofday-tests
hard-locale
hard-locale-tests
ignore-value
ignore-value-tests
include_next
inet_pton
inet_pton-tests
intprops
intprops-tests
inttypes
inttypes-incomplete
inttypes-tests
ioctl
ioctl-tests
isblank
isblank-tests
langinfo
langinfo-tests
largefile
libc-config
limits-h
limits-h-tests
listen
listen-tests
localcharset
localcharset-tests
locale
locale-tests
localeconv
localeconv-tests
localename
localename-tests
lock
lock-tests
lstat
lstat-tests
malloc-posix
malloca
malloca-tests
mbrtowc
mbrtowc-tests
mbsinit
mbsinit-tests
mbtowc
msvc-inval
msvc-nothrow
multiarch
nanosleep
nanosleep-tests
netinet_in
netinet_in-tests
nl_langinfo
nl_langinfo-tests
open
open-tests
pathmax
pathmax-tests
perror
perror-tests
pipe-posix
pipe-posix-tests
pthread-h
pthread-h-tests
pthread-thread
pthread-thread-tests
pthread_sigmask
pthread_sigmask-tests
putenv
raise
raise-tests
  regex
regex-tests
same-inode
sched
sched-tests
select
select-tests
setenv
setenv-tests
setlocale
setlocale-null
setlocale-null-tests
setlocale-tests
setsockopt
setsockopt-tests
sigaction
sigaction-tests
signal-h
signal-h-tests
sigprocmask
sigprocmask-tests
sleep
sleep-tests
snippet/_Noreturn
snippet/arg-nonnull
snippet/c++defs
snippet/warn-on-use
socket
socketlib
sockets
sockets-tests
socklen
ssize_t
stat
stat-tests
stat-time
stat-time-tests
std-gnu11
stdalign
stdalign-tests
stdbool
stdbool-tests
stddef
stddef-tests
stdint
stdint-tests
stdio
stdio-tests
stdlib
stdlib-tests
streq
strerror
strerror-override
strerror-tests
strerror_r-posix
strerror_r-posix-tests
string
string-tests
symlink
symlink-tests
sys_ioctl
sys_ioctl-tests
sys_select
sys_select-tests
sys_socket
sys_socket-tests
sys_stat
sys_stat-tests
sys_time
sys_time-tests
sys_types
sys_types-tests
sys_uio
sys_uio-tests
test-framework-sh
test-framework-sh-tests
thread
thread-optim
thread-tests
threadlib
time
time-tests
unistd
unistd-tests
unsetenv
unsetenv-tests
usleep
usleep-tests
verify
verify-tests
wchar
wchar-tests
wcrtomb
wcrtomb-tests
wctob
wctomb
wctype-h
wctype-h-tests
windows-mutex
windows-once
windows-recmutex
windows-rwlock
windows-thread
windows-tls
xalloc-oversized
yield
File list:
  lib/_Noreturn.h
  lib/accept.c
  lib/alloca.c
  lib/alloca.in.h
  lib/arg-nonnull.h
  lib/arpa_inet.in.h
  lib/attribute.h
  lib/binary-io.c
  lib/binary-io.h
  lib/bind.c
  lib/btowc.c
  lib/c++defs.h
  lib/c-ctype.c
  lib/c-ctype.h
  lib/c-strcase.h
  lib/c-strcasecmp.c
  lib/c-strncasecmp.c
  lib/cdefs.h
  lib/cloexec.c
  lib/cloexec.h
  lib/close.c
  lib/connect.c
  lib/ctype.in.h
  lib/dup2.c
  lib/errno.in.h
  lib/fcntl.c
  lib/fcntl.in.h
  lib/fd-hook.c
  lib/fd-hook.h
  lib/fdopen.c
  lib/filename.h
  lib/flexmember.h
  lib/fstat.c
  lib/ftruncate.c
  lib/get

Re: cmp/diff

[Simon Josefsson via Gnulib discussion list]

lör 2020-12-26 klockan 11:12 -0800 skrev Paul Eggert:
> On 12/26/20 10:49 AM, Bruno Haible wrote:
> > > It would be nice to reduce that set of tools.
> > Well, both 'cmp' and 'diff' are part of POSIX:
> 
> Even awk is on that list, and it's much fancier than cmp or diff. And
> I wouldn't support removing awk either, as it's a handy tool in
> Makefiles when the shell isn't up to a text-processing task.

Okay, let's wait ten years and revisit that :-)

I believe awk will be more of a rarity in the future.

> Looking at that list now, the only obvious tool to remove is egrep, 
> since 'grep -E' subsumes it nowadays.

Yes, I think that should be dropped.

What about 'install-info'?  It is quite GNU-specific.  Nothing in
gnulib appears to refer to it (except for the make-stds manual..).

/Simon



signature.asc
Description: This is a digitally signed message part

Re: cmp/diff

[Paul Eggert]

On 12/26/20 1:55 PM, Simon Josefsson wrote:

Looking at that list now, the only obvious tool to remove is egrep,
since 'grep -E' subsumes it nowadays.

Yes, I think that should be dropped.

What about 'install-info'?  It is quite GNU-specific.  Nothing in
gnulib appears to refer to it (except for the make-stds manual..).


Sure, attached is a proposed patch to make-stds.texi. This patch also 
removes fgrep, which is in the same category as egrep. I am sending this 
email to bug-standards so that they can see it as a suggestion.


Also, it puzzles me that the set of pre-installation and 
post-installation commands is a superset of the set of commands that can 
appear in "the Makefile rules for building and installation". Since pre- 
and post-installation commands appear in Makefiles, shouldn't their 
commands be a subset, not a superset?
Index: make-stds.texi
===
RCS file: /sources/gnustandards/gnustandards/make-stds.texi,v
retrieving revision 1.69
diff -c -r1.69 make-stds.texi
*** make-stds.texi  1 Jun 2020 17:53:39 -   1.69
--- make-stds.texi  26 Dec 2020 23:08:50 -
***
*** 160,166 
  @c mkfifo mknod tee uname
  
  @example
! awk cat cmp cp diff echo egrep expr false grep install-info ln ls
  mkdir mv printf pwd rm rmdir sed sleep sort tar test touch tr true
  @end example
  
--- 160,166 
  @c mkfifo mknod tee uname
  
  @example
! awk cat cmp cp diff echo expr false grep ln ls
  mkdir mv printf pwd rm rmdir sed sleep sort tar test touch tr true
  @end example
  
***
*** 174,179 
--- 174,182 
  for parallel execution.  For a list of known incompatibilities, see
  @ref{Portable Shell, Portable Shell Programming,, autoconf, Autoconf}.
  
+ Programs like @code{egrep} and @code{install-info} can be used in
+ rules that have a fallback if the programs are absent.
+ @xref{Standard Targets} for an example of this.
  
  It is a good idea to avoid creating symbolic links in makefiles, since a
  few file systems don't support them.
***
*** 1131,1142 
  which of the dependencies actually run.
  
  Pre-installation and post-installation commands should not run any
! programs except for these:
  
  @example
  [ basename bash cat chgrp chmod chown cmp cp dd diff echo
! egrep expand expr false fgrep find getopt grep gunzip gzip
! hostname install install-info kill ldconfig ln ls md5sum
  mkdir mkfifo mknod mv printenv pwd rm rmdir sed sort tee
  test touch true uname xargs yes
  @end example
--- 1134,1145 
  which of the dependencies actually run.
  
  Pre-installation and post-installation commands should not run any
! programs directly except for these:
  
  @example
  [ basename bash cat chgrp chmod chown cmp cp dd diff echo
! expand expr false find getopt grep gunzip gzip
! hostname install kill ldconfig ln ls md5sum
  mkdir mkfifo mknod mv printenv pwd rm rmdir sed sort tee
  test touch true uname xargs yes
  @end example

Re: cmp/diff

[Alfred M. Szmidt]

   >> Looking at that list now, the only obvious tool to remove is egrep,
   >> since 'grep -E' subsumes it nowadays.
   > Yes, I think that should be dropped.

Maybe from usage in scripts or Makefile, but from a users point these
do not replace "grep -E" or "grep -F" which are far more annoying to
type than egrep or fgrep.

Re: cmp/diff

[Alfred M. Szmidt]

   > What about 'install-info'?  It is quite GNU-specific.  Nothing in
   > gnulib appears to refer to it (except for the make-stds manual..).

install-info is specific to GNU yes, the GNU standards are for the GNU
system and GNU project after all.

install-info does not have an replacement, like say egrep/fgrep --
this is how we install a dir entry for a info manual.  Removing
install-info would be a regression.

Re: gnulib-tool --conditional-dependencies --test problem

[Bruno Haible]

Hi Paul,

> When I run this command:
> 
> ./gnulib-tool --conditional-dependencies --test regex
> 
> on Fedora 33, it fails as shown in the attached file.

Fixed through the patch below.

> The problem occurs 
> because --test brings in binary-io for some reason, but doesn't bring in 
> the fcntl-h module that binary-io depends on.

Not exactly. The problem occurs because gltests/Makefile.am attempts to
compile all tests, even those of the modules that are not enabled.

> A simple workaround would be to have gnulib-tool's --test option disable 
> the conditionalization of dependencies, or to have gnulib-tool diagnose 
> and fail any attempt to combine the two options

We removed the diagnostic
  "gnulib-tool: option --conditional-dependencies is not supported with 
--with-tests"
on 2017-12-28, when we thought that it works. Obviously, it hasn't been
tested a lot.


2020-12-26  Bruno Haible  

gnulib-tool: Make --conditional-dependencies work with --with-tests.
Reported by Paul Eggert in
.
* gnulib-tool (func_emit_tests_Makefile_am): Handle conditional
dependencies like func_emit_lib_Makefile_am.

diff --git a/gnulib-tool b/gnulib-tool
index 8b7e305..faed693 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -3634,7 +3634,6 @@ func_update_file ()
 # - uses_subdirsnonempty if object files in subdirs exist
 func_emit_lib_Makefile_am ()
 {
-
   # When using GNU make, or when creating an includable Makefile.am snippet,
   # augment variables with += instead of assigning them.
   if $gnu_make || test -n "$makefile_name"; then
@@ -4028,7 +4027,7 @@ func_emit_tests_Makefile_am ()
   fi
   if test -n "$module"; then
 {
-  func_get_automake_snippet "$module" |
+  func_get_automake_snippet_conditional "$module" |
 LC_ALL=C \
 sed -e 's,lib_LIBRARIES,lib%_LIBRARIES,g' \
 -e 's,lib_LTLIBRARIES,lib%_LTLIBRARIES,g' \
@@ -4042,9 +4041,13 @@ func_emit_tests_Makefile_am ()
 echo "libtests_a_LIBADD += @ALLOCA@"
 echo "libtests_a_DEPENDENCIES += @ALLOCA@"
   fi
-} > "$tmp"/amsnippet
+} | combine_lines "libtests_a_SOURCES" > "$tmp"/amsnippet1
+{
+  func_get_automake_snippet_unconditional "$module" |
+LC_ALL=C sed -e 's,lib_\([A-Z][A-Z]*\),libtests_a_\1,g'
+} > "$tmp"/amsnippet2
 # Skip the contents if it's entirely empty.
-if grep '[^ ]' "$tmp"/amsnippet > /dev/null ; then
+if grep '[^ ]' "$tmp"/amsnippet1 "$tmp"/amsnippet2 > /dev/null ; 
then
   # Mention long-running tests at the end.
   ofd=3
   for word in `func_get_status "$module"`; do
@@ -4056,9 +4059,28 @@ func_emit_tests_Makefile_am ()
   { echo "## begin gnulib module $module"
 if $gnu_make; then
   echo "ifeq (,\$(OMIT_GNULIB_MODULE_$module))"
+  convert_to_gnu_make='s/^if \(.*\)/ifneq (,$(\1))/'
+else
+  convert_to_gnu_make=
 fi
 echo
-cat "$tmp"/amsnippet
+if test "$cond_dependencies" = true; then
+  if func_cond_module_p "$module"; then
+func_module_conditional_name "$module"
+if $gnu_make; then
+  echo "ifneq (,\$($conditional))"
+else
+  echo "if $conditional"
+fi
+  fi
+fi
+sed "$convert_to_gnu_make" "$tmp"/amsnippet1
+if test "$cond_dependencies" = true; then
+  if func_cond_module_p "$module"; then
+echo "endif"
+  fi
+fi
+sed "$convert_to_gnu_make" "$tmp"/amsnippet2
 if $gnu_make; then
   echo "endif"
 fi
@@ -4066,7 +4088,7 @@ func_emit_tests_Makefile_am ()
 echo
   } >&$ofd
 fi
-rm -f "$tmp"/amsnippet
+rm -f "$tmp"/amsnippet1 "$tmp"/amsnippet2
 # Test whether there are some source files in subdirectories.
 for f in `func_get_filelist "$module"`; do
   case $f in
@@ -6255,7 +6277,7 @@ func_create_testdir ()
   # Note:
   # If $single_configure, we use the module lists $main_modules and
   # $testsrelated_modules; $modules is merely a temporary variable.
-  # Whereas if ! $configure, the module list is $modules.
+  # Whereas if ! $single_configure, the module list is $modules.
 
   # Show banner notice of every module.
   if $single_configure; then

Re: cmp/diff

[Paul Eggert]

On 12/26/20 4:07 PM, Alfred M. Szmidt wrote:

install-info does not have an replacement, like say egrep/fgrep --
this is how we install a dir entry for a info manual.  Removing
install-info would be a regression.


In practice, GNU installation procedures use install-info in the way 
that's described in the proposed patch: they test whether install-info 
is available, and if so they use it. The make-stds.texi file already 
recomments this practice in its "Standard Targets" section. The proposed 
patch is doing merely making make-stds coherent; it's not advocating any 
change to existing best practice for install-info.

Re: egrep, fgrep, and install-info

[Bruno Haible]

Paul Eggert wrote:
> ***
> *** 174,179 
> --- 174,182 
>   for parallel execution.  For a list of known incompatibilities, see
>   @ref{Portable Shell, Portable Shell Programming,, autoconf, Autoconf}.
>   
> + Programs like @code{egrep} and @code{install-info} can be used in
> + rules that have a fallback if the programs are absent.
> + @xref{Standard Targets} for an example of this.
>   
>   It is a good idea to avoid creating symbolic links in makefiles, since a
>   few file systems don't support them.

When you remove 'egrep', 'fgrep', or 'install-info' from the previously
allowed list of commands, everyone will ask: "ok, so now what replacement
should I use?"

1) For egrep and fgrep, probably the answer would be "use 'grep -E' or
'grep -F', respectively" — because POSIX supports these options.


Unfortunately, Solaris 10 doesn't:

$ type grep
grep is /usr/bin/grep
$ grep -F haible /etc/passwd
grep: illegal option -- F
Usage: grep -hblcnsviw pattern file . . .
$ grep -E '(haible|eggert)' /etc/passwd
grep: illegal option -- E
Usage: grep -hblcnsviw pattern file . . .

Likewise for /usr/5bin/grep.

2) For 'install-info', what would be the replacement? Even though Automake
knows how to handle its absence [1], it would be good to know.

Bruno

[1] https://www.gnu.org/software/automake/manual/html_node/Texinfo.html

Re: cmp/diff

[Alfred M. Szmidt]

   On 12/26/20 4:07 PM, Alfred M. Szmidt wrote:
   > install-info does not have an replacement, like say egrep/fgrep --
   > this is how we install a dir entry for a info manual.  Removing
   > install-info would be a regression.

   In practice, GNU installation procedures use install-info in the
   way that's described in the proposed patch: they test whether
   install-info is available, and if so they use it.  The
   make-stds.texi file already recomments this practice in its
   "Standard Targets" section. The proposed patch is doing merely
   making make-stds coherent; it's not advocating any change to
   existing best practice for install-info.

The example entry in '(standards) Standard Targets' is I think
orthogonal, it is for the benefit of the user where installing the
node entry is not considered an error but only a "warning" (and then
some extra checks because we want to treat real errors as such).
Which is quite different from the behaviour we have for other programs
-- if you are missing md5sum (or even fgrep) you'd get a hard error.

So I really don't see how it makes it coherent, having install-info as
a "safe" requirement makes logical sense since our prefered
documentation format is info (the original rationale for removing
install-info was "its GNU specific"), and why we do some extra sanity
checks is to be nice to the user.  The change also removes
install-info the only rule where it makes sense to use install-info --
post-installation.

Re: egrep, fgrep, and install-info

[Alfred M. Szmidt]

   2) For 'install-info', what would be the replacement? Even though
   Automake knows how to handle its absence [1], it would be good to
   know.

On systems where install-info is lacking, we simply don't install a
node entry in the dir file and continue with the post-install.  This
is a slightly different behaviour than what we have for other programs
which we generally consider a hard error.  E.g., if you don't have
grep -E, and no egrep or some other such ... you're shit outta luck.