For a project with enough files, such as libvirt, vc-list-files can produce so much input that it can lead to SIGPIPE to earlier parts of a pipeline when later parts do a quick filter. Also, many buildbot environments (annoyingly) ignore SIGPIPE, which causes a number of tools to be rather chatty about reporting EPIPE write failures. It doesn't help that POSIX has standardized that the shell is unable to revert SIGPIPE to unignored status if it inherits it as ignored - otherwise, the solution would just be to re-enable SIGPIPE anywhere we expect to benefit from early filtering exits. Here's a short demonstration:
$ ( trap '' PIPE; build-aux/vc-list-files | grep -l '\.c$' >/dev/null) sed: couldn't write 16 items to stdout: Broken pipe and a link to the much larger buildbot results against libvirt which provoked this patch: http://honk.sigxcpu.org:8001/job/libvirt-syntax-check/2465/ But look at the above example: we are piping data to grep -l, and then discarding that output. At most, data | grep -l will output "(standard input)", and exit early if the first match is found before the end of a page (causing SIGPIPE to the process feeding the pipe). It makes much more sense to use grep -l when searching for a subset of files that have a match among a larger set of file names passed as arguments, and NOT when used to filter stdin. Sure, we're burning a bit more CPU power by processing the full list instead of exiting early, but at least it cuts down on the noise. * top/maint.mk (_sc_header_without_use) (sc_require_config_h_first): Parse full list. Signed-off-by: Eric Blake <ebl...@redhat.com> --- I'll push this to gnulib to work around the issue. But it really begs the question - can sed and grep be taught a way to silently ignore EPIPE errors? See also http://austingroupbugs.net/view.php?id=789, which is considering standardizing the shell's 'set -o pipefail', and where it becomes vital to be able to exit with 0 status when used on the left side of a pipe if the only reason we are exiting early is because the right side of the pipe is also exiting early without consuming everything we are shoving into the pipe. It is unclear at this point whether POSIX would recommend that filter applications should _always_ exit with 0 status on pipe failure, or only do this for EPIPE write failures when SIGPIPE is ignored, or whether it should be optional behavior that must be explicitly enabled via a command-line option and/or system-wide environment variable. But the point remains that among all possible write failures, the failure to write to a pipe is often expected as part of an optimized pipeline in order to reduce CPU usage, and there should be a way to handle it silently. ChangeLog | 6 ++++++ top/maint.mk | 4 ++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/ChangeLog b/ChangeLog index e793866..a93f468 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +2014-07-08 Eric Blake <ebl...@redhat.com> + + maint.mk: less syntax-check noise when SIGPIPE is ignored + * top/maint.mk (_sc_header_without_use) + (sc_require_config_h_first): Parse full list. + 2014-06-27 Paul Eggert <egg...@cs.ucla.edu> mktime: merge #if/#ifdef usage from glibc diff --git a/top/maint.mk b/top/maint.mk index 3f369b7..0cc769c 100644 --- a/top/maint.mk +++ b/top/maint.mk @@ -440,7 +440,7 @@ sc_require_config_h: # You must include <config.h> before including any other header file. # This can possibly be via a package-specific header, if given by cfg.mk. sc_require_config_h_first: - @if $(VC_LIST_EXCEPT) | grep -l '\.c$$' > /dev/null; then \ + @if $(VC_LIST_EXCEPT) | grep '\.c$$' > /dev/null; then \ fail=0; \ for i in $$($(VC_LIST_EXCEPT) | grep '\.c$$'); do \ grep '^# *include\>' $$i | $(SED) 1q \ @@ -464,7 +464,7 @@ sc_prohibit_HAVE_MBRTOWC: define _sc_header_without_use dummy=; : so we do not need a semicolon before each use; \ h_esc=`echo '[<"]'"$$h"'[">]'|$(SED) 's/\./\\\\./g'`; \ - if $(VC_LIST_EXCEPT) | grep -l '\.c$$' > /dev/null; then \ + if $(VC_LIST_EXCEPT) | grep '\.c$$' > /dev/null; then \ files=$$(grep -l '^# *include '"$$h_esc" \ $$($(VC_LIST_EXCEPT) | grep '\.c$$')) && \ grep -LE "$$re" $$files | grep . && \ -- 1.9.3
