On 2/15/19 8:43 AM, 積丹尼 Dan Jacobson wrote: > Things start out cheery, but quickly get ugly, > > $ for i in 9 99 999 9999 99999; do seq $i|sort -n|sed 5q|wc -l; done > 5 > 5 > 5 > 5 > sort: write failed: 'standard output': Broken pipe > sort: write error > 5 > sort: write failed: 'standard output': Broken pipe > sort: write error
Your code is demonstrating what happens when sed ends processing without consuming all of sort's output, to the point that sort is producing more output than what stdio can buffer, and thus with enough pending output will trigger a situation of SIGPIPE - but when SIGPIPE is ignored, that instead turns into an EPIPE failure to write(), and sort treats all write() failures as worthy of error reporting. If you want sort to die silently, then don't ignore SIGPIPE, so that sort won't see EPIPE and thus won't be noisy. $ (trap '' PIPE; seq 9999 | sort -n | sed 5q | wc -l) 5 sort: write failed: 'standard output': Broken pipe sort: write error $ (trap - PIPE; seq 9999 | sort -n | sed 5q | wc -l) 5 Except that POSIX has the nasty requirement that sh started with an inherited ignored SIGPIPE must silently ignore all attempts from within the shell to restore SIGPIPE handling to child processes of the shell: $ (trap '' PIPE; bash -c 'trap - PIPE; \ seq 9999 | sort -n | sed 5q | wc -l') 5 sort: write failed: 'standard output': Broken pipe sort: write error And unfortunately, there are several common cases of badly-behaved environment setups that leave SIGPIPE ignored in the child (for example, a quick google search found this: https://blog.nelhage.com/2010/02/a-very-subtle-bug/) You HAVE to use some other intermediate program if you want to override an inherited ignored SIGPIPE in sh into an inherited default-behavior SIGPIPE in sort. Perhaps coreutils should teach 'env' a command-line option to forcefully reset SIGPIPE back to default behavior (or add a new coreutil that does the same idea), as a way to work around POSIX' requirement on sh. If we did that, then even if your sh is started with SIGPIPE ignored (so that the shell itself can't restore default behavior), you could do this theoretical invocation: $ seq 9999 | env --default-signal PIPE sort -n | sed 5q | wc -l 5 and avoid the EPIPE failures because sort is forced to start with SIGPIPE handling rather than ignored. > > Therefore, kindly add a sort --limit=n, Not scalable. The problem you encountered is NOT the fault of sort, but is common to ALL utilities which dutifully report write() failures on EPIPE errors, which in turn happens when SIGPIPE is ignored. Adding an option to every such utility that produces outtput is not a good use of time. If you want to change things universally, perhaps you should petition POSIX to change the requirements to allow applications the option to silently exit without an error message on EPIPE failures to write(), instead of the current wording that all write() errors must be diagnosed. > and/or on (info "(coreutils) sort invocation") > admit the problem, and give some workarounds, lest > our scripts occasionally spew error messages seemingly randomly, > just when the boss is looking. The problem is not in sort, but in the fact that your environment is ignoring SIGPIPE. Documenting something in sort doesn't scale, but perhaps the documentation could mention SIGPIPE considerations in a more global chapter covering all of the coreutils. > > And no fair saying "just save the output" (could be big) "into a file > first, and do head(1) or sed(1) on that." > If you have an app that exits noisily on write failures to an early-exit pipe, your solutions are to quit ignoring SIGPIPE, or to change the pipeline to consume all of the output instead of exiting early and causing write failure. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
signature.asc
Description: OpenPGP digital signature
