GNU bug report logs -
#17981
[PATCH] maint.mk: less syntax-check noise when SIGPIPE is ignored
Previous Next
Reported by: Eric Blake <eblake <at> redhat.com>
Date: Wed, 9 Jul 2014 23:47:02 UTC
Severity: wishlist
Tags: patch
Done: Paul Eggert <eggert <at> cs.ucla.edu>
Bug is archived. No further changes may be made.
Full log
View this message in rfc822 format
[Message part 1 (text/plain, inline)]
Your bug report
#17981: [PATCH] maint.mk: less syntax-check noise when SIGPIPE is ignored
which was filed against the grep package, has been closed.
The explanation is attached below, along with your original report.
If you require more details, please reply to 17981 <at> debbugs.gnu.org.
--
17981: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17981
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems
[Message part 2 (message/rfc822, inline)]
On 7/11/14 2:12 PM, Paul Eggert wrote:
> Yes, that's the basic idea. Sorry I did not explain it clearly enough. Please
> feel free to forward this on to the Austin group.
Later Austin group commentary
<https://www.austingroupbugs.net/view.php?id=789#c2689> suggested that this
issue is no reason to change how 'grep' behaves, so I'm closing the grep bug report.
[Message part 3 (message/rfc822, inline)]
For a project with enough files, such as libvirt, vc-list-files
can produce so much input that it can lead to SIGPIPE to earlier
parts of a pipeline when later parts do a quick filter. Also,
many buildbot environments (annoyingly) ignore SIGPIPE, which
causes a number of tools to be rather chatty about reporting
EPIPE write failures. It doesn't help that POSIX has standardized
that the shell is unable to revert SIGPIPE to unignored status
if it inherits it as ignored - otherwise, the solution would just
be to re-enable SIGPIPE anywhere we expect to benefit from early
filtering exits. Here's a short demonstration:
$ ( trap '' PIPE; build-aux/vc-list-files | grep -l '\.c$' >/dev/null)
sed: couldn't write 16 items to stdout: Broken pipe
and a link to the much larger buildbot results against libvirt
which provoked this patch:
http://honk.sigxcpu.org:8001/job/libvirt-syntax-check/2465/
But look at the above example: we are piping data to grep -l,
and then discarding that output. At most, data | grep -l will
output "(standard input)", and exit early if the first match
is found before the end of a page (causing SIGPIPE to the process
feeding the pipe). It makes much more sense to use grep -l when
searching for a subset of files that have a match among a larger
set of file names passed as arguments, and NOT when used to
filter stdin. Sure, we're burning a bit more CPU power by
processing the full list instead of exiting early, but at least
it cuts down on the noise.
* top/maint.mk (_sc_header_without_use)
(sc_require_config_h_first): Parse full list.
Signed-off-by: Eric Blake <eblake <at> redhat.com>
---
I'll push this to gnulib to work around the issue. But it really
begs the question - can sed and grep be taught a way to silently
ignore EPIPE errors?
See also http://austingroupbugs.net/view.php?id=789, which is
considering standardizing the shell's 'set -o pipefail', and where
it becomes vital to be able to exit with 0 status when used on
the left side of a pipe if the only reason we are exiting early
is because the right side of the pipe is also exiting early without
consuming everything we are shoving into the pipe. It is unclear
at this point whether POSIX would recommend that filter
applications should _always_ exit with 0 status on pipe failure,
or only do this for EPIPE write failures when SIGPIPE is ignored,
or whether it should be optional behavior that must be explicitly
enabled via a command-line option and/or system-wide environment
variable. But the point remains that among all possible write
failures, the failure to write to a pipe is often expected as part
of an optimized pipeline in order to reduce CPU usage, and there
should be a way to handle it silently.
ChangeLog | 6 ++++++
top/maint.mk | 4 ++--
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index e793866..a93f468 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2014-07-08 Eric Blake <eblake <at> redhat.com>
+
+ maint.mk: less syntax-check noise when SIGPIPE is ignored
+ * top/maint.mk (_sc_header_without_use)
+ (sc_require_config_h_first): Parse full list.
+
2014-06-27 Paul Eggert <eggert <at> cs.ucla.edu>
mktime: merge #if/#ifdef usage from glibc
diff --git a/top/maint.mk b/top/maint.mk
index 3f369b7..0cc769c 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -440,7 +440,7 @@ sc_require_config_h:
# You must include <config.h> before including any other header file.
# This can possibly be via a package-specific header, if given by cfg.mk.
sc_require_config_h_first:
- @if $(VC_LIST_EXCEPT) | grep -l '\.c$$' > /dev/null; then \
+ @if $(VC_LIST_EXCEPT) | grep '\.c$$' > /dev/null; then \
fail=0; \
for i in $$($(VC_LIST_EXCEPT) | grep '\.c$$'); do \
grep '^# *include\>' $$i | $(SED) 1q \
@@ -464,7 +464,7 @@ sc_prohibit_HAVE_MBRTOWC:
define _sc_header_without_use
dummy=; : so we do not need a semicolon before each use; \
h_esc=`echo '[<"]'"$$h"'[">]'|$(SED) 's/\./\\\\./g'`; \
- if $(VC_LIST_EXCEPT) | grep -l '\.c$$' > /dev/null; then \
+ if $(VC_LIST_EXCEPT) | grep '\.c$$' > /dev/null; then \
files=$$(grep -l '^# *include '"$$h_esc" \
$$($(VC_LIST_EXCEPT) | grep '\.c$$')) && \
grep -LE "$$re" $$files | grep . && \
--
1.9.3
This bug report was last modified 4 years and 259 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.