GNU bug report logs -
#22195
deviation from POSIX in tee
Previous Next
Reported by: Eric Renouf <erenouf <at> skaion.com>
Date: Thu, 17 Dec 2015 20:35:01 UTC
Severity: normal
Tags: notabug
Done: Assaf Gordon <assafgordon <at> gmail.com>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 22195 in the body.
You can then email your comments to 22195 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-coreutils <at> gnu.org
:
bug#22195
; Package
coreutils
.
(Thu, 17 Dec 2015 20:35:01 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Eric Renouf <erenouf <at> skaion.com>
:
New bug report received and forwarded. Copy sent to
bug-coreutils <at> gnu.org
.
(Thu, 17 Dec 2015 20:35:01 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
When using tee with process substitution, if the process exits early it
will cause tee to get a SIGPIPE after trying to write to the closed named
pipe (per man 2 write). This causes tee to exit without finishing writing
to any additional files or processes, which seems to be a violation of the
POSIX standard at
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/tee.html
specifically the Consequences of Errors section which states
"If a write to any successfully opened file operand fails, writes to other
successfully opened file operands and standard output shall continue, but
the exit status shall be non-zero. Otherwise, the default actions
specified in Utility Description Defaults apply."
To recreate the problem I ran the following under bash:
for i in {1..300}; do
echo "$i"
echo "$i" >&2
sleep 1
done | tee >(head -1 > h.txt; echo "Head done") \
>(tail -1 > t.txt) >/dev/null
And I see output like the following:
1
Head done
2
and then it exits with echo "${PIPESTATUS[@]}" showing
141 141
and t.txt from the second process substitution only having "1" in it
This was found from a Stack Overflow question at
http://stackoverflow.com/questions/34340706/incorrect-results-with-bash-process-substitution-and-tail
I have verified this behavior with tee (GNU coreutils) 8.5 and tee (GNU
coreutils) 8.13
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#22195
; Package
coreutils
.
(Fri, 18 Dec 2015 03:23:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 22195 <at> debbugs.gnu.org (full text, mbox):
Eric Renouf wrote:
> If a write to any successfully opened file operand fails
But the write didn't fail here. Instead, a signal was sent to 'tee'. If you
don't want the signal, trap it. E.g.:
trap '' PIPE
for i in {1..300}; do
echo "$i"
echo "$i" >&2
sleep 1
done | tee >(head -1 > h.txt; echo "Head done") \
>(tail -1 > t.txt) >/dev/null
will give the behavior you want.
So there is no deviation from POSIX here.
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#22195
; Package
coreutils
.
(Fri, 18 Dec 2015 07:20:01 GMT)
Full text and
rfc822 format available.
Message #11 received at 22195 <at> debbugs.gnu.org (full text, mbox):
On 12/18/2015 04:22 AM, Paul Eggert wrote:
> Eric Renouf wrote:
>> If a write to any successfully opened file operand fails
>
> But the write didn't fail here. Instead, a signal was sent to 'tee'. If you
> don't want the signal, trap it. E.g.:
>
> trap '' PIPE
> for i in {1..300}; do
> echo "$i"
> echo "$i" >&2
> sleep 1
> done | tee >(head -1 > h.txt; echo "Head done") \
> >(tail -1 > t.txt) >/dev/null
>
> will give the behavior you want.
>
> So there is no deviation from POSIX here.
Furthermore, tee got the new --output-error=MODE option in
version 8.24 to have more control over this:
$ src/tee --help
...
-p diagnose errors writing to non pipes
--output-error[=MODE] set behavior on write error. See MODE below
...
MODE determines behavior with write errors on the outputs:
'warn' diagnose errors writing to any output
'warn-nopipe' diagnose errors writing to any output not a pipe
'exit' exit on error writing to any output
'exit-nopipe' exit on error writing to any output not a pipe
The default MODE for the -p option is 'warn-nopipe'.
The default operation when --output-error is not specified, is to
exit immediately on error writing to a pipe, and diagnose errors
writing to non pipe outputs.
A minor note to your command: instead of redirecting tee's output to
/dev/null, you could use another pipe to save the write operations
in tee to it:
producer \
| tee --output-error=warn \
>(head -1 > h.txt; echo "Head done") \
| tail -1 > t.txt
Have a nice day,
Berny
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#22195
; Package
coreutils
.
(Fri, 18 Dec 2015 11:14:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 22195 <at> debbugs.gnu.org (full text, mbox):
tag 22195 notabug
close 22195
stop
On 18/12/15 03:22, Paul Eggert wrote:
> Eric Renouf wrote:
>> If a write to any successfully opened file operand fails
>
> But the write didn't fail here. Instead, a signal was sent to 'tee'. If you
> don't want the signal, trap it. E.g.:
>
> trap '' PIPE
> for i in {1..300}; do
> echo "$i"
> echo "$i" >&2
> sleep 1
> done | tee >(head -1 > h.txt; echo "Head done") \
> >(tail -1 > t.txt) >/dev/null
>
> will give the behavior you want.
>
> So there is no deviation from POSIX here.
Generally you don't want to ignore SIGPIPE.
http://pixelbeat/programming/sigpipe_handling.html
as then you have to deal with EPIPE from write():
$ trap '' PIPE
$ seq 100000 | tee >(head -n1) > >(sed -n '/10000/{p;q}')
1
10000
seq: write error
As Bernhard said, the new -p option is a solution to this:
$ seq 100000 | tee >(head -n1) > >(tail -n1)
1
14139
$ seq 100000 | tee -p >(head -n1) > >(tail -n1)
1
100000
Note that's an option, rather than the default behavior
as POSIX states to handle SIGPIPE in the default manner,
and more importantly to support existing cases like the
following (yes we don't support both modes of operation):
$ yes | tee log | timeout process
Since this is somewhat of a subjective default,
and this issue pops up so frequently, I've just added it to:
http://www.pixelbeat.org/docs/coreutils-gotchas.html#tee
cheers,
Pádraig
Information forwarded
to
bug-coreutils <at> gnu.org
:
bug#22195
; Package
coreutils
.
(Fri, 18 Dec 2015 18:39:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 22195 <at> debbugs.gnu.org (full text, mbox):
Pádraig Brady wrote:
> Paul Eggert wrote:
> > trap '' PIPE
>
> Generally you don't want to ignore SIGPIPE.
> http://pixelbeat/programming/sigpipe_handling.html
> as then you have to deal with EPIPE from write():
I wanted to add emphasis to this. Ignoring SIGPIPE causes a cascade
of associated problems. Best not to do it.
Bob
P.S. Typo alert:
http://pixelbeat/programming/sigpipe_handling.html
Should be:
http://www.pixelbeat.org/programming/sigpipe_handling.html
Added tag(s) notabug.
Request was from
Assaf Gordon <assafgordon <at> gmail.com>
to
control <at> debbugs.gnu.org
.
(Wed, 24 Oct 2018 21:37:02 GMT)
Full text and
rfc822 format available.
bug closed, send any further explanations to
22195 <at> debbugs.gnu.org and Eric Renouf <erenouf <at> skaion.com>
Request was from
Assaf Gordon <assafgordon <at> gmail.com>
to
control <at> debbugs.gnu.org
.
(Wed, 24 Oct 2018 21:37:02 GMT)
Full text and
rfc822 format available.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Thu, 22 Nov 2018 12:24:10 GMT)
Full text and
rfc822 format available.
This bug report was last modified 6 years and 262 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.