GNU bug report logs - #9089
pipe failure with cat and head of coreutils 6.12

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 9089 in the body.
You can then email your comments to 9089 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philipp Thomas <pth <at> suse.de>
To: bug-coreutils <at> gnu.org
Subject: pipe failure with cat and head of coreutils 6.12
Date: Fri, 15 Jul 2011 13:30:23 +0200

I'm trying to track down a bug in cat of coreutils 6.12. Doing 

cat /var/log/Xorg.0.log | head -n70

under ksh consistently fails with 'cat: write error: Connection reset by
peer'.  It does not fail when run under bash and it does not fail in current
coreutils .  It seems that ksh implementing pipes by socketpair(2) is the
reason for this failure.

Does somebody have a hint how to track this down? When trying to use strace
it won't fail and I don't know how to get this to run under gdb.

Philipp

Message #8 received at 9089 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Philipp Thomas <pth <at> suse.de>
Cc: 9089 <at> debbugs.gnu.org
Subject: Re: bug#9089: pipe failure with cat and head of coreutils 6.12
Date: Fri, 15 Jul 2011 07:03:00 -0600

[Message part 1 (text/plain, inline)]

On 07/15/2011 05:30 AM, Philipp Thomas wrote:
> I'm trying to track down a bug in cat of coreutils 6.12. Doing 
> 
> cat /var/log/Xorg.0.log | head -n70
> 
> under ksh consistently fails with 'cat: write error: Connection reset by
> peer'.  It does not fail when run under bash and it does not fail in current
> coreutils .  It seems that ksh implementing pipes by socketpair(2) is the
> reason for this failure.

And that behavior of ksh is probably a violation of POSIX:
http://austingroupbugs.net/view.php?id=205

There, the Austin Group discussion around the issue was that since
socketpair() creates observable differences from pipe(), including your
example (where socket disconnect behaves differently than EPIPE errors),
and many programs are not expecting these differences, that ksh should
probably be fixed to use pipe() like everyone else.

Conversely, if cat already has any special handling for EPIPE, then we
should probably do that same handling for socket disconnects.

> Does somebody have a hint how to track this down? When trying to use strace
> it won't fail and I don't know how to get this to run under gdb.

gdb --args ksh -c 'cat /var/log/Xorg.0.log | head -n70'

then use 'set follow-fork-mode child' at the appropriate point to get
gdb to move over to the cat subprocess.  Except that by default, you
won't have any time to put any breakpoints in cat after exec(), unless
you also recompile cat to inject a sleep() at the top of main() to give
you some reaction time to get gdb to put useful breaks into the cat
subprocess.

Painful, yes, but at least it would give you the same setup as what you
are trying to debug.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #11 received at 9089 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Eric Blake <eblake <at> redhat.com>
Cc: Philipp Thomas <pth <at> suse.de>, 9089 <at> debbugs.gnu.org
Subject: Re: bug#9089: pipe failure with cat and head of coreutils 6.12
Date: Fri, 15 Jul 2011 14:26:54 +0100

On 15/07/11 14:03, Eric Blake wrote:
> On 07/15/2011 05:30 AM, Philipp Thomas wrote:
>> I'm trying to track down a bug in cat of coreutils 6.12. Doing 
>>
>> cat /var/log/Xorg.0.log | head -n70
>>
>> under ksh consistently fails with 'cat: write error: Connection reset by
>> peer'.  It does not fail when run under bash and it does not fail in current
>> coreutils .  It seems that ksh implementing pipes by socketpair(2) is the
>> reason for this failure.
> 
> And that behavior of ksh is probably a violation of POSIX:
> http://austingroupbugs.net/view.php?id=205
> 
> There, the Austin Group discussion around the issue was that since
> socketpair() creates observable differences from pipe(), including your
> example (where socket disconnect behaves differently than EPIPE errors),
> and many programs are not expecting these differences, that ksh should
> probably be fixed to use pipe() like everyone else.

Note the reason ksh does this, is to allow commands
to efficiently inspect the input, rather than reading byte by byte.
This is related to the recent `stdbuf -i0` discussions.

> Conversely, if cat already has any special handling for EPIPE, then we
> should probably do that same handling for socket disconnects.

Probably

>> Does somebody have a hint how to track this down? When trying to use strace
>> it won't fail

Note it seems to be timing dependent whether ECONNRESET or EPIPE is returned.
To get ECONNRESET reliably I need to:

ksh -c "strace yes | (sleep 1; head -n10) > /dev/null"

cheers,
Pádraig.

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philipp Thomas <pth <at> suse.de>
To: bug-coreutils <at> gnu.org
Subject: Re: bug#9089: pipe failure with cat and head of coreutils 6.12
Date: Fri, 15 Jul 2011 17:02:38 +0200

* Pádraig Brady (P <at> draigBrady.com) [20110715 16:20]:

> Note the reason ksh does this, is to allow commands
> to efficiently inspect the input, rather than reading byte by byte.
> This is related to the recent `stdbuf -i0` discussions.

And that noticeable speedup is one of ksh's main advantages, at least
according to our ksh maintainer.

> Note it seems to be timing dependent whether ECONNRESET or EPIPE is returned.
> To get ECONNRESET reliably I need to:
> 
> ksh -c "strace yes | (sleep 1; head -n10) > /dev/null"

As far as I've come in tracing the failure is in gnulibs full_write and
neither simple_cat nor full_write seem to have any handling for either error
code.

Philipp

Message #17 received at 9089 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Thomas <pth <at> suse.de>
To: 9089 <at> debbugs.gnu.org
Subject: Re: bug#9089: pipe failure with cat and head of coreutils 6.12
Date: Fri, 15 Jul 2011 17:14:28 +0200

* Eric Blake (eblake <at> redhat.com) [20110715 15:03]:

> And that behavior of ksh is probably a violation of POSIX:
> http://austingroupbugs.net/view.php?id=205

that discussion is from 2006 and ksh has had that feature for quite a bit
longer. The reasin is that sockpairs are faster than pipes and and much
faster when piping large amounts of data.

And how about this comment from geoffclare on 2010-01-06 15:04:

<cite>
  I tried the latest ksh93 binary (93t+) and found that it uses a socket
  pair on Linux, but a pipe on OpenSolaris. (I originally tried 93s+ on
  Linux, then downloaded the latest to make sure it wasn't a difference
  between 93s+ and 93t+.) I assume there must be a good reason for it
  choosing to use a socket pair on some systems instead of a pipe, so I
  agree the wording should allow this behaviour.

  I suggest:

      "For each command but the last, the shell shall connect the
      standard output of the command to the standard input of the
      next command by creating either a pipe or a unidirectional
      socket pair (an unbound pair of connected sockets of type
      SOCK_STREAM with one of the pair shut down for sending and
      the other shut down for receiving - see [xref to socketpair()]
      and [xref to shutdown()]) and passing the writable end of
      the pipe or socket pair as the standard output of the command
      and the readable end of the pipe or socket pair as the standard
      input of the next command."
</cite>

Philipp

Message #20 received at 9089 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Philipp Thomas <pth <at> suse.de>
Cc: 9089 <at> debbugs.gnu.org
Subject: Re: bug#9089: pipe failure with cat and head of coreutils 6.12
Date: Fri, 15 Jul 2011 09:58:39 -0600

[Message part 1 (text/plain, inline)]

On 07/15/2011 09:14 AM, Philipp Thomas wrote:
> * Eric Blake (eblake <at> redhat.com) [20110715 15:03]:
> 
>> And that behavior of ksh is probably a violation of POSIX:
>> http://austingroupbugs.net/view.php?id=205
> 
> that discussion is from 2006 and ksh has had that feature for quite a bit
> longer. The reasin is that sockpairs are faster than pipes and and much
> faster when piping large amounts of data.
> 
> And how about this comment from geoffclare on 2010-01-06 15:04:

That comment was discussed, but rejected.  Notice how the bug states
that the final accepted text is
http://austingroupbugs.net/view.php?id=205#c385

For more details on the topic, refer to the mailing list archives around
that time:

https://www.opengroup.org/sophocles/show_archive.tpl?source=L&listname=austin-group-l&first=1&pagesize=80&searchstring=socketpair&zone=F

Including this message from David Korn, regarding ksh's position:

https://www.opengroup.org/sophocles/show_mail.tpl?CALLER=show_archive.tpl&source=L&listname=austin-group-l&id=13316

and this one explaining why the Austin Group does not like sockets:

https://www.opengroup.org/sophocles/show_mail.tpl?CALLER=show_archive.tpl&source=L&listname=austin-group-l&id=13315

It may be worth re-raising the issue with the Austin Group.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #23 received at 9089 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: David Korn <dgk <at> research.att.com>
Cc: gsf <at> research.att.com, dcragun <at> sonic.net, 9089 <at> debbugs.gnu.org,
	austin-group-l <at> opengroup.org
Subject: Re: [1003.1(2008)/Issue 7 0000205]: Shell pipeline connection
	requirement too vague
Date: Fri, 15 Jul 2011 11:04:28 -0600

[Message part 1 (text/plain, inline)]

According to David Korn on 1/7/2010 1:46 PM:
> cc: gsf <at> research.att.com austin-group-l <at> opengroup.org
> Subject: Re: Re: [1003.1(2008)/Issue 7 0000205]: Shell pipeline connection requirement too vague

Reviving an older Austin Group conversation, and adding GNU coreutils in cc:

> --------
> 
>> This topic was discussed during today's conference call.
>>
>> We did not reach any conclusion about how this should be resolved yet.
>>
>> But, as noted on a few messages on this alias on this topic, there was
>> agreement that many applications do depend on pipe behaviors (write
>> atomicity guarantees, generation of SIGPIPE signals, etc.) that are not
>> satisfied by a socketpair implementation.  We would much prefer to see
>> an explicit option or syntax extension to enable higher performance
>> pipes rather than having "|" silently provide behavior that is not what
>> would be expected from a pipe created by a call to pipe().
> If recv() were required to work on pipes, there would be no need
> for the shell to use socketpair().  On Linux (and I don't know if
> there are any other systems), recv() doesn't work on pipes.  However,
> socketpair() does generate SIGPIPE when writing to socket created
> by socketpair that doesn't have a reader.  I don't know whether
> sockpair() file descriptors have the same atomicity requirements
> as pipes, but I have never seen a bug filed against ksh along
> this line.

Here's an example bug report where ksh behavior is observable, resulting
in a bug report being issued against coreutils.  That is, using
socketpair() instead of pipe() risks introducing ECONNRESET instead of
EPIPE when the consumer goes away, with the result being subject to
scheduling races:
http://lists.gnu.org/archive/html/bug-coreutils/2011-07/msg00085.html.

Is cat required to treat ECONNRESET like EPIPE, or is ksh at fault here
for using socketpair()?

> The only problem with socketpair on Linux is that files that inherit
> a socketpair file descriptor cannot use /dev/fd/n to get an open
> file descriptor.  I think that this is a Linux bug.
>>
>> We understand that pipes require single-byte reads (at least for
>> non-STREAMs based pipes) and that this does provide a significant
>> performance penalty in some circumstances.  The committee would be
>> happy to consider enhancement requests to add relative seeks on pipes
>> up to the last read() (as David Korn mentioned in email sequence
>> #13298) or other features that would help alleviate this issue.
> As long as the standard requires a way to peek ahead or to
> seek back to the previous read(), then the shell can revert back
> to using pipes.
> The simplest change to the standard would be to require that recv()
> work on pipes as well as sockets.

Is this something we should pursue further, to get to a point where pipe()
is not so bad to use?

-- 
Don't work too hard, make some time for fun as well!

Eric Blake             ebb9 <at> byu.net

[signature.asc (application/pgp-signature, attachment)]

Message #26 received at 9089 <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Nieder <jrnieder <at> gmail.com>
To: Eric Blake <eblake <at> redhat.com>
Cc: gsf <at> research.att.com, David Korn <dgk <at> research.att.com>, dcragun <at> sonic.net,
	9089 <at> debbugs.gnu.org, austin-group-l <at> opengroup.org
Subject: Re: [1003.1(2008)/Issue 7 0000205]: Shell pipeline connection
	requirement too vague
Date: Fri, 15 Jul 2011 13:38:44 -0500

Hi,

Eric Blake wrote:
> According to David Korn on 1/7/2010 1:46 PM:

>> The simplest change to the standard would be to require that recv()
>> work on pipes as well as sockets.
>
> Is this something we should pursue further, to get to a point where pipe()
> is not so bad to use?

Does any Unix-like platform aside from Linux lack that feature?  Is
there any architectural consideration that would make MSG_PEEK
difficult to implement for Linux's pipes?  (For the former, I have no
clue, while for the latter, I suspect not.)

If not, I suppose we should take this to LKML.  Thanks; this sounds
like it would be a nice feature.

Regards,
Jonathan

Message #29 received at 9089 <at> debbugs.gnu.org (full text, mbox):

From: Garrett Wollman <wollman+austin-group <at> lcs.mit.edu>
To: Jonathan Nieder <jrnieder <at> gmail.com>
Cc: gsf <at> research.att.com, Eric Blake <eblake <at> redhat.com>,
	David Korn <dgk <at> research.att.com>, austin-group-l <at> opengroup.org,
	9089 <at> debbugs.gnu.org, dcragun <at> sonic.net
Subject: Re: [1003.1(2008)/Issue 7 0000205]: Shell pipeline connection
	requirement too vague
Date: Fri, 15 Jul 2011 14:47:19 -0400

<<On Fri, 15 Jul 2011 13:38:44 -0500, Jonathan Nieder <jrnieder <at> gmail.com> said:

> Does any Unix-like platform aside from Linux lack that feature?  Is
> there any architectural consideration that would make MSG_PEEK
> difficult to implement for Linux's pipes?  (For the former, I have no
> clue, while for the latter, I suspect not.)

On FreeBSD it will error out with [ENOTSOCK].

I would in general caution against going down this particular
direction.  Some systems implemented pipes on top of sockets, but this
is an implementation detail and applications which rely on that
behavior are in error.

-GAWollman

Message #32 received at 9089 <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Nieder <jrnieder <at> gmail.com>
To: Glenn Fowler <gsf <at> research.att.com>
Cc: eblake <at> redhat.com, dgk <at> research.att.com, austin-group-l <at> opengroup.org,
	9089 <at> debbugs.gnu.org, dcragun <at> sonic.net, wollman+austin-group <at> lcs.mit.edu
Subject: Re: [1003.1(2008)/Issue 7 0000205]: Shell pipeline connection
	requirement too vague
Date: Mon, 18 Jul 2011 14:15:37 -0500

Glenn Fowler wrote:

> we need a standard way to peek read data on a pipe without blocking
> or consuming the data
>
> we use ioctl(I_PEEK) on pipe() on systems that feature test ok for that
> and if not then recv(MSG_PEEK) on socketpair()
> otherwise the much slower "cant't peek pipe data" code kicks in
>
> rather than getting stuck on using a socket api for pipes
> can we first determine that peeking pipe read data is useful
> and then determine an api

Yes, sorry for the lack of clarity on my part.  Being able to peek
ahead when reading from a pipe would be very useful imho.  All I meant
is that and I was sloppy about the words used (i.e., the details of
the API do not matter much).

I will try to find time to mock up a pipe_peek() syscall (just as a
prototype) for Linux some time this week.  I would be happy if people
on other platforms without support for this (and others on Linux, too)
try something similar so we can compare notes and decide what to
standardize on.

Thanks.

Message #35 received at 9089 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Fowler <gsf <at> research.att.com>
To: jrnieder <at> gmail.com, wollman+austin-group <at> lcs.mit.edu
Cc: dcragun <at> sonic.net, dgk <at> research.att.com, 9089 <at> debbugs.gnu.org,
	eblake <at> redhat.com, austin-group-l <at> opengroup.org
Subject: Re: [1003.1(2008)/Issue 7 0000205]: Shell pipeline connection
	requirement too vague
Date: Mon, 18 Jul 2011 15:08:25 -0400

we need a standard way to peek read data on a pipe without blocking or consuming the data

we use ioctl(I_PEEK) on pipe() on systems that feature test ok for that
and if not then recv(MSG_PEEK) on socketpair()
otherwise the much slower "cant't peek pipe data" code kicks in

rather than getting stuck on using a socket api for pipes
can we first determine that peeking pipe read data is useful
and then determine an api

On Fri, 15 Jul 2011 14:47:19 -0400 Garrett Wollman wrote:
> <<On Fri, 15 Jul 2011 13:38:44 -0500, Jonathan Nieder <jrnieder <at> gmail.com> said:

> > Does any Unix-like platform aside from Linux lack that feature?  Is
> > there any architectural consideration that would make MSG_PEEK
> > difficult to implement for Linux's pipes?  (For the former, I have no
> > clue, while for the latter, I suspect not.)

> On FreeBSD it will error out with [ENOTSOCK].

> I would in general caution against going down this particular
> direction.  Some systems implemented pipes on top of sockets, but this
> is an implementation detail and applications which rely on that
> behavior are in error.

> -GAWollman

Message #38 received at 9089 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Philipp Thomas <pth <at> suse.de>, 9089 <at> debbugs.gnu.org
Subject: Re: bug#9089: pipe failure with cat and head of coreutils 6.12
Date: Fri, 18 Jan 2019 00:17:35 -0700

close 9089
stop

(triaging old bugs)

Hello,

On 2011-07-15 5:30 a.m., Philipp Thomas wrote:
> I'm trying to track down a bug in cat of coreutils 6.12. Doing
> 
> cat /var/log/Xorg.0.log | head -n70
> 
> under ksh consistently fails with 'cat: write error: Connection reset by
> peer'.  It does not fail when run under bash and it does not fail in current
> coreutils .

I'm still able to reproduce this problem (i.e. "cat: write error"
with coreutils 6.12 under ksh on Linux 4.9.0).

However,
Given that it is has been seven and a half years ago since the last
comment, and even then it was already acknowledged that the problem does
not happen in later versions, and it happens because ksh uses 
socketpairs instead of pipes - I'm closing this bug.

If the issue of cat(1) supporting socketpair/ECONNRESET instead of
pipes/EPIPE is still relevant, we can re-open the bug.

regards,
 - assaf

Message #43 received at 9089 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Assaf Gordon <assafgordon <at> gmail.com>, Philipp Thomas <pth <at> suse.de>,
 9089 <at> debbugs.gnu.org
Subject: Re: bug#9089: pipe failure with cat and head of coreutils 6.12
Date: Sat, 19 Jan 2019 10:38:54 -0800

Assaf Gordon wrote:
> If the issue of cat(1) supporting socketpair/ECONNRESET instead of
> pipes/EPIPE is still relevant, we can re-open the bug.

'cat' treats EPIPE on pipes the same way it treats ECONNRESET on socket pairs:

$ (trap '' PIPE; cat /usr/bin/emacs) | :
cat: write error: Broken pipe

The difference in observed behavior occurs only because most scripts (rightly) 
don't ignore SIGPIPE. If we change 'cat' to ignore ECONNRESET, for consistency 
we should also change it to ignore EPIPE.

The POSIX spec[1] allows 'write' to send a SIGPIPE and set errno to EPIPE for 
sockets that are no longer connected. I wonder why the Linux kernel doesn't do 
that? It would seem to be more consistent and useful.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.