GNU bug report logs - #18449
"cat x >> x" error even when x is empty

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 18449 in the body.
You can then email your comments to 18449 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Vincent Lefevre <vincent <at> vinc17.net>
To: bug-coreutils <at> gnu.org
Subject: "cat x >> x" error even when x is empty
Date: Thu, 11 Sep 2014 15:00:44 +0200

With coreutils 8.23 under Debian/unstable:

ypig% : > x
ypig% cat x >> x
cat: x: input file is output file
ypig% POSIXLY_CORRECT=1 cat x >> x
cat: x: input file is output file

while there's no reason to return an error in this case: the file
should just remain empty. Using the same file for input and output
isn't disallowed by POSIX, AFAIK.

This may not seem really useful here, but this can potentially break
scripts with things like:

  cat "$foo" >> "$bar"

where "$foo" may be the same file as "$bar" only if it is empty.

BTW, when x isn't empty, I wonder whether an error is correct if
POSIXLY_CORRECT is set. The result will typically depend on the
implementation and possibly be non-deterministic, but POSIX doesn't
seem to allow an error (except FS errors, such as disk full).

-- 
Vincent Lefèvre <vincent <at> vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Message #8 received at 18449 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Vincent Lefevre <vincent <at> vinc17.net>, 18449 <at> debbugs.gnu.org
Subject: Re: bug#18449: "cat x >> x" error even when x is empty
Date: Thu, 11 Sep 2014 14:20:06 +0100

On 09/11/2014 02:00 PM, Vincent Lefevre wrote:
> With coreutils 8.23 under Debian/unstable:
> 
> ypig% : > x
> ypig% cat x >> x
> cat: x: input file is output file
> ypig% POSIXLY_CORRECT=1 cat x >> x
> cat: x: input file is output file
> 
> while there's no reason to return an error in this case: the file
> should just remain empty. Using the same file for input and output
> isn't disallowed by POSIX, AFAIK.
> 
> This may not seem really useful here, but this can potentially break
> scripts with things like:
> 
>   cat "$foo" >> "$bar"
> 
> where "$foo" may be the same file as "$bar" only if it is empty.

I'm struggling to see that use case TBH.

> BTW, when x isn't empty, I wonder whether an error is correct if
> POSIXLY_CORRECT is set. The result will typically depend on the
> implementation and possibly be non-deterministic, but POSIX doesn't
> seem to allow an error (except FS errors, such as disk full).

This is just a protection against non deterministic usage.
I.E. if there is data in the file then it'll probably run
until the file system is exhausted.  This is what happens
on FreeBSD BTW, though I see that Solaris has the same
protections as GNU.

I wouldn't be on for removing this protection which
has existed from the initial GNU implementation.

cheers,
Pádraig.

Message #11 received at 18449 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Pádraig Brady <P <at> draigBrady.com>,
 Vincent Lefevre <vincent <at> vinc17.net>, 18449 <at> debbugs.gnu.org
Subject: Re: bug#18449: "cat x >> x" error even when x is empty
Date: Thu, 11 Sep 2014 08:56:10 -0600

[Message part 1 (text/plain, inline)]

On 09/11/2014 07:20 AM, Pádraig Brady wrote:

>> while there's no reason to return an error in this case: the file
>> should just remain empty.

Testing for an empty file is enough of an additional special case over
the existing check for same files that I don't think it is worth it.


>> BTW, when x isn't empty, I wonder whether an error is correct if
>> POSIXLY_CORRECT is set. The result will typically depend on the
>> implementation and possibly be non-deterministic, but POSIX doesn't
>> seem to allow an error (except FS errors, such as disk full).

I've raised this as an issue for the Austin Group to rule on:
http://austingroupbugs.net/view.php?id=876

> 
> This is just a protection against non deterministic usage.
> I.E. if there is data in the file then it'll probably run
> until the file system is exhausted.  This is what happens
> on FreeBSD BTW, though I see that Solaris has the same
> protections as GNU.

I'm hoping that POSIX standardizes the existing practice of forbidding
known disk-filling operations.

> 
> I wouldn't be on for removing this protection which
> has existed from the initial GNU implementation.

My thoughts, too.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #14 received at 18449 <at> debbugs.gnu.org (full text, mbox):

From: Vincent Lefevre <vincent <at> vinc17.net>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 18449 <at> debbugs.gnu.org
Subject: Re: bug#18449: "cat x >> x" error even when x is empty
Date: Thu, 11 Sep 2014 17:46:56 +0200

On 2014-09-11 14:20:06 +0100, Pádraig Brady wrote:
> On 09/11/2014 02:00 PM, Vincent Lefevre wrote:
> > This may not seem really useful here, but this can potentially break
> > scripts with things like:
> > 
> >   cat "$foo" >> "$bar"
> > 
> > where "$foo" may be the same file as "$bar" only if it is empty.
> 
> I'm struggling to see that use case TBH.

Seriously, I don't know whether this could occur in practice, but if
it does, I wouldn't be surprised. If this may be regarded as an error,
it is important that this is specified; the issue raised by Eric for
the Austin Group may makes this clear. Let's just hope that users
won't forget this particular case.

As a comparison, one may think that in C, a shift like 0 << n, where
n >= int width, would never occur in practice, but it did in one of
my codes (I made sure that there was no integer overflow but forgot
the constraint on the shift count, which adds a restriction only when
the first argument is 0).

-- 
Vincent Lefèvre <vincent <at> vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Message #17 received at 18449 <at> debbugs.gnu.org (full text, mbox):

From: Leslie S Satenstein <lsatenstein <at> yahoo.com>
To: Pádraig Brady <P <at> draigBrady.com>,
 Vincent Lefevre <vincent <at> vinc17.net>,
 "18449 <at> debbugs.gnu.org" <18449 <at> debbugs.gnu.org>
Subject: Re: bug#18449: "cat x >> x" error even when x is empty
Date: Thu, 11 Sep 2014 06:31:46 -0700

[Message part 1 (text/plain, inline)]

With my Fedora version,   there is an error message   (input file is output file)

 cat x >x
cat: x: input file is output file


 
Regards 

 Leslie





>________________________________
> From: Pádraig Brady <P <at> draigBrady.com>
>To: Vincent Lefevre <vincent <at> vinc17.net>; 18449 <at> debbugs.gnu.org 
>Sent: Thursday, September 11, 2014 9:20 AM
>Subject: bug#18449: "cat x >> x" error even when x is empty
> 
>
>On 09/11/2014 02:00 PM, Vincent Lefevre wrote:
>> With coreutils 8.23 under Debian/unstable:
>> 
>> ypig% : > x
>> ypig% cat x >> x
>> cat: x: input file is output file
>> ypig% POSIXLY_CORRECT=1 cat x >> x
>> cat: x: input file is output file
>> 
>> while there's no reason to return an error in this case: the file
>> should just remain empty. Using the same file for input and output
>> isn't disallowed by POSIX, AFAIK.
>> 
>> This may not seem really useful here, but this can potentially break
>> scripts with things like:
>> 
>>   cat "$foo" >> "$bar"
>> 
>> where "$foo" may be the same file as "$bar" only if it is empty.
>
>I'm struggling to see that use case TBH.
>
>
>> BTW, when x isn't empty, I wonder whether an error is correct if
>> POSIXLY_CORRECT is set. The result will typically depend on the
>> implementation and possibly be non-deterministic, but POSIX doesn't
>> seem to allow an error (except FS errors, such as disk full).
>
>This is just a protection against non deterministic usage.
>I.E. if there is data in the file then it'll probably run
>until the file system is exhausted.  This is what happens
>on FreeBSD BTW, though I see that Solaris has the same
>protections as GNU.
>
>I wouldn't be on for removing this protection which
>has existed from the initial GNU implementation.
>
>cheers,
>Pádraig.
>
>
>
>
>
>
>

[Message part 2 (text/html, inline)]

Message #22 received at 18449-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eric Blake <eblake <at> redhat.com>, Pádraig Brady
 <P <at> draigBrady.com>, Vincent Lefevre <vincent <at> vinc17.net>, 
 18449-done <at> debbugs.gnu.org
Subject: Re: bug#18449: "cat x >> x" error even when x is empty
Date: Thu, 11 Sep 2014 08:52:17 -0700

[Message part 1 (text/plain, inline)]

Eric Blake wrote:
> Testing for an empty file is enough of an additional special case over
> the existing check for same files that I don't think it is worth it.

It's a really cheap check, as a system call is needed only when the 
input and output files are the same, and even then it's only an lseek 
with SEEK_CUR so it's all in-memory.  Plus, the POSIX 'cat' spec gives 
an example of using 'cat' to copy an empty regular file to itself, and 
(not unreasonably) says that should work.

As far as POSIX conformance goes, I don't think we need to worry about 
POSIXLY_CORRECT here.  We'll get POSIX fixed instead.  Eric has already 
started the ball rolling on that (thanks).  Though the POSIX fix should 
involve updating its example -- and quite possibly POSIX should continue 
to allow catting an empty file to itself.

I installed the attached patch and am marking this as done.  The 'cat' 
source code is simpler now, so that's a win anyway....

[0001-cat-allow-copying-empty-files-to-themselves.patch (text/plain, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.