GNU bug report logs - #25817
rm: add option to avoid early exit on "."

Previous Next

To reply to this bug, email your comments to 25817 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: bug-coreutils <at> gnu.org
Subject: Why were Gnu coding standards violated in favor of posix for 'rm
 -fr .'?: request for reversion of behavior
Date: Mon, 20 Feb 2017 11:41:36 -0800

In reading Gnu's Coding Standards (
https://www.gnu.org/prep/standards/standards.html#Non_002dGNU-Standards),
Under non-Gnu-Standards -- it is specifically talking about POSIX
compatibility when it says:

   In particular, don’t reject a new feature, or remove an old one,
   merely because a standard says it is “forbidden” or “deprecated”.

So... why should 'rm' not be able to start it's deletion
from the inside of a directory? (@ "." )?

FWIW, because of the above change, rm is no longer consistent in its
counting.  With "one-file-system", it means "1fs/starting path",
not 1fs /rm command, whereas with "-I", it creates a global
limit of '3' deletions before asking -- not 3 deletions/starting path.

From the above, changing 'rm' to disallow '.' in a path shouldn't have
been done.

Can this be fixed?  :-)

Thanks!
-linda

Message #8 received at 25817 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: L A Walsh <coreutils <at> tlinx.org>, 25817 <at> debbugs.gnu.org
Subject: Re: bug#25817: Why were Gnu coding standards violated in favor of
 posix for 'rm -fr .'?: request for reversion of behavior
Date: Mon, 20 Feb 2017 14:33:19 -0600

[Message part 1 (text/plain, inline)]

tag 25817 needinfo
thanks

On 02/20/2017 01:41 PM, L A Walsh wrote:

> So... why should 'rm' not be able to start it's deletion
> from the inside of a directory? (@ "." )?

Please give more details as to what you think is broken.  Instead of
describing the problem in vague prose, please show a shell transcript
that creates a sample directory layout and cd's into the place that you
want, then attempts the removal that currently fails, as well as
explaining what you hoped to have happen instead of an error message.  I
will demonstrate below.  As written, your report is too vague to
definitively state whether coreutils behavior has even changed over
time, let alone whether such a change is, as you claim, a violation of
GNU Coding Standards.

> 
> FWIW, because of the above change, rm is no longer consistent in its
> counting.  With "one-file-system", it means "1fs/starting path",
> not 1fs /rm command, whereas with "-I", it creates a global
> limit of '3' deletions before asking -- not 3 deletions/starting path.

I can't make enough sense of this paragraph without actual shell
commands being demonstrated to see what you are complaining about.

If you are complaining that:

$ mkdir tmp
$ cd tmp
$ touch file
$ rm -rf .
rm: refusing to remove '.' or '..' directory: skipping '.'
$ ls
file

fails not only to delete the current working directory, but refuses to
even attempt to remove 'file' within that directory, please remember
that the behavior I demonstrated is compliant with this wording in POSIX
2008 (including amendments by TC2 in 2016):
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html

"If either of the files dot or dot-dot are specified as the basename
portion of an operand (that is, the final pathname component) or if an
operand resolves to the root directory, rm shall write a diagnostic
message to standard error and do nothing more with such operands."

At the time of this email, I have not researched whether this wording
has changed over time from older versions of POSIX.

Are you arguing that older versions of 'rm' (whether GNU or non-GNU)
and/or older versions of POSIX had different behavior/requirements?  If
so, can you please quote chapter-and-verse of those other standards, or
call out version numbers (or even commit ids) where it did what you
want, or any other thing you can do to make your point stronger that an
intentional (or possibly unintentional) change in behavior occurred, and
that there is indeed an alternative behavior worth supporting (even if
such alternative is not the default, it could still be triggered by a
new command-line option).

Or are you arguing that contents within the directory should be removed,
even though the directory itself cannot be?  Again, more details in your
complaint would go a long way to making a decision whether there is an
actual bug, or just a misunderstanding of current behavior.  It helps if
you can focus on the facts at hand ("what happens, what did you want to
happen") rather than making it a political attack ("you broke things and
aren't consistent" or in general any rant against POSIX).

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Eric Blake <eblake <at> redhat.com>
Cc: bug-coreutils <at> gnu.org
Subject: Re: bug#25817: Why were Gnu coding standards violated in favor of
 posix for 'rm -fr .'?: request for reversion of behavior
Date: Mon, 20 Feb 2017 13:10:24 -0800

Eric Blake wrote:
> tag 25817 needinfo
> thanks
> 
> On 02/20/2017 01:41 PM, L A Walsh wrote:
> 
>> So... why should 'rm' not be able to start it's deletion
>> from the inside of a directory? (@ "." )?
> 
> Please give more details as to what you think is broken.  Instead of
> describing the problem in vague prose, please show a shell transcript
> that creates a sample directory layout and cd's into the place that you
> want, then attempts the removal that currently fails, as well as
> explaining what you hoped to have happen instead of an error message.
-----
#!/bin/bash
cd /tmp || exit 1
mkdir tstdir || exit 1
mkdir tstdir/usr
mkdir tstdir/var
mount --bind /var /tmp/tstdir/var
mount --bind /usr /tmp/tstdir/usr
mkdir tstdir/{1,2,3}
cd /tmp/tstdir || exit
rm -fr . --one-file-system
---------
Output:
rm: refusing to remove ‘.’ or ‘..’ directory: skipping ‘.’
----
Instead, it should have removed dirs 1,2,3 and not 
not 'var' or 'usr'.


----

rm originally did a depth first traversal and deletion
of files under it.

a few to several years ago a posix change was added to 
change the depth-first traversal and to
inspect local paths before doing the depth-first traversal
of directories like '.' -- and if it found '.' to disallow
any paths below '.' (prior to the depth-first traversal that
used to happen).

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Eric Blake <eblake <at> redhat.com>
Cc: bug-coreutils <at> gnu.org
Subject: Re: bug#25817: Why were Gnu coding standards violated in favor of
 posix	for 'rm -fr .'?: request for reversion of behavior
Date: Tue, 21 Feb 2017 00:41:26 -0800

> Or are you arguing that contents within the directory should be removed,
> even though the directory itself cannot be? 
---
	That's the way a recursive descent algorithm
works: it processes the contents, before the parents.

When it gets to the parent, it couldn't remove it, but
due to the "-f", silently failed.

	Do you really need me to find the older version
of 'rm' in your source tree?

	you know as well as I do, that the verbiage
about disallowing all subpaths under '.' wasn't in earlier
posix versions (say 2001 or before).

	It's not a matter of finding where it says it
is "allowed", rm was always recursive descent and posix
didn't disallow it. 

	Gnu says not to remove features or
behaviors due to changing posix standards...
So the old behavior should be reinstated as
default, and those that want the new behavior can
use POSIX_CORRECTLY (or whatever it is)...

Message #17 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: L A Walsh <coreutils <at> tlinx.org>
Cc: bug-coreutils <at> gnu.org
Subject: Re: bug#25817: Why were Gnu coding standards violated in favor of
 posix for 'rm -fr .'?: request for reversion of behavior
Date: Tue, 21 Feb 2017 07:42:41 -0600

[Message part 1 (text/plain, inline)]

On 02/21/2017 02:41 AM, L A Walsh wrote:
> 
>     Do you really need me to find the older version
> of 'rm' in your source tree?

It wouldn't hurt to point out which commit id changed behavior, if you
indeed want to call that commit a regression.  Being able to read the
commit itself, as well as mailing list traffic at the time, will help
current readers get a more-informed picture of whether such a change was
intentional or accidental.

> 
>     you know as well as I do, that the verbiage
> about disallowing all subpaths under '.' wasn't in earlier
> posix versions (say 2001 or before).

So I spent 20 more minutes on this, and researched a bit further: here's
what POSIX 2001 says:

http://pubs.opengroup.org/onlinepubs/009695399/utilities/rm.html

"If either of the files dot or dot-dot are specified as the basename
portion of an operand (that is, the final pathname component), rm shall
write a diagnostic message to standard error and do nothing more with
such operands."

I then went back to SUSv2, published in 1997:

http://pubs.opengroup.org/onlinepubs/007908799/xcu/rm.html

"If either of the files dot or dot-dot are specified as the basename
portion of an operand (that is, the final pathname component), rm will
write a diagnostic message to standard error and do nothing more with
such operands."

You ARE correct that it was POSIX 2008 that introduced the wording "or
if an operand resolves to the root directory" to cover an early exit on
an attempt to remove '/', but the discussion here is about an early exit
on an attempt to remove '.', which, contrary to your claim, appears to
always have been in POSIX as far as I can tell (and even if it has not
always been in POSIX, the fact that I can quote a 20-year-old document
that describes current behavior means that any change to a default now
would be breaking 20 years of what script writers have come to rely on).

I'm all ears if you can point to an even-older standards document worded
differently, but at this point, I don't think you're going to find one.

And I still maintain that a patch that adds a new option to avoid the
early exit on '.' may be acceptable, but that it would have to be a new
option and not the default behavior.  But I'm not bothered enough by the
current situation to be the one to write such a patch.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: L A Walsh <coreutils <at> tlinx.org>
Cc: bug-coreutils <at> gnu.org
Subject: Re: bug#25817: Why were Gnu coding standards violated in favor of
 posix for 'rm -fr .'?: request for reversion of behavior
Date: Tue, 21 Feb 2017 08:08:16 -0600

[Message part 1 (text/plain, inline)]

On 02/21/2017 07:42 AM, Eric Blake wrote:
> On 02/21/2017 02:41 AM, L A Walsh wrote:
>>
>>     Do you really need me to find the older version
>> of 'rm' in your source tree?
> 
> It wouldn't hurt to point out which commit id changed behavior, if you
> indeed want to call that commit a regression.  Being able to read the
> commit itself, as well as mailing list traffic at the time, will help
> current readers get a more-informed picture of whether such a change was
> intentional or accidental.

A bit more research shows that:

1) we don't seem to have any tests that cover 'rm -r .' behavior (so
even if we change the code now, we'd first need to add a test; and
without a test, it's entirely possible that a commit in the past altered
behavior without being noticed because of lack of testsuite coverage)

$ git grep 'rm .*r.* \.' tests/rm
$

So a testsuite addition is welcome, and if nothing else, this bug should
not be closed until we have testsuite coverage, whether or not we change
rm itself.

2) the current behavior is due to this code in src/remove.c:rm_fts():

        {
          /* POSIX says:
             If the basename of a command line argument is "." or "..",
             diagnose it and do nothing more with that argument.  */
          if (dot_or_dotdot (last_component (ent->fts_accpath)))

and a git blame easily shows that portion of the function has been in
place since at least commit 4f73ecaf "rm: rewrite to use fts" (Sep 2009,
release 8.0).  Prior to that date, there is no rm_fts(), so the
archaeology gets trickier, but even as far back as ff59218d "New file.
Contains guts of old rm.c." (Jan 1998, release 4.5.1), there is:

static enum RM_status
rm (struct File_spec *fs, int user_specified_name)
{
  mode_t filetype_mode;

  if (user_specified_name)
    {
      char *base = base_name (fs->filename);

      if (DOT_OR_DOTDOT (base))
        {
          error (0, 0, _("cannot remove `.' or `..'"));
          return RM_ERROR;
        }
    }

I've already spent too much time on this, and did not go digging through
earlier history of rm.c or reading any changelogs prior to that point.
Please, if you have evidence of a version where GNU rm behaved
differently, you will do yourself a favor by pointing it out to us,
rather than making a vague assertion that we have changed behavior with
nothing to back up your claim.  A little bit of research on your side
will go a long ways to making your report credible.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #23 received at 25817 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eric Blake <eblake <at> redhat.com>, L A Walsh <coreutils <at> tlinx.org>
Cc: 25817 <at> debbugs.gnu.org
Subject: Re: bug#25817: Why were Gnu coding standards violated in favor of
 posix for 'rm -fr .'?: request for reversion of behavior
Date: Tue, 21 Feb 2017 08:55:47 -0800

On 02/21/2017 05:42 AM, Eric Blake wrote:
> "If either of the files dot or dot-dot are specified as the basename
> portion of an operand (that is, the final pathname component), rm will
> write a diagnostic message to standard error and do nothing more with
> such operands."

The same wording is in the first version of POSIX that standardized 
'rm', namely IEEE Std 1003.2-1992 section 4.53.2 lines 8384-6. So we are 
looking at 25 years' worth of standardization here.

Going back even further in time, 7th Edition Unix 'rm' was confused in 
this area. Although 'rm -r ..' had the POSIX-specified behavior, 'rm -r 
.' removed all subfiles and then quietly succeeded without removing '.', 
and there were other complications. Presumably this mess is what the 
early-1990 standardizers were trying to avoid.

At any rate I agree that the requested behavior should be enabled only 
via a new option. Regardless of what one thinks 'rm' should do if we 
could redesign it from scratch, there's too much dead weight of history 
here.

Message #26 received at submit <at> debbugs.gnu.org (full text, mbox):

From: L A Walsh <coreutils <at> tlinx.org>
To: Eric Blake <eblake <at> redhat.com>
Cc: bug-coreutils <at> gnu.org
Subject: Re: bug#25817: Why were Gnu coding standards violated in favor of
 posix for 'rm -fr .'?: request for reversion of behavior
Date: Tue, 21 Feb 2017 10:55:37 -0800

Eric Blake wrote:
>  the discussion here is about an early exit
> on an attempt to remove '.', which, contrary to your claim, appears to
> always have been in POSIX as far as I can tell (and even if it has not
> always been in POSIX, the fact that I can quote a 20-year-old document
> that describes current behavior means that any change to a default now
> would be breaking 20 years of what script writers have come to rely on).
---
	My initial post under this subject said that it was a matter
of not following gnu policy -- of not removing a feature just because
posix said to do so.

	I was cited posix as the justification for this change.  
Apparently that was incorrect, as you point out.  My point was
that the behavior/feature was allowed in coreutils and removed with
a justification that posix disallowed it.

	I doubt I can find a commit, but I may be able to find
a version of coreutils where this is the case, but likely, not
quickly.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.