GNU bug report logs - #18491
rm -r fails to delete entire hierarchy when path goes in and out of it

Previous Next

To reply to this bug, email your comments to 18491 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Gian Ntzik <gian.ntzik08 <at> imperial.ac.uk>
To: bug-coreutils <at> gnu.org
Subject: rm -r fails to delete entire hierarchy when path goes in and out of it
Date: Wed, 17 Sep 2014 06:38:56 +0100

Hello,

It seems that using rm -r with a path that goes into a (non-empty)
directory intended for removal (and back up e.g. using dot-dots) fails
to remove the directory. The directory is rendered empty, but itself not
removed.

For example,

$ mkdir -p /tmp/a/b/c
$ mkdir -p /tmp/a/e
$ rm -r /tmp/a/b/../../a
rm: cannot remove ‘/tmp/a/b/../../a’: No such file or directory

Then,

$ ls /tmp/a

yeilds empty contents, and

$ ls -d /tmp/a
/tmp/a

yields that the directory intended for removal still exists.

I have tested this in Ubuntu 14.04 where coreutils is at version 8.21. I
have also tested it with version 8.23 built from source also on Ubuntu
14.04.

I do not believe this is the intended behavior, but a bug.

Obviously, once the contents of /tmp/a have been removed, the path
/tmp/a/b/../../a is no longer resolvable. However, the same applies for
any sub-directory of /tmp/a. For example, once /tmp/a/b has been
removed, /tmp/a/b/../../a/e is also no longer resolvable. It seems that
rm -r takes care to avoid such issues during recursion but fails to do
so at the very end.

Thank you,

Gian Ntzik

Message #8 received at 18491 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Gian Ntzik <gian.ntzik08 <at> imperial.ac.uk>, 18491 <at> debbugs.gnu.org
Subject: Re: bug#18491: rm -r fails to delete entire hierarchy when path goes
 in and out of it
Date: Wed, 17 Sep 2014 13:08:15 -0700

On 09/16/2014 10:38 PM, Gian Ntzik wrote:
> It seems that
> rm -r takes care to avoid such issues during recursion but fails to do
> so at the very end.

A clever example but it's not clear it's a bug, or how to "fix" it.  For 
what it's worth, Solaris 10 /bin/rm behaves the same way.

Message #11 received at 18491 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: Gian Ntzik <gian.ntzik08 <at> imperial.ac.uk>
Cc: 18491 <at> debbugs.gnu.org
Subject: Re: bug#18491: rm -r fails to delete entire hierarchy when path goes
 in and out of it
Date: Thu, 18 Sep 2014 15:28:01 -0600

Gian Ntzik wrote:
> It seems that using rm -r with a path that goes into a (non-empty)
> directory intended for removal (and back up e.g. using dot-dots) fails
> to remove the directory. The directory is rendered empty, but itself not
> removed.
> 
> For example,
> 
> $ mkdir -p /tmp/a/b/c
> $ mkdir -p /tmp/a/e
> $ rm -r /tmp/a/b/../../a
> rm: cannot remove ‘/tmp/a/b/../../a’: No such file or directory

I don't think this can reasonably be called a bug.  A depth first
removal is required.  a/b must be removed before a is removed.  But
the relative reference requires a/b to exist in order to obtain
b/.. in order to obtain b/../.. in order to obtain b/../../a but a/b
gets removed first since a depth first removal is required.

Trying to do anything to work around this seems wrong to me since it
will require keeping track of the state before and simulating to
create the desired state afterward and then applying a derived state
change to the file system.  That is much too complex for this simple
operation.

> Obviously, once the contents of /tmp/a have been removed, the path
> /tmp/a/b/../../a is no longer resolvable. However, the same applies for
> any sub-directory of /tmp/a. For example, once /tmp/a/b has been
> removed, /tmp/a/b/../../a/e is also no longer resolvable. It seems that
> rm -r takes care to avoid such issues during recursion but fails to do
> so at the very end.

I don't think it matters that a/e and a/b do actually get removed
revealing that in the implementation some ordering is happening.
Regardless of that I don't think it is reasonable to require that rm
remember the old path through a relative path that gets removed along
the way.

Bob

Message #14 received at 18491 <at> debbugs.gnu.org (full text, mbox):

From: "Linda A. Walsh" <coreutils <at> tlinx.org>
To: Bob Proulx <bob <at> proulx.com>
Cc: Gian Ntzik <gian.ntzik08 <at> imperial.ac.uk>, 18491 <at> debbugs.gnu.org
Subject: Re: bug#18491: rm -r fails to delete entire hierarchy when path goes
 in	and out of it
Date: Thu, 18 Sep 2014 16:40:41 -0700

Bob Proulx wrote:
> Gian Ntzik wrote:
>> It seems that using rm -r with a path that goes into a (non-empty)
>> directory intended for removal (and back up e.g. using dot-dots) fails
>> to remove the directory. The directory is rendered empty, but itself not
>> removed.
>>
>> For example,
>>
>> $ mkdir -p /tmp/a/b/c
>> $ mkdir -p /tmp/a/e
>> $ rm -r /tmp/a/b/../../a
>> rm: cannot remove ‘/tmp/a/b/../../a’: No such file or directory
>
> I don't think this can reasonably be called a bug.  A depth first
> removal is required. 
----
   I would tend to agree, but if you try to remove /tmp/a/.
a "depth first removal" won't be tried -- so it is obviously not required.

>
> Trying to do anything to work around this seems wrong to me since it
> will require keeping track of the state before and simulating to
> create the desired state afterward and then applying a derived state
> change to the file system.  That is much too complex for this simple
> operation.
----
   One would think the same for rm -fr "foo/.", but the
straight-forward application of the depth-first removal was
removed from "rm" for special cases.  One would think
that the underlying tree might be easily addressed:

function rmr {
 local rd=$(cd "$1"; /bin/pwd)
 echo rm -r "$rd"
}

same dir struct as above:
>  tree /tmp/a
/tmp/a
├── b
│   └── c
└── e

>  rmr /tmp/a
rm -r /tmp/a

---
If rm should delete  the directory and its contents as it is documented
to do except where POSIX prohibits it, (like  "rm -fr dir/."),
then except for POSIX instructions to the contrary, it seems it
should make at least as trivial an effort as the above.  :-|

Message #17 received at 18491 <at> debbugs.gnu.org (full text, mbox):

From: Gian Ntzik <gian.ntzik08 <at> imperial.ac.uk>
To: Bob Proulx <bob <at> proulx.com>
Cc: "18491 <at> debbugs.gnu.org" <18491 <at> debbugs.gnu.org>
Subject: Re: bug#18491: rm -r fails to delete entire hierarchy when path goes
 in and out of it
Date: Fri, 19 Sep 2014 04:30:29 +0100

Bob Proulx <bob <at> proulx.com> writes:

> Gian Ntzik wrote:
>> It seems that using rm -r with a path that goes into a (non-empty)
>> directory intended for removal (and back up e.g. using dot-dots) fails
>> to remove the directory. The directory is rendered empty, but itself not
>> removed.
>> 
>> For example,
>> 
>> $ mkdir -p /tmp/a/b/c
>> $ mkdir -p /tmp/a/e
>> $ rm -r /tmp/a/b/../../a
>> rm: cannot remove ‘/tmp/a/b/../../a’: No such file or directory

Bear in mind that the issue is not restricted to dot-dots within the
pathname argument. The problem is with any pathname that traverses
the same directory we want to remove. The same thing can happen with
symlinks.

For example,
$ mkdir -p /tmp/a/b/c
$ mkdir -p /tmp/a/e
$ ln -s /tmp /tmp/a/e/x
$ ls /tmp/a
b  e
$ ls /tmp/a/e/x/a
b  e
$ rm -r /tmp/a/e/x/a
rm: cannot remove ‘/tmp/a/e/x/a’: No such file or directory
$ ls /tmp/a
$

> I don't think this can reasonably be called a bug.  

Assume this is not a bug as you claim. 

Then this means that given an acceptable path argument (not ending in dot,
dot-dot and not resolving to /) that successfully resolves to a
directory, rm -r may or may not delete the directory (assuming no
interference from a concurrently running process). So, either your rm -r
behaviour is non-deterministic even without concurrency, or rm -r is not
intended to be used by all acceptable paths, but by some subset
satisfying a property P.

A definition of this property P could be:
Path x satisfies P iff 
x is acceptable by rm, and
if x resolves to a directory then, there does not exist a prefix y of x
such that y resolves to the same entry as x.

For paths not satisfying P, rm -r does something else, or is
undefined.

There are two major problems with is:
1). This usage restriction is not mentioned in this implementation's
documentation. If this is not a bug then it really should, because
clients of rm wanting to really remove a directory must make sure their
paths meet your requirements.
2). The POSIX rm specification does not restrict the paths that can be
used to remove directories (other than those that rm does not accept in
the first place). In fact this most probably deviates from the standard.

>
> Trying to do anything to work around this seems wrong to me since it
> will require keeping track of the state before and simulating to
> create the desired state afterward and then applying a derived state
> change to the file system.  That is much too complex for this simple
> operation.
>

Yes, this is way too complex. However, for every path there is a
canonical path that does not use dot-dots or reference symbolic
links. Canonical paths would never run into such problems because they
do not traverse what they resolve to. After initial checks to see if the
argument is acceptable (not ending in dot-dot, dot or not being /), one
could take the canonical path of the argument, e.g. through a simple
realpath(), and start the recursive removal with that. This seems much
simpler.

Message #20 received at 18491 <at> debbugs.gnu.org (full text, mbox):

From: Gian Ntzik <gian.ntzik08 <at> imperial.ac.uk>
To: "Linda A. Walsh" <coreutils <at> tlinx.org>
Cc: "18491 <at> debbugs.gnu.org" <18491 <at> debbugs.gnu.org>,
 Bob Proulx <bob <at> proulx.com>
Subject: Re: bug#18491: rm -r fails to delete entire hierarchy when path goes
 in and out of it
Date: Fri, 19 Sep 2014 04:47:38 +0100

"Linda A. Walsh" <coreutils <at> tlinx.org> writes:

> Bob Proulx wrote:
>> Gian Ntzik wrote:
>>> It seems that using rm -r with a path that goes into a (non-empty)
>>> directory intended for removal (and back up e.g. using dot-dots) fails
>>> to remove the directory. The directory is rendered empty, but itself not
>>> removed.
>>>
>>> For example,
>>>
>>> $ mkdir -p /tmp/a/b/c
>>> $ mkdir -p /tmp/a/e
>>> $ rm -r /tmp/a/b/../../a
>>> rm: cannot remove ‘/tmp/a/b/../../a’: No such file or directory
>>
>> Trying to do anything to work around this seems wrong to me since it
>> will require keeping track of the state before and simulating to
>> create the desired state afterward and then applying a derived state
>> change to the file system.  That is much too complex for this simple
>> operation.
> ----
>     One would think the same for rm -fr "foo/.", but the
> straight-forward application of the depth-first removal was
> removed from "rm" for special cases.  One would think
> that the underlying tree might be easily addressed:
>
> function rmr {
>   local rd=$(cd "$1"; /bin/pwd)
>   echo rm -r "$rd"
> }
>

Yes, this would work because getcwd() gives a canonical path (no dot-dot
or symlinks). Issues like the one reported do not arise with canonical paths.
Another simple way to address the issue could be:

if [ -d $1 ]
then
    rm -r "$(/bin/readlink -e $1)"
else
    rm -r "$1"
fi

which does not change the cwd.

(assume that sufficient on the argument ending in dot, dot-dot or being
/ are done beforehand). 

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.