GNU bug report logs - #24730
rmdir/mkdir error(s) and/or not working "reciprocally" w/each other

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 24730 in the body.
You can then email your comments to 24730 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "L. A. Walsh" <coreutils <at> tlinx.org>
To: bug-coreutils <at> gnu.org
Subject: rmdir/mkdir error(s) and/or not working "reciprocally" w/each other
Date: Tue, 18 Oct 2016 12:49:44 -0700

It doesn't seem rmdir and mkdir are behaving "reciprocally"...

If I type

mkdir -p ./a/b/c   # no error
rmdir -p ./a/b/c   # get error msg, but a,b,c removed.

1) thinking either rmdir shouldn't generate an error or mkdir should

mkdir -p a/../b      # no error
rmdir -p a/../b      # error, but a & b removed

2) similar comment to above -- leading to:

for rmdir, if "-p" is used, then as similar to "mkdir -p":
  (no error if existing, make parent directories as needed)

rmdir -p should be
  "no error if dir not empty, but directories are followed
and deleted as possible".

======> seems to be best wording & solution:

"mkdir -p", it seems should really be restated to:

   follow given path and make directories as possible"

then "rmdir -p" could be

   "follow given path and delete directories if empty"


Does that look reasonable?

linda w.


  

Message #12 received at 24730-done <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: "L. A. Walsh" <coreutils <at> tlinx.org>, 24730-done <at> debbugs.gnu.org
Subject: Re: bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally"
 w/each other
Date: Tue, 18 Oct 2016 15:33:43 -0500

[Message part 1 (text/plain, inline)]

tag 24730 notabug
thanks

On 10/18/2016 02:49 PM, L. A. Walsh wrote:
> 
> It doesn't seem rmdir and mkdir are behaving "reciprocally"...
> 
> If I type
> 
> mkdir -p ./a/b/c   # no error

. already exists, so mkdir silently does nothing,
./a needs to be created,
./a/b needs to be created,
./a/b/c needs to be created

> rmdir -p ./a/b/c   # get error msg, but a,b,c removed.

./a/b/c needs to be removed,
./a/b needs to be removed,
./a needs to be removed,
. needs to be removed, but you can't do that, hence the error

The apparent asymmetry is due to the POSIX rules on how '.' is treated;
but we can't change the behavior of either of these commands, because
they are both following the POSIX rules.

If you want symmetry, omit the leading './', as in:

$ mkdir -p a/b/c
$ rmdir -p a/b/c

> 
> 1) thinking either rmdir shouldn't generate an error or mkdir should
> 
> mkdir -p a/../b      # no error

a needs to be created,
a/.. already exists, so it silently does nothing,
a/../b needs to be created

> rmdir -p a/../b      # error, but a & b removed

a/../b needs to be removed,
a/.. needs to be removed, but you can't do that,
at this point, POSIX is fuzzy whether to attempt to remove 'a', or to
give up since 'a/..' was already an error; but obviously coreutils
removes 'a'

> 
> 2) similar comment to above -- leading to:
> 
> for rmdir, if "-p" is used, then as similar to "mkdir -p":
>   (no error if existing, make parent directories as needed)
> 
> rmdir -p should be
>   "no error if dir not empty, but directories are followed
> and deleted as possible".

Sadly, while that might be a nicer definition for 'rmdir -p', it doesn't
match the POSIX requirements nor the historical behavior, so we can't
really change it now.

So I'm marking this as not a bug, as there is nothing to change.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #15 received at 24730-done <at> debbugs.gnu.org (full text, mbox):

From: Reuti <reuti <at> staff.uni-marburg.de>
To: Eric Blake <eblake <at> redhat.com>
Cc: 24730-done <at> debbugs.gnu.org, "L. A. Walsh" <coreutils <at> tlinx.org>
Subject: Re: bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally"
 w/each other
Date: Tue, 18 Oct 2016 22:50:57 +0200

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Am 18.10.2016 um 22:33 schrieb Eric Blake:

> tag 24730 notabug
> thanks
> 
> On 10/18/2016 02:49 PM, L. A. Walsh wrote:
>> 
>> It doesn't seem rmdir and mkdir are behaving "reciprocally"...
>> 
>> If I type
>> 
>> mkdir -p ./a/b/c   # no error
> 
> . already exists, so mkdir silently does nothing,
> ./a needs to be created,
> ./a/b needs to be created,
> ./a/b/c needs to be created
> 
>> rmdir -p ./a/b/c   # get error msg, but a,b,c removed.
> 
> ./a/b/c needs to be removed,
> ./a/b needs to be removed,
> ./a needs to be removed,
> . needs to be removed, but you can't do that, hence the error
> 
> The apparent asymmetry is due to the POSIX rules on how '.' is treated;
> but we can't change the behavior of either of these commands, because
> they are both following the POSIX rules.
> 
> If you want symmetry, omit the leading './', as in:
> 
> $ mkdir -p a/b/c
> $ rmdir -p a/b/c
> 
>> 
>> 1) thinking either rmdir shouldn't generate an error or mkdir should
>> 
>> mkdir -p a/../b      # no error
> 
> a needs to be created,
> a/.. already exists, so it silently does nothing,
> a/../b needs to be created
> 
>> rmdir -p a/../b      # error, but a & b removed
> 
> a/../b needs to be removed,
> a/.. needs to be removed, but you can't do that,
> at this point, POSIX is fuzzy whether to attempt to remove 'a', or to
> give up since 'a/..' was already an error; but obviously coreutils
> removes 'a'

What version of core-utils shows this behavior. In the latest one it's not removed AFAICS.

- -- Reuti


> 
>> 
>> 2) similar comment to above -- leading to:
>> 
>> for rmdir, if "-p" is used, then as similar to "mkdir -p":
>>  (no error if existing, make parent directories as needed)
>> 
>> rmdir -p should be
>>  "no error if dir not empty, but directories are followed
>> and deleted as possible".
> 
> Sadly, while that might be a nicer definition for 'rmdir -p', it doesn't
> match the POSIX requirements nor the historical behavior, so we can't
> really change it now.
> 
> So I'm marking this as not a bug, as there is nothing to change.
> 
> -- 
> Eric Blake   eblake redhat com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
> 

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - https://gpgtools.org

iEYEARECAAYFAlgGizIACgkQo/GbGkBRnRrB9ACgpbqqSW0ZF27H5nSKF97zrDth
kLMAnRDKMmGxZ9p2SAqp/g+8AIkinqtU
=ykPn
-----END PGP SIGNATURE-----

Message #18 received at 24730-done <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Reuti <reuti <at> staff.uni-marburg.de>
Cc: 24730-done <at> debbugs.gnu.org, "L. A. Walsh" <coreutils <at> tlinx.org>
Subject: Re: bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally"
 w/each other
Date: Tue, 18 Oct 2016 15:54:30 -0500

[Message part 1 (text/plain, inline)]

On 10/18/2016 03:50 PM, Reuti wrote:

>>>
>>> 1) thinking either rmdir shouldn't generate an error or mkdir should
>>>
>>> mkdir -p a/../b      # no error
> 
>> a needs to be created,
>> a/.. already exists, so it silently does nothing,
>> a/../b needs to be created
> 
>>> rmdir -p a/../b      # error, but a & b removed
> 
>> a/../b needs to be removed,
>> a/.. needs to be removed, but you can't do that,
>> at this point, POSIX is fuzzy whether to attempt to remove 'a', or to
>> give up since 'a/..' was already an error; but obviously coreutils
>> removes 'a'
> 
> What version of core-utils shows this behavior. In the latest one it's not removed AFAICS.

Hmm, you're right. I was going off the (incorrect) comment in the text
above, rather than actually testing it; so it looks like coreutils gives
up on the first error, rather than trying to remove a.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #21 received at 24730 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: "L. A. Walsh" <coreutils <at> tlinx.org>, 24730 <at> debbugs.gnu.org
Subject: Re: bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally"
 w/each other
Date: Tue, 18 Oct 2016 16:57:45 -0400

Hello,

Before deciding on the wording, it's worth nothing that the errors and reasons for the errors are different between mkdir and rmdir, and between the two cases.

On 10/18/2016 03:49 PM, L. A. Walsh wrote:

> mkdir -p ./a/b/c   # no error
> rmdir -p ./a/b/c   # get error msg, but a,b,c removed.

The error in this case (at least on Linux) is "Invalid Argument",
because 'rmdir .' is invalid and rejected by the kernel (EINVAL)
while 'mkdir .' returns EEXISTS - and '-p' specifically instruct it to silently ignore EEXIST.

Demonstrated another way:

'mkdir' without -p:

    $ strace -e mkdir mkdir .
    mkdir(".", 0777)                        = -1 EEXIST (File exists)
    mkdir: cannot create directory ‘.’: File exists
    +++ exited with 1 +++

'mkdir' with -p:

    $ strace -e mkdir mkdir -p .
    mkdir(".", 0777)                        = -1 EEXIST (File exists)
    +++ exited with 0 +++

but 'rmdir' gives an unrecoverable error because of the kernel,
not because of coreutils' code:

    $ strace -e rmdir rmdir .
    rmdir(".")                              = -1 EINVAL (Invalid argument)
    rmdir: failed to remove '.': Invalid argument

This is also mandated by posix:
   http://pubs.opengroup.org/onlinepubs/9699919799/functions/rmdir.html
   "If the path argument refers to a path whose final component is either dot or dot-dot, rmdir() shall fail."


However, if there is no dot component in the path, they do behave similarly (or reciprocally, as you've said).

> mkdir -p a/../b      # no error
> rmdir -p a/../b      # error, but a & b removed

At least on my system (Linux kernel 3.13), 'a' is not removed in the above example,
and the error is due to non-empty directory:

    $ strace -e mkdir mkdir -p a/../b
    mkdir("a", 0777)                        = 0
    mkdir("b", 0777)                        = 0
    +++ exited with 0 +++

    $ strace -e rmdir rmdir -p a/../b
    rmdir("a/../b")                         = 0
    rmdir("a/..")                           = -1 ENOTEMPTY (Directory not empty)
    rmdir: failed to remove directory 'a/..': Directory not empty
    +++ exited with 1 +++


So two cases these are slightly different (EINVAL vs ENOTEMPTY).

Note that coreutils' mkdir contains an optimization not to try and make '..' as it is
guaranteed to exist (implemented in gnulib's mkancesdirs.c).

However, by definition, when 'rmdir' traverses the directories on the given path,
the directory 'a/..' is not empty (it contains 'a') - so this must fail.


If you want to 'rmdir' to silently ignore non-empty directories,
there's a gnu extension option of 'rmdir --ignore-fail-on-non-empty':

    $ strace -e rmdir rmdir --ignore-fail-on-non-empty -p a/../b
    rmdir("a/../b")                         = 0
    rmdir("a/..")                           = -1 ENOTEMPTY (Directory not empty)
    +++ exited with 0 +++

But note that this causes 'rmdir' to stop upon first failure, and 'a' is still not removed.

> ======> seems to be best wording & solution:
>
> "mkdir -p", it seems should really be restated to:
>
>    follow given path and make directories as possible"
>
> then "rmdir -p" could be
>
>    "follow given path and delete directories if empty"

This does not accurately reflect how 'rmdir' is currently implemented.
A more accurate description is "follow given path and delete directories, until the first encountered failure".


    $ mkdir -p a/../b/../c/../d

    $ strace -e rmdir rmdir -p a/../b/../c/../d
    rmdir("a/../b/../c/../d")               = 0
    rmdir("a/../b/../c/..")                 = -1 ENOTEMPTY (Directory not empty)
    rmdir: failed to remove directory 'a/../b/../c/..': Directory not empty
    +++ exited with 1 +++

    $ ls
    a  b  c

If you want a behavior where 'rmdir' will continue beyond the first failure
and try to delete all directories in the given path (e.g. a/b/c in the example above),
then it sounds like a new feature request, and a non-standard behavior which will be a gnu extension (and also quite confusing behavior, IMHO).



regards,
 - assaf

Message #24 received at 24730 <at> debbugs.gnu.org (full text, mbox):

From: Linda Walsh <coreutils <at> tlinx.org>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: 24730 <at> debbugs.gnu.org
Subject: Re: bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally"
 w/each other
Date: Tue, 18 Oct 2016 14:35:55 -0700

Assaf Gordon wrote:
> Hello,
> 
> Before deciding on the wording, it's worth nothing that the errors and 
> reasons for the errors are different between mkdir and rmdir, and 
> between the two cases.
> 
> On 10/18/2016 03:49 PM, L. A. Walsh wrote:
> 
>> mkdir -p ./a/b/c   # no error
>> rmdir -p ./a/b/c   # get error msg, but a,b,c removed.
> 
> The error in this case (at least on Linux) is "Invalid Argument",
> because 'rmdir .' is invalid and rejected by the kernel (EINVAL)
> while 'mkdir .' returns EEXISTS - and '-p' specifically instruct it to 
> silently ignore EEXIST.
----
	I see... so in ".a/b/c", a,b,c are removed, but the error
comes in "."?



>     $ strace -e rmdir rmdir .
>     rmdir(".")                              = -1 EINVAL (Invalid argument)
>     rmdir: failed to remove '.': Invalid argument
> 
> This is also mandated by posix:
>    http://pubs.opengroup.org/onlinepubs/9699919799/functions/rmdir.html
>    "If the path argument refers to a path whose final component is 
> either dot or dot-dot, rmdir() shall fail."
---
	Ok, but is "-p" a posix switch in mkdir or rmdir?
If not, wouldn't the behavior be undefined?



>> mkdir -p a/../b      # no error
>> rmdir -p a/../b      # error, but a & b removed
> 
> At least on my system (Linux kernel 3.13), 'a' is not removed in the 
> above example,
> and the error is due to non-empty directory:
---
	Right..., my bad, only checked 'b'. (*duh*)


> Note that coreutils' mkdir contains an optimization not to try and make 
> '..' as it is
> guaranteed to exist (implemented in gnulib's mkancesdirs.c).
---
	Right, same when rmdir traverses, when it hits "..", that
must exist (even in '/').  


> However, by definition, when 'rmdir' traverses the directories on the 
> given path,
> the directory 'a/..' is not empty (it contains 'a') - so this must fail.
----
	with "-p"?  Am talking about the case where you 
create a dir with "mkdir -p "$dirname"
and later, rmdir -p "$dirname" (where dirname is passed as a parameter, but
is not user-input).


	
> 
> 
> If you want to 'rmdir' to silently ignore non-empty directories,
> there's a gnu extension option of 'rmdir --ignore-fail-on-non-empty':
> 
>     $ strace -e rmdir rmdir --ignore-fail-on-non-empty -p a/../b
>     rmdir("a/../b")                         = 0
>     rmdir("a/..")                           = -1 ENOTEMPTY (Directory 
> not empty)
>     +++ exited with 0 +++
> 
> But note that this causes 'rmdir' to stop upon first failure, and 'a' is 
> still not removed.
> 
>> ======> seems to be best wording & solution:
>>
>> "mkdir -p", it seems should really be restated to:
>>
>>    follow given path and make directories as possible"
>>
>> then "rmdir -p" could be
>>
>>    "follow given path and delete directories if empty"
> 
> This does not accurately reflect how 'rmdir' is currently implemented.
> A more accurate description is "follow given path and delete 
> directories, until the first encountered failure".
----
	Right, but am trying to get rmdir to be a useful
"opposite" to a "mkdir" -- at least w/r/t "-p"...


> If you want a behavior where 'rmdir' will continue beyond the first failure
> and try to delete all directories in the given path (e.g. a/b/c in the 
> example above),
> then it sounds like a new feature request, and a non-standard behavior 
> which will be a gnu extension (and also quite confusing behavior, IMHO).
---
	Yeah -- new feature, but confusing?

I think the fact that you can mkdir -p XXX and later  canNOT
rmdir -p XXX, is quite confusing behavior...

But is -p's behavior in mkdir and rmdir mandated by POSIX?

Message #27 received at 24730 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Linda Walsh <coreutils <at> tlinx.org>
Cc: 24730 <at> debbugs.gnu.org
Subject: Re: bug#24730: rmdir/mkdir error(s) and/or not working "reciprocally"
 w/each other
Date: Tue, 18 Oct 2016 20:14:27 -0400

Hello,

> On Oct 18, 2016, at 17:35, Linda Walsh <coreutils <at> tlinx.org> wrote:
> 
> Assaf Gordon wrote:
>>> rmdir -p ./a/b/c   # get error msg, but a,b,c removed.
> 	I see... so in ".a/b/c", a,b,c are removed, but the error
> comes in "."?

Yes.

> 	Ok, but is "-p" a posix switch in mkdir or rmdir?

Yes, it is defined in posix for both mkdir(1) and rmdir(1):

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rmdir.html
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/mkdir.html


>> However, by definition, when 'rmdir' traverses the directories on the given path,
>> the directory 'a/..' is not empty (it contains 'a') - so this must fail.
> ----
> 	with "-p"?  Am talking about the case where you create a dir with "mkdir -p "$dirname"
> and later, rmdir -p "$dirname" (where dirname is passed as a parameter, but
> is not user-input).

Yes. By posix definition, For "rmdir -p DIR" where DIR contains multiple path components,
rmdir should behave as if run with "rmdir -p $(dirname DIR)".

And for example:
      $ dirname "a/../b"
      a/..

And trying to call rmdir(2) on "a/.." will fail (definitely fails  on Linux kernel, though I suspect on other systems as well).


> 	Right, but am trying to get rmdir to be a useful
> "opposite" to a "mkdir" -- at least w/r/t "-p"...

"useful" is somewhat subjective.
For the more common case of directories without ".." or "." - this works quite well,
and that is very likely the intended goal:

     mkdir -p a/b/c
     rmdir -p a/b/c


As the current behavior is mandated by POSIX, If you want to suggest a different behavior,
the best place to suggest this is to the posix group ( https://www.opengroup.org/austin/ ).
If the standard for 'rmdir(1)' is modified or adjusted, GNU coreutils will surely follow suit.

> But is -p's behavior in mkdir and rmdir mandated by POSIX?

Yes, see above links.

regards,
 - assaf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.