GNU bug report logs - #24714
delete-directory race condition

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 24714 in the body.
You can then email your comments to 24714 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #3 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: submit <at> debbugs.gnu.org
Subject: delete-directory race condition
Date: Sun, 16 Oct 2016 22:20:45 -0400

Package: emacs
Version: 25.1

On current Debian testing, many tests in package-test.el fail for me with:

    Test package-test-update-listing condition:
    (file-error "Removing old name" "No such file or directory"
    "/tmp/pkg-test-user-dir-27293kBj/gnupg/S.gpg-agent.rstrd")

I believe this is due to a race condition in delete-directory.
Emacs seems to be lacking an equivalent of "rm -rf".

(delete-directory "/tmp/foo")

will fail with "No such file or directory" if a file in /tmp/foo
happen to be deleted by some other process in between the time that
delete-directory calls directory-files and the time it calls delete-file.

Message #6 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Mon, 17 Oct 2016 09:19:34 +0300

> From: Glenn Morris <rgm <at> gnu.org>
> Date: Sun, 16 Oct 2016 22:20:45 -0400
> 
> I believe this is due to a race condition in delete-directory.
> Emacs seems to be lacking an equivalent of "rm -rf".
> 
> (delete-directory "/tmp/foo")
> 
> will fail with "No such file or directory" if a file in /tmp/foo
> happen to be deleted by some other process in between the time that
> delete-directory calls directory-files and the time it calls delete-file.

IMO, delete-directory should simply catch ENOENT errors and ignore
them when it deletes files and subdirectories under the "recursive"
option.  Other errors should signal an error as they do now.

Message #9 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Mon, 17 Oct 2016 11:52:31 -0400

Eli Zaretskii wrote:

>> will fail with "No such file or directory" if a file in /tmp/foo
>> happen to be deleted by some other process in between the time that
>> delete-directory calls directory-files and the time it calls delete-file.
>
> IMO, delete-directory should simply catch ENOENT errors and ignore
> them when it deletes files and subdirectories under the "recursive"
> option.

I don't think that's enough, since a file could equally well be
_created_ by some other process after delete-directory calls directory-files.
Frankly I don't see how Emacs's delete-directory can work reliably as
currently implemented.

Message #12 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Mon, 17 Oct 2016 19:11:05 +0300

> From: Glenn Morris <rgm <at> gnu.org>
> Cc: 24714 <at> debbugs.gnu.org
> Date: Mon, 17 Oct 2016 11:52:31 -0400
> 
> Eli Zaretskii wrote:
> 
> >> will fail with "No such file or directory" if a file in /tmp/foo
> >> happen to be deleted by some other process in between the time that
> >> delete-directory calls directory-files and the time it calls delete-file.
> >
> > IMO, delete-directory should simply catch ENOENT errors and ignore
> > them when it deletes files and subdirectories under the "recursive"
> > option.
> 
> I don't think that's enough, since a file could equally well be
> _created_ by some other process after delete-directory calls directory-files.

It will solve your use case, with files under /tmp, won't it?  A
partial solution is better than no solution at all, IMO.

> Frankly I don't see how Emacs's delete-directory can work reliably as
> currently implemented.

Patches to reimplement it are also welcome.

Message #15 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Mon, 17 Oct 2016 19:00:55 +0200

On Okt 17 2016, Glenn Morris <rgm <at> gnu.org> wrote:

> I don't think that's enough, since a file could equally well be
> _created_ by some other process after delete-directory calls directory-files.

That is ok.  The operation cannot successfully be completed in this case.

> Frankly I don't see how Emacs's delete-directory can work reliably as
> currently implemented.

A property it shares with any other operation recursing on directories,
in Emacs or elsewhere.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #18 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Tue, 18 Oct 2016 09:44:56 -0700

[Message part 1 (text/plain, inline)]

Although I didn't reproduce the bug, I changed delete-directory 
according to Eli's suggestion by installing the attached patch into 
master. Glenn, can you please try it in your environment?

[0001-delete-directory-no-longer-errors-when-racing.patch (application/x-patch, attachment)]

Message #21 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Tue, 18 Oct 2016 12:50:32 -0400

Andreas Schwab wrote:

> That is ok.  The operation cannot successfully be completed in this case.
>
>> Frankly I don't see how Emacs's delete-directory can work reliably as
>> currently implemented.
>
> A property it shares with any other operation recursing on directories,
> in Emacs or elsewhere.

That's interesting. I'd assumed there was a "Right Way" to do it, and
that it would be whatever coreutil's rm did. If not then we are back to:

> delete-directory should simply catch ENOENT errors and ignore them
> when it deletes files and subdirectories under the "recursive" option.
> Other errors should signal an error as they do now.

This doesn't seem possible without changes at the C level.
Should there be a standard Lisp error for ENOENT?
Currently report_file_errno only does EEXIST -> file_already_exists.

Or should delete-file get a FORCE argument akin to "rm -f"?

Message #24 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Tue, 18 Oct 2016 12:55:55 -0400

I haven't tested it, but the files--force implementation looks a little
unaesthetic to me. Did you consider adding a standard condition error in
report_file_errno for ENOENT? There would then be symmetry with the
file-already-exists error for EEXIST.

Also, IMO this is purely a bug fix, and should not be mentioned in NEWS,
the lispref, or the doc.

But thanks for the fix!

Message #27 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Tue, 18 Oct 2016 12:59:26 -0400

Glenn Morris wrote:

> I haven't tested it, but the files--force implementation looks a little
> unaesthetic to me.

PS will it work in non-English locales?

Message #30 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Wed, 19 Oct 2016 10:23:26 +0300

> From: Glenn Morris <rgm <at> gnu.org>
> Date: Tue, 18 Oct 2016 12:59:26 -0400
> Cc: 24714 <at> debbugs.gnu.org
> 
> Glenn Morris wrote:
> 
> > I haven't tested it, but the files--force implementation looks a little
> > unaesthetic to me.
> 
> PS will it work in non-English locales?

Paul, what about this Glenn's question?  The error strings are
localized, I think.

Message #33 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>, Glenn Morris <rgm <at> gnu.org>
Cc: 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Wed, 19 Oct 2016 23:50:36 -0700

Eli Zaretskii wrote:
> Paul, what about this Glenn's question?  The error strings are
> localized, I think.

Yes, his comments are spot on. I have written a fix but want to test it more 
before installing.

Message #36 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Eli Zaretskii <eliz <at> gnu.org>, Glenn Morris <rgm <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Fri, 21 Oct 2016 04:53:40 -0700

On 10/19/2016 12:23 AM, Eli Zaretskii wrote:
>> From: Glenn Morris <rgm <at> gnu.org>
>> Date: Tue, 18 Oct 2016 12:59:26 -0400
>> Cc: 24714 <at> debbugs.gnu.org
>>
>> Glenn Morris wrote:
>>
>>> I haven't tested it, but the files--force implementation looks a little
>>> unaesthetic to me.
>>
>> PS will it work in non-English locales?
>
> Paul, what about this Glenn's question?  The error strings are
> localized, I think.

What about attaching the numeric errno value to the error message as a 
text property in report_file_errno?

Message #39 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: Glenn Morris <rgm <at> gnu.org>, Eli Zaretskii <eliz <at> gnu.org>,
 eggert <at> cs.ucla.edu, 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Fri, 21 Oct 2016 14:09:27 +0200

On Okt 21 2016, Daniel Colascione <dancol <at> dancol.org> wrote:

> What about attaching the numeric errno value to the error message as a
> text property in report_file_errno?

errno values aren't portable either.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #42 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: Glenn Morris <rgm <at> gnu.org>, Eli Zaretskii <eliz <at> gnu.org>,
 eggert <at> cs.ucla.edu, 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Fri, 21 Oct 2016 05:10:34 -0700

On 10/21/2016 05:09 AM, Andreas Schwab wrote:
> On Okt 21 2016, Daniel Colascione <dancol <at> dancol.org> wrote:
>
>> What about attaching the numeric errno value to the error message as a
>> text property in report_file_errno?
>
> errno values aren't portable either.

True, but you can provide the value of EEXIST and others as constants 
accessible to lisp.

Message #45 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: rgm <at> gnu.org, eggert <at> cs.ucla.edu, 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Fri, 21 Oct 2016 15:55:09 +0300

> Cc: eggert <at> cs.ucla.edu, 24714 <at> debbugs.gnu.org
> From: Daniel Colascione <dancol <at> dancol.org>
> Date: Fri, 21 Oct 2016 04:53:40 -0700
> 
> > Paul, what about this Glenn's question?  The error strings are
> > localized, I think.
> 
> What about attaching the numeric errno value to the error message as a 
> text property in report_file_errno?

IMO, Glenn suggested a cleaner solution: to have a symbol for ENOENT.

Message #48 received at 24714 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 24714 <at> debbugs.gnu.org
Subject: Re: bug#24714: delete-directory race condition
Date: Fri, 21 Oct 2016 13:07:47 -0700

[Message part 1 (text/plain, inline)]

On 10/18/2016 09:55 AM, Glenn Morris wrote:
> Did you consider adding a standard condition error in
> report_file_errno for ENOENT? There would then be symmetry with the
> file-already-exists error for EEXIST.

Again, thanks for pointing that out. I installed the attached patch, 
which does that. This also should fix the locale problem you mentioned 
later.

[0001-New-error-file-missing.patch (application/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.