GNU bug report logs - #36490
26.1; directory-files-recursively breaks when it encounters a directory named "~"

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 36490 in the body.
You can then email your comments to 36490 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Erik Hahn <erik_hahn <at> gmx.de>
To: bug-gnu-emacs <at> gnu.org
Subject: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Wed, 03 Jul 2019 20:07:26 +0200

Call (directory-files-recursively DIR ".*") where DIR contains a
subdirectory named "~" but is not the parent of your home directory. The
function will recurse into your home directory. This is probably because
(expand-file-name "~" ...) returns the home directory.


In GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, X toolkit)
 of 2018-11-03 built on p5
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description:	Gentoo Base System release 2.6

Recent messages:
Configured using:
 'configure --prefix=/usr --build=x86_64-pc-linux-gnu
 --host=x86_64-pc-linux-gnu --mandir=/usr/share/man
 --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc
 --localstatedir=/var/lib --disable-silent-rules
 --docdir=/usr/share/doc/emacs-26.1-r3
 --htmldir=/usr/share/doc/emacs-26.1-r3/html --libdir=/usr/lib64
 --program-suffix=-emacs-26 --infodir=/usr/share/info/emacs-26
 --localstatedir=/var
 --enable-locallisppath=/etc/emacs:/usr/share/emacs/site-lisp
 --without-compress-install --without-hesiod --without-pop
 --with-file-notification=inotify --enable-acl --with-dbus
 --without-modules --without-gameuser --with-gpm --without-kerberos
 --without-kerberos5 --without-lcms2 --with-xml2 --without-mailutils
 --without-selinux --with-gnutls --with-libsystemd --with-threads
 --without-wide-int --with-zlib --with-sound=alsa --with-x --without-ns
 --without-gconf --without-gsettings --without-toolkit-scroll-bars
 --with-gif --with-jpeg --with-png --with-rsvg --with-tiff --with-xpm
 --without-imagemagick --without-xft --without-cairo --without-libotf
 --without-m17n-flt --with-x-toolkit=lucid --with-xaw3d 'CFLAGS=-O2
 -pipe -march=native' CPPFLAGS= 'LDFLAGS=-Wl,-O1 -Wl,--as-needed''

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS NOTIFY ACL GNUTLS
LIBXML2 ZLIB LUCID X11 THREADS LIBSYSTEMD

Important settings:
  value of $LC_MONETARY: en_US.UTF-8
  value of $LC_NUMERIC: en_US.UTF-8
  value of $LC_TIME: en_GB.UTF-8
  value of $LANG: en_GB.utf8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Message #8 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Erik Hahn <erik_hahn <at> gmx.de>
Cc: 36490 <at> debbugs.gnu.org
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Mon, 08 Jul 2019 22:59:37 +0200

Erik Hahn <erik_hahn <at> gmx.de> writes:

> Call (directory-files-recursively DIR ".*") where DIR contains a
> subdirectory named "~" but is not the parent of your home directory. The
> function will recurse into your home directory. This is probably because
> (expand-file-name "~" ...) returns the home directory.

Oops:

(expand-file-name "~" "/tmp/")
=> "/home/larsi"

Yup; that function should be rewritten to have no instances of
`expand-file-name', which is too DWIM for a function like this.  I'll do
a rewrite...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #13 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Erik Hahn <erik_hahn <at> gmx.de>
Cc: 36490 <at> debbugs.gnu.org
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Mon, 08 Jul 2019 23:08:46 +0200

Actually, the doc string of expand-file-name is either wrong, or the
implementation is.

It says:

---
An initial ‘~/’ expands to your home directory.
An initial ‘~USER/’ expands to USER’s home directory.
---

Assuming the "An initial" refers to the first parameter, then

(expand-file-name "~/" "/tmp/")
=> "/home/larsi/"

works as advertised, but

(expand-file-name "~" "/tmp/")
=> "/home/larsi"

is a different thing: "~" is a perfectly valid file name, so having this
function map that to something else is just...  wrong.

(expand-file-name "~larsi" "/tmp/")
=> "/home/larsi"

is the same: Also wrong and undocumented.

The doc string continues with further confusion:

---
See also the function ‘substitute-in-file-name’.
---

See it for...  what?  For further expansions this function is going to
do?  Fortunately not:

(expand-file-name "$HOME" "/tmp/")
=> "/tmp/$HOME"

So that's probably just meant as "that's also a function that does file
name stuff, but it has nothing to do with this ~ thing we just
discussed"?

So what to do here?  I think the current, undocumented

(expand-file-name "~" "/tmp/")
=> "/home/larsi"

must surely be an error, and that should be fixed instead of the
callers?  Opinions?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #16 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Erik Hahn <erik_hahn <at> gmx.de>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 36490 <at> debbugs.gnu.org
Subject: Re: bug#36490: 26.1; directory-files-recursively breaks when it
 encounters a directory named "~"
Date: Mon, 8 Jul 2019 23:30:38 +0200

On 08/07/19 23:08, Lars Ingebrigtsen wrote:
> The doc string continues with further confusion:
>
> ---
> See also the function ‘substitute-in-file-name’.
> ---
>
> See it for...  what?  For further expansions this function is going to
> do?

It is related in that it also chokes on files named "~":

(substitute-in-file-name "/tmp/~") => "~"

Its doc string is also wrong. It says

> If ‘/~’ appears, all of FILENAME through that ‘/’ is discarded.

but

(substitute-in-file-name "/tmp/~foo") => "/tmp/~foo"

Message #19 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Erik Hahn <erik_hahn <at> gmx.de>
Cc: 36490 <at> debbugs.gnu.org
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Mon, 08 Jul 2019 23:56:40 +0200

Erik Hahn <erik_hahn <at> gmx.de> writes:

> It is related in that it also chokes on files named "~":
>
> (substitute-in-file-name "/tmp/~") => "~"
>
> Its doc string is also wrong. It says
>
>> If ‘/~’ appears, all of FILENAME through that ‘/’ is discarded.
>
> but
>
> (substitute-in-file-name "/tmp/~foo") => "/tmp/~foo"

Yup, it's only if the user exists:

(substitute-in-file-name "/tmp/~larsi")
=> "~larsi"

Same as with expand-file-name:

(expand-file-name "~larsi" "/tmp/")
=> "/home/larsi"

(expand-file-name "~foo" "/tmp/")
=> "/tmp/~foo"

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #22 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 18:35:46 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Date: Mon, 08 Jul 2019 23:08:46 +0200
> Cc: 36490 <at> debbugs.gnu.org
> 
> Actually, the doc string of expand-file-name is either wrong, or the
> implementation is.
> 
> It says:
> 
> ---
> An initial ‘~/’ expands to your home directory.
> An initial ‘~USER/’ expands to USER’s home directory.
> ---
> 
> Assuming the "An initial" refers to the first parameter, then
> 
> (expand-file-name "~/" "/tmp/")
> => "/home/larsi/"
> 
> works as advertised, but
> 
> (expand-file-name "~" "/tmp/")
> => "/home/larsi"
> 
> is a different thing: "~" is a perfectly valid file name, so having this
> function map that to something else is just...  wrong.

If you want "~" to be interpreted literally, you need to protect it
with "/:".

> (expand-file-name "~larsi" "/tmp/")
> => "/home/larsi"
> 
> is the same: Also wrong and undocumented.

Why would we want to document that?

> The doc string continues with further confusion:
> 
> ---
> See also the function ‘substitute-in-file-name’.
> ---
> 
> See it for...  what?

For expanding environment variables, and for the special effect of
"//" etc.

> So what to do here?  I think the current, undocumented
> 
> (expand-file-name "~" "/tmp/")
> => "/home/larsi"
> 
> must surely be an error, and that should be fixed instead of the
> callers?  Opinions?

It is not an error.

Message #25 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Erik Hahn <erik_hahn <at> gmx.de>
Cc: 36490 <at> debbugs.gnu.org, larsi <at> gnus.org
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 18:39:15 +0300

> From: Erik Hahn <erik_hahn <at> gmx.de>
> Date: Mon, 8 Jul 2019 23:30:38 +0200
> Cc: 36490 <at> debbugs.gnu.org
> 
>  > If ‘/~’ appears, all of FILENAME through that ‘/’ is discarded.
> 
> but
> 
> (substitute-in-file-name "/tmp/~foo") => "/tmp/~foo"

Do you have a user named "foo" on that system?  If not, try a real
user name instead of "foo".

Message #28 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 17:50:49 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> An initial ‘~/’ expands to your home directory.
>> An initial ‘~USER/’ expands to USER’s home directory.
>> ---
>> 
>> Assuming the "An initial" refers to the first parameter, then

[...]

> If you want "~" to be interpreted literally, you need to protect it
> with "/:".

Sorry; I don't quite follow you here.  The doc string says that "~/" is
interpreted specially.  There's no "/" in "~".  :-)

That 

(expand-file-name "~/" "/tmp/")
=> "/home/larsi/"

is fine, because "~/" isn't a valid file name.  But "~" is, and that's
the problem.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #31 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 19:12:04 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: erik_hahn <at> gmx.de,  36490 <at> debbugs.gnu.org
> Date: Tue, 09 Jul 2019 17:50:49 +0200
> 
> > If you want "~" to be interpreted literally, you need to protect it
> > with "/:".
> 
> Sorry; I don't quite follow you here.  The doc string says that "~/" is
> interpreted specially.  There's no "/" in "~".  :-)

But it does NOT say that "~" will NOT be interpreted specially.

> That 
> 
> (expand-file-name "~/" "/tmp/")
> => "/home/larsi/"
> 
> is fine, because "~/" isn't a valid file name.  But "~" is, and that's
> the problem.

No, I think the problem is that the caller should have protected the
literal "~", as I said.

Message #34 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 36490 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 18:17:51 +0200

On Jul 09 2019, Lars Ingebrigtsen <larsi <at> gnus.org> wrote:

> is fine, because "~/" isn't a valid file name.

"~/" is a valid file name, that happens to be a directory.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Message #37 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 18:27:28 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> > If you want "~" to be interpreted literally, you need to protect it
>> > with "/:".
>> 
>> Sorry; I don't quite follow you here.  The doc string says that "~/" is
>> interpreted specially.  There's no "/" in "~".  :-)
>
> But it does NOT say that "~" will NOT be interpreted specially.

I think it's a good idea to specify what strings will be interpreted
specially -- otherwise it's a bit difficult to use that function.

>> is fine, because "~/" isn't a valid file name.  But "~" is, and that's
>> the problem.
>
> No, I think the problem is that the caller should have protected the
> literal "~", as I said.

Any function that iterates over a list of directory files (as returned
by directory-files) would then have to quote whatever it passes to
expand-file-name -- or avoid that function completely.  That seems like
an odd design.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #40 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 19:44:31 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: erik_hahn <at> gmx.de,  36490 <at> debbugs.gnu.org
> Date: Tue, 09 Jul 2019 18:27:28 +0200
> 
> > No, I think the problem is that the caller should have protected the
> > literal "~", as I said.
> 
> Any function that iterates over a list of directory files (as returned
> by directory-files) would then have to quote whatever it passes to
> expand-file-name -- or avoid that function completely.  That seems like
> an odd design.

What happens if a function that iterates over a directory finds a
subdirectory whose name is literally "~"?

Message #43 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 18:50:44 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> Any function that iterates over a list of directory files (as returned
>> by directory-files) would then have to quote whatever it passes to
>> expand-file-name -- or avoid that function completely.  That seems like
>> an odd design.
>
> What happens if a function that iterates over a directory finds a
> subdirectory whose name is literally "~"?

I think I must be missing something, because I don't think anything
special should happen?

(find-file "/tmp/~/foo")

works fine, for instance.  And

(expand-file-name "foo" "/tmp/~")
=> "/tmp/~/foo"

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #46 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: larsi <at> gnus.org
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 19:52:52 +0300

> Date: Tue, 09 Jul 2019 19:44:31 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
> 
> What happens if a function that iterates over a directory finds a
> subdirectory whose name is literally "~"?

Sorry, that's not what I meant to say.  I meant to say how would a
Lisp program know whether (expand-file-name "~/") means the home
directory or a directory whose name is literally "~"?

Btw, stuff like (expand-file-name "foo/~/") already does what you
want, so the problem is only with the leading '~', and can be avoided
if we avoid that situation.  IOW, why should this example:

  (expand-file-name "~" "/tmp/")
     => "/home/larsi"

determine how directory-files-recursively behaves?

Message #49 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 19:00:13 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> Sorry, that's not what I meant to say.  I meant to say how would a
> Lisp program know whether (expand-file-name "~/") means the home
> directory or a directory whose name is literally "~"?

Well, we have documented that in expand-file-name "~/" means the home
directory, and I have no problems with that.

"~/" isn't something you'll ever get from functions like
directory-files, so it's not something you'd feed to expand-file-name in
these situations...

> Btw, stuff like (expand-file-name "foo/~/") already does what you
> want, so the problem is only with the leading '~', and can be avoided
> if we avoid that situation.  IOW, why should this example:
>
>   (expand-file-name "~" "/tmp/")
>      => "/home/larsi"
>
> determine how directory-files-recursively behaves?

expand-file-name's use case is to (basically) concatenate a directory
name and a file name, but it's used instead of concat because nobody
wants to care about whether the directory name has a trailing slash or
not.

(concat "/tmp/" "foo")
=> "/tmp/foo" ; Good

(concat "/tmp" "foo")
=> "/tmpfoo" ; Bad.

(expand-file-name "foo" "/tmp")
=> "/tmp/foo" ; Yay

That's basically the use case for expand-file-name, and using it has
avoided a lot of basic concatenation problems over the years (because
Emacs allows sloppy handling of directory file names in most
situations).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #52 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 20:23:02 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: 36490 <at> debbugs.gnu.org,  erik_hahn <at> gmx.de
> Date: Tue, 09 Jul 2019 19:00:13 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > Sorry, that's not what I meant to say.  I meant to say how would a
> > Lisp program know whether (expand-file-name "~/") means the home
> > directory or a directory whose name is literally "~"?
> 
> Well, we have documented that in expand-file-name "~/" means the home
> directory, and I have no problems with that.

Documenting a problem doesn't necessarily solve it.  E.g., it is also
documented that you must quote file names with special characters, but
you still raised the objection that the "~" use case makes that "odd".

I'm saying that the "~/" use case is "odd" as well, and for the same
reasons.

> "~/" isn't something you'll ever get from functions like
> directory-files

That's sheer luck, because:

  (file-name-as-directory "~")
    => "~/"

So just running "~" through an innocent API gives you a "magic"
directory name (if you consider "~" not "magic" by itself).  How is
this different from the "odd" use case where one must quote "~" to
avoid its interpretation as the home directory?  Who can guarantee
that some day directory-files-recursively will not want to do
something like the above?  If it does, we will be right back at the
same problem.

I say we should fix this problem in a way that isn't fragile, and
doesn't crucially depend on what the current code does or avoids
doing.

> > Btw, stuff like (expand-file-name "foo/~/") already does what you
> > want, so the problem is only with the leading '~', and can be avoided
> > if we avoid that situation.  IOW, why should this example:
> >
> >   (expand-file-name "~" "/tmp/")
> >      => "/home/larsi"
> >
> > determine how directory-files-recursively behaves?
> 
> expand-file-name's use case is to (basically) concatenate a directory
> name and a file name, but it's used instead of concat because nobody
> wants to care about whether the directory name has a trailing slash or
> not.

Ah, but when the file name begins with a "~", the "concatenation" does
more than what meets the eye.

> That's basically the use case for expand-file-name, and using it has
> avoided a lot of basic concatenation problems over the years (because
> Emacs allows sloppy handling of directory file names in most
> situations).

I think this is a simplification.  It ignores the fact that
expand-file-name interprets ~/, it ignores the fact that it does
arbitrary stuff for "remote" file names, it ignores the fact that on
Windows it prepends the drive letter if there isn't one already, etc.
IOW, expand-file-name is concatenation-like, but it has a few tricks
up its sleeve, and in this case the trick works against us.  We need
to disable that trick to support files and directories whose names
begin with a literal "~".  I see no way around that.

Message #55 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36490 <at> debbugs.gnu.org, Lars Ingebrigtsen <larsi <at> gnus.org>, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 20:16:56 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> expand-file-name's use case is to (basically) concatenate a directory
>> name and a file name, but it's used instead of concat because nobody
>> wants to care about whether the directory name has a trailing slash or
>> not.
>
> Ah, but when the file name begins with a "~", the "concatenation" does
> more than what meets the eye.
>
>> That's basically the use case for expand-file-name, and using it has
>> avoided a lot of basic concatenation problems over the years (because
>> Emacs allows sloppy handling of directory file names in most
>> situations).
>
> I think this is a simplification.  It ignores the fact that
> expand-file-name interprets ~/, it ignores the fact that it does
> arbitrary stuff for "remote" file names, it ignores the fact that on
> Windows it prepends the drive letter if there isn't one already, etc.
> IOW, expand-file-name is concatenation-like, but it has a few tricks
> up its sleeve, and in this case the trick works against us.  We need
> to disable that trick to support files and directories whose names
> begin with a literal "~".  I see no way around that.

For the records, I second Eli. See also the discussion in bug#16984. And
yes, I believe it makes sense to quote file names (suppress special
meaning of "~") in the loop of directory-files-recursively.

Best regards, Michael.

Message #58 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36490 <at> debbugs.gnu.org, Lars Ingebrigtsen <larsi <at> gnus.org>, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 19:58:18 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Lars Ingebrigtsen <larsi <at> gnus.org>
>> Cc: erik_hahn <at> gmx.de,  36490 <at> debbugs.gnu.org
>> Date: Tue, 09 Jul 2019 17:50:49 +0200
>> 
>> > If you want "~" to be interpreted literally, you need to protect it
>> > with "/:".
>> 
>> Sorry; I don't quite follow you here.  The doc string says that "~/" is
>> interpreted specially.  There's no "/" in "~".  :-)
>
> But it does NOT say that "~" will NOT be interpreted specially.

Indeed it explicitly says that "~" will be interpreted specially:

  Second arg DEFAULT-DIRECTORY is directory to start with if NAME is
  relative (does not start with slash or tilde); both the directory name
  and a directory's file name are accepted.

So AIUI (expand-file-name "~") should be equivalent to
(directory-file-name (expand-file-name "~/")).

-- 
Basil

Message #61 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 36490 <at> debbugs.gnu.org, larsi <at> gnus.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Tue, 09 Jul 2019 22:19:43 +0300

> From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
> Cc: Lars Ingebrigtsen <larsi <at> gnus.org>,  36490 <at> debbugs.gnu.org,  erik_hahn <at> gmx.de
> Date: Tue, 09 Jul 2019 19:58:18 +0100
> 
> So AIUI (expand-file-name "~") should be equivalent to
> (directory-file-name (expand-file-name "~/")).

And that's exactly what it does.

Message #64 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Wed, 10 Jul 2019 13:55:33 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> "~/" isn't something you'll ever get from functions like
>> directory-files
>
> That's sheer luck, because:
>
>   (file-name-as-directory "~")
>     => "~/"
>
> So just running "~" through an innocent API gives you a "magic"
> directory name (if you consider "~" not "magic" by itself).  How is
> this different from the "odd" use case where one must quote "~" to
> avoid its interpretation as the home directory?  Who can guarantee
> that some day directory-files-recursively will not want to do
> something like the above?  If it does, we will be right back at the
> same problem.

Well...  That kinda sounds odd to me.

"~/" is not, and never will be, a valid file name in any OS that Emacs
is going to support from now on.  So having that have a special meaning
in `expand-file-name' is not surprising.  Having "~" do something
special is surprising.

But changing that is probably not going to happen, so how about just
clarifying the documentation in that function to say what "~" means
explicitly instead of the caller having to guess?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #67 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Erik Hahn <erik_hahn <at> gmx.de>
Cc: 36490 <at> debbugs.gnu.org
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Wed, 10 Jul 2019 14:04:42 +0200

Erik Hahn <erik_hahn <at> gmx.de> writes:

> Call (directory-files-recursively DIR ".*") where DIR contains a
> subdirectory named "~" but is not the parent of your home directory. The
> function will recurse into your home directory. This is probably because
> (expand-file-name "~" ...) returns the home directory.

This is now fixed on the trunk.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #74 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Wed, 10 Jul 2019 17:56:57 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: 36490 <at> debbugs.gnu.org,  erik_hahn <at> gmx.de
> Date: Wed, 10 Jul 2019 13:55:33 +0200
> 
> >   (file-name-as-directory "~")
> >     => "~/"
> >
> > So just running "~" through an innocent API gives you a "magic"
> > directory name (if you consider "~" not "magic" by itself).  How is
> > this different from the "odd" use case where one must quote "~" to
> > avoid its interpretation as the home directory?  Who can guarantee
> > that some day directory-files-recursively will not want to do
> > something like the above?  If it does, we will be right back at the
> > same problem.
> 
> Well...  That kinda sounds odd to me.
> 
> "~/" is not, and never will be, a valid file name in any OS that Emacs
> is going to support from now on.

I don't think I follow you.  "~" is a perfectly valid name of a
directory, and Emacs does support such names in general.  So I don't
think I understand why you are saying this will not be a valid file
name.  What did I miss?

> But changing that is probably not going to happen, so how about just
> clarifying the documentation in that function to say what "~" means
> explicitly instead of the caller having to guess?

I don't object to documenting this.

Message #77 received at 36490 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 36490 <at> debbugs.gnu.org, erik_hahn <at> gmx.de
Subject: Re: bug#36490: 26.1;
 directory-files-recursively breaks when it encounters a directory
 named "~"
Date: Thu, 11 Jul 2019 17:05:49 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> "~/" is not, and never will be, a valid file name in any OS that Emacs
>> is going to support from now on.
>
> I don't think I follow you.  "~" is a perfectly valid name of a
> directory, and Emacs does support such names in general.  So I don't
> think I understand why you are saying this will not be a valid file
> name.  What did I miss?

"~" and "~/" are not the same thing in any OS that Emacs is currently
supporting.  The first is a file name (including directories, which are
just files), while the second is a file name with a slash after it.

Inside Emacs, though, things are different.

>> But changing that is probably not going to happen, so how about just
>> clarifying the documentation in that function to say what "~" means
>> explicitly instead of the caller having to guess?
>
> I don't object to documenting this.

OK; I'll do so.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.