GNU bug report logs - #8035
Processing of .. in a file path after going thru symlink

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 8035 in the body.
You can then email your comments to 8035 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: spucci <slpnabble <at> blackberry-hill.com>
To: Bug-gnu-emacs <at> gnu.org
Subject: Processing of .. in a file path after going thru symlink
Date: Sun, 13 Feb 2011 19:05:40 -0800 (PST)

The behavior is general, but it's a particular problem with next-error: 

When emacs attempts to find a file with "../" as a path component, it
appears to be "smart" about it and simply remove the previous directory path
(e.g., "foo/bar/../x" gets converted to "foo/x").  But if bar is a symlink,
then it doesn't properly find the file.  So in compiler output, which
references such files, the next-error function fails to find the file with
the given name. 

mkdir dest 
mkdir dest/subdir 
mkdir src 
ln -s ../dest/subdir src/subdir 
echo "#error This is an error" > dest/foo.c 

Now M-x compile, and give it cc -c src/subdir/../foo.c 

*compilation* buffer has: 
cc -c src/subdir/../foo.c 
src/subdir/../foo.c:1:2: error: #error This is an error 

and do a next-error: Emacs complains it can't find the file.  And if you try
to find-file that file path (src/subdir/../foo.c) it doesn't work either. 
Nor does "emacsclient src/subdir/../foo.c.  I couldn't find an option to
disable this behavior; it seems like there should be one even if the default
continues to be as it is today.

This is Gnu Emacs 23.2.1 I built myself on MacOS X 10.6.6. 

Thanks, 
  Steve 

-- 
View this message in context: http://old.nabble.com/Processing-of-..-in-a-file-path-after-going-thru-symlink-tp30918394p30918394.html
Sent from the Emacs - Bugs mailing list archive at Nabble.com.

Message #8 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: spucci <slpnabble <at> blackberry-hill.com>
Cc: 8035 <at> debbugs.gnu.org
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Sat, 19 Feb 2011 16:37:29 -0500

spucci wrote:

> When emacs attempts to find a file with "../" as a path component, it
> appears to be "smart" about it and simply remove the previous directory path
> (e.g., "foo/bar/../x" gets converted to "foo/x").  But if bar is a symlink,
> then it doesn't properly find the file.

This appears to be a feature of expand-file-name (and possibly other
things). It does seem a little weird that there isn't even an option to
have a more thorough expansion...

Message #11 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Steve Pucci <spucci <at> mac.com>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 8035 <at> debbugs.gnu.org
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Sat, 19 Feb 2011 18:35:35 -0800

On Feb 19, 2011, at 1:37 PM, Glenn Morris wrote:

> This appears to be a feature of expand-file-name (and possibly other
> things). It does seem a little weird that there isn't even an option to
> have a more thorough expansion...

So I ran an experiment, redefining expand-file-name as follows to skip the ".." processing (except in default-directory) and otherwise do the rest:

(or (fboundp 'save-expand-file-name)
    (fset 'save-expand-file-name (symbol-function 'expand-file-name)))
(defun expand-file-name (NAME &optional DEFAULT-DIRECTORY)
  (cond ((string-match "^/" NAME)
         NAME)
        ((string-match "^\\(~[^/]*\\)\\(.*\\)$" NAME)
         (let ((userdir (match-string 1 NAME))
               (rest (match-string 2 NAME)))
           (concat (save-expand-file-name userdir) rest)))
        (t (concat (save-expand-file-name (if DEFAULT-DIRECTORY
                                              DEFAULT-DIRECTORY
                                            default-directory))
                   NAME))))

While this does the trick (it expands only ~ and relative paths and properly leaves the OP path intact), it fails to fix the problem with next-error.  So I conclude you're right there are other places that do the same thing, apparently.

My workaround now is to wrap my build script in a Perl script which rewrites all "../" paths it finds within its output.  So I'm ok, though I'm surprised this hasn't come up before...

Message #14 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Steve Pucci <spucci <at> mac.com>
To: Glenn Morris <rgm <at> gnu.org>, 8035 <at> debbugs.gnu.org
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Sat, 19 Feb 2011 18:41:11 -0800

On Feb 19, 2011, at 6:35 PM, Steve Pucci wrote:

> So I conclude you're right there are other places that do the same thing, apparently.

Well, my experiment didn't intercept the (literally) 98 places that call Fexpand_file_name() directly from the C code, of course...

Message #17 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: spucci <slpnabble <at> blackberry-hill.com>
Cc: 8035 <at> debbugs.gnu.org
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Thu, 26 Aug 2021 18:29:54 +0200

spucci <slpnabble <at> blackberry-hill.com> writes:

> When emacs attempts to find a file with "../" as a path component, it
> appears to be "smart" about it and simply remove the previous directory path
> (e.g., "foo/bar/../x" gets converted to "foo/x").  But if bar is a symlink,
> then it doesn't properly find the file.  So in compiler output, which
> references such files, the next-error function fails to find the file with
> the given name. 
>
> mkdir dest 
> mkdir dest/subdir 
> mkdir src 
> ln -s ../dest/subdir src/subdir 
> echo "#error This is an error" > dest/foo.c 
>
> Now M-x compile, and give it cc -c src/subdir/../foo.c 
>
> *compilation* buffer has: 
> cc -c src/subdir/../foo.c 
> src/subdir/../foo.c:1:2: error: #error This is an error 
>
> and do a next-error: Emacs complains it can't find the file. 

This is an even more foundational problem:

(file-exists-p "/tmp/comp/src/subdir/../foo.c")
=> nil
(file-truename "/tmp/comp/src/subdir/../foo.c")
=> "/tmp/comp/dest/foo.c"
(file-exists-p (file-truename "/tmp/comp/src/subdir/../foo.c"))
=> t

And this is because:

static Lisp_Object
check_file_access (Lisp_Object file, Lisp_Object operation, int amode)
{
  file = Fexpand_file_name (file, Qnil);

I'm guessing it's calling expand-file-name here to resolve "~"?  It's
also a micro-optimisation, I guess -- collapsing "parent/.." textually
without checking the file system is very cheap.

But...  this does mean that `file-exists-p' and friends are unreliable
in the presence of symlinks, which is pretty depressing.  I think that
`expand-file-name' call in check_file_access should be changed to
something that just does the "~" expansion.

Any opinions?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #22 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 8035 <at> debbugs.gnu.org, slpnabble <at> blackberry-hill.com
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Thu, 26 Aug 2021 19:44:11 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Date: Thu, 26 Aug 2021 18:29:54 +0200
> Cc: 8035 <at> debbugs.gnu.org
> 
> (file-exists-p "/tmp/comp/src/subdir/../foo.c")
> => nil
> (file-truename "/tmp/comp/src/subdir/../foo.c")
> => "/tmp/comp/dest/foo.c"
> (file-exists-p (file-truename "/tmp/comp/src/subdir/../foo.c"))
> => t
> 
> And this is because:
> 
> static Lisp_Object
> check_file_access (Lisp_Object file, Lisp_Object operation, int amode)
> {
>   file = Fexpand_file_name (file, Qnil);
> 
> I'm guessing it's calling expand-file-name here to resolve "~"?

No, because we always must call expand-file-name before invoking a
libc function that accesses files -- to support file names relative to
their buffer's default-directory.

> I think that `expand-file-name' call in check_file_access should be
> changed to something that just does the "~" expansion.

I don't think we can do that, for the reason explained above.

Message #25 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 8035 <at> debbugs.gnu.org, spucci <slpnabble <at> blackberry-hill.com>
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Thu, 26 Aug 2021 18:53:31 +0200

On Aug 26 2021, Lars Ingebrigtsen wrote:

> But...  this does mean that `file-exists-p' and friends are unreliable
> in the presence of symlinks,

This is not true.  It is consistent with all other file handling
functions in Emacs.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Message #28 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 8035 <at> debbugs.gnu.org, slpnabble <at> blackberry-hill.com
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Thu, 26 Aug 2021 19:03:24 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> I'm guessing it's calling expand-file-name here to resolve "~"?
>
> No, because we always must call expand-file-name before invoking a
> libc function that accesses files -- to support file names relative to
> their buffer's default-directory.

These are absolute file names, though.  I can understand expanding if
they're not.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #31 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Andreas Schwab <schwab <at> linux-m68k.org>
Cc: 8035 <at> debbugs.gnu.org, spucci <slpnabble <at> blackberry-hill.com>
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Thu, 26 Aug 2021 19:04:28 +0200

Andreas Schwab <schwab <at> linux-m68k.org> writes:

> On Aug 26 2021, Lars Ingebrigtsen wrote:
>
>> But...  this does mean that `file-exists-p' and friends are unreliable
>> in the presence of symlinks,
>
> This is not true.  It is consistent with all other file handling
> functions in Emacs.

I don't see a contradiction between `file-exists-p' being unreliable
and being consistent with all other file handling functions in Emacs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #34 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 8035 <at> debbugs.gnu.org, slpnabble <at> blackberry-hill.com
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Thu, 26 Aug 2021 20:08:30 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: slpnabble <at> blackberry-hill.com,  8035 <at> debbugs.gnu.org
> Date: Thu, 26 Aug 2021 19:03:24 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> I'm guessing it's calling expand-file-name here to resolve "~"?
> >
> > No, because we always must call expand-file-name before invoking a
> > libc function that accesses files -- to support file names relative to
> > their buffer's default-directory.
> 
> These are absolute file names, though.  I can understand expanding if
> they're not.

You asked why expand-file-name is there in general, so I gave a
general explanation.

expand-file-name also normalizes file names, though.  Which is also
needed in some use cases, e.g. when the /foo/bar/ isn't accessible,
but /foo/ is, in which case failing to normalize /foo/bar/../quux
could fail some file I/O function.

Message #37 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 8035 <at> debbugs.gnu.org, spucci <slpnabble <at> blackberry-hill.com>
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Thu, 26 Aug 2021 19:19:58 +0200

On Aug 26 2021, Lars Ingebrigtsen wrote:

> Andreas Schwab <schwab <at> linux-m68k.org> writes:
>
>> On Aug 26 2021, Lars Ingebrigtsen wrote:
>>
>>> But...  this does mean that `file-exists-p' and friends are unreliable
>>> in the presence of symlinks,
>>
>> This is not true.  It is consistent with all other file handling
>> functions in Emacs.
>
> I don't see a contradiction between `file-exists-p' being unreliable
> and being consistent with all other file handling functions in Emacs.

It would be unreliable if it were inconsistent.

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Message #40 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 8035 <at> debbugs.gnu.org, slpnabble <at> blackberry-hill.com
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Thu, 26 Aug 2021 19:24:28 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> expand-file-name also normalizes file names, though.  Which is also
> needed in some use cases, e.g. when the /foo/bar/ isn't accessible,
> but /foo/ is, in which case failing to normalize /foo/bar/../quux
> could fail some file I/O function.

Hm, yes, that's a point...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #43 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: spucci <slpnabble <at> blackberry-hill.com>
Cc: 8035 <at> debbugs.gnu.org
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Thu, 26 Aug 2021 19:30:16 +0200

spucci <slpnabble <at> blackberry-hill.com> writes:

> mkdir dest 
> mkdir dest/subdir 
> mkdir src 
> ln -s ../dest/subdir src/subdir 
> echo "#error This is an error" > dest/foo.c 
>
> Now M-x compile, and give it cc -c src/subdir/../foo.c 

This should now be fixed in Emacs 28.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #46 received at 8035 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 8035 <at> debbugs.gnu.org, slpnabble <at> blackberry-hill.com
Subject: Re: bug#8035: Processing of .. in a file path after going thru symlink
Date: Thu, 26 Aug 2021 20:30:29 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: slpnabble <at> blackberry-hill.com,  8035 <at> debbugs.gnu.org
> Date: Thu, 26 Aug 2021 19:24:28 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > expand-file-name also normalizes file names, though.  Which is also
> > needed in some use cases, e.g. when the /foo/bar/ isn't accessible,
> > but /foo/ is, in which case failing to normalize /foo/bar/../quux
> > could fail some file I/O function.
> 
> Hm, yes, that's a point...

The OP's use case is clearly documented in the ELisp manual:

     Note also that ‘expand-file-name’ does not follow symbolic links at
     any level.  This results in a difference between the way
     ‘file-truename’ and ‘expand-file-name’ treat ‘..’.  Assuming that
     ‘/tmp/bar’ is a symbolic link to the directory ‘/tmp/foo/bar’ we
     get:

          (file-truename "/tmp/bar/../myfile")
               ⇒ "/tmp/foo/myfile"
          (expand-file-name "/tmp/bar/../myfile")
               ⇒ "/tmp/myfile"

     If you may need to follow symbolic links preceding ‘..’, you should
     make sure to call ‘file-truename’ without prior direct or indirect
     calls to ‘expand-file-name’.  *Note Truenames::.

So I think we need to use file-truename if we want to support such
uses of symlinks in "M-x compile" etc.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.