GNU bug report logs - #77122
[PATCH] project--find-in-directory resolves symlinks

Previous Next

To reply to this bug, email your comments to 77122 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [PATCH] project--find-in-directory resolves symlinks
Date: Wed, 19 Mar 2025 12:55:09 -0400

[Message part 1 (text/plain, inline)]

This avoids duplicate projects accessed via symlinks that share resolved
directory locations.  This has bothered me for a while, so here we go.

-Stephane

[Message part 2 (text/html, inline)]

[0001-project-find-in-directory-resolves-symlinks-bug-xxx.patch (application/octet-stream, attachment)]

Message #8 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: 77122 <at> debbugs.gnu.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Wed, 19 Mar 2025 13:04:32 -0400

[Message part 1 (text/plain, inline)]

On Wed, Mar 19, 2025 at 12:57 PM Ship Mints <shipmints <at> gmail.com> wrote:

> This avoids duplicate projects accessed via symlinks that share resolved
> directory locations.  This has bothered me for a while, so here we go.
>

FWIW, I consider this a bug and would like to see this in Emacs 30.2.

[Message part 2 (text/html, inline)]

Message #11 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Wed, 19 Mar 2025 19:09:44 +0200

> From: Ship Mints <shipmints <at> gmail.com>
> Date: Wed, 19 Mar 2025 12:55:09 -0400
> 
> This avoids duplicate projects accessed via symlinks that share resolved directory locations.  This has
> bothered me for a while, so here we go.
> 
> From 074905ff365c6bcd7cbcf72e0158ae9db0908a20 Mon Sep 17 00:00:00 2001
> From: shipmints <shipmints <at> gmail.com>
> Date: Wed, 19 Mar 2025 12:50:28 -0400
> Subject: [PATCH] project--find-in-directory resolves symlinks (bug#xxx)
> 
> This avoids duplicate projects that share resolved directory locations.
> 
> * lisp/progmodes/project.el (project--find-in-directory): Invoke
> 'project-find-functions' on 'file-truename' resolved dir.
> ---
>  lisp/progmodes/project.el | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
> index 4eac541167a..45f5328f6be 100644
> --- a/lisp/progmodes/project.el
> +++ b/lisp/progmodes/project.el
> @@ -251,6 +251,10 @@ project-current
>      pr))
>  
>  (defun project--find-in-directory (dir)
> +  "Invoke `project-find-functions' for DIR.
> +DIR is resolved to its true name, 'chasing' symbolic links, to avoid
> +duplicate projects that share resolved directory locations."
> +  (setq dir (file-truename dir))

That's not necessarily what the user will want, since it changes the
name of the file (and potentially its directory as well).  If the file
or the directory have some "magical" name that matters, this will
backfire.

I think it is better to detect the fact that two directories point to
the same place via symlinks (e.g., with file-equal-p), and avoid
duplication in that case, but without actually substituting the file's
name by its truename.

But of course the final word is for Dmitry.

Message #14 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Wed, 19 Mar 2025 19:16:56 +0200

> From: Ship Mints <shipmints <at> gmail.com>
> Date: Wed, 19 Mar 2025 13:04:32 -0400
> 
> On Wed, Mar 19, 2025 at 12:57 PM Ship Mints <shipmints <at> gmail.com> wrote:
> 
>  This avoids duplicate projects accessed via symlinks that share resolved directory locations.  This has
>  bothered me for a while, so here we go.
> 
> FWIW, I consider this a bug and would like to see this in Emacs 30.2.

Let's first discuss how to fix the original problem.  If it turns out
that the best fix is what you proposed (and I explained why I'm not
sure it is), then it's unlikely to be in Emacs 30.2 because it changes
behavior as a side effect of the fix, and that change could cause more
serious problems.

Thanks.

Message #17 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77122 <at> debbugs.gnu.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Wed, 19 Mar 2025 13:18:49 -0400

[Message part 1 (text/plain, inline)]

On Wed, Mar 19, 2025 at 1:10 PM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Ship Mints <shipmints <at> gmail.com>
> > Date: Wed, 19 Mar 2025 12:55:09 -0400
> >
> > This avoids duplicate projects accessed via symlinks that share resolved
> directory locations.  This has
> > bothered me for a while, so here we go.
> >
> > From 074905ff365c6bcd7cbcf72e0158ae9db0908a20 Mon Sep 17 00:00:00 2001
> > From: shipmints <shipmints <at> gmail.com>
> > Date: Wed, 19 Mar 2025 12:50:28 -0400
> > Subject: [PATCH] project--find-in-directory resolves symlinks (bug#xxx)
> >
> > This avoids duplicate projects that share resolved directory locations.
> >
> > * lisp/progmodes/project.el (project--find-in-directory): Invoke
> > 'project-find-functions' on 'file-truename' resolved dir.
> > ---
> >  lisp/progmodes/project.el | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> > diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
> > index 4eac541167a..45f5328f6be 100644
> > --- a/lisp/progmodes/project.el
> > +++ b/lisp/progmodes/project.el
> > @@ -251,6 +251,10 @@ project-current
> >      pr))
> >
> >  (defun project--find-in-directory (dir)
> > +  "Invoke `project-find-functions' for DIR.
> > +DIR is resolved to its true name, 'chasing' symbolic links, to avoid
> > +duplicate projects that share resolved directory locations."
> > +  (setq dir (file-truename dir))
>
> That's not necessarily what the user will want, since it changes the
> name of the file (and potentially its directory as well).  If the file
> or the directory have some "magical" name that matters, this will
> backfire.
>
> I think it is better to detect the fact that two directories point to
> the same place via symlinks (e.g., with file-equal-p), and avoid
> duplication in that case, but without actually substituting the file's
> name by its truename.
>
> But of course the final word is for Dmitry.
>

I expect that (project-current) whether invoked when default-directory
points to a symlinked project or not, returns the same thing.  It's an
opaque value to users?

[Message part 2 (text/html, inline)]

Message #20 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Ship Mints <shipmints <at> gmail.com>, 77122 <at> debbugs.gnu.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Thu, 20 Mar 2025 04:15:49 +0200

Hi!

On 19/03/2025 18:55, Ship Mints wrote:
> This avoids duplicate projects accessed via symlinks that share resolved 
> directory locations.  This has bothered me for a while, so here we go.

Could you describe the reasons why your setup is the way it is?

Personally, I've never used symlinks at this level, and if I did, I'm 
not sure I would see the problem with project-root returning different 
strings for those.

What exactly is the problem? Having the "same" project multiple times in 
project history?

I can name a couple of (probably minor) downsides:

* The project detection would be slower (more disk I/O).
* project-root would more often return a string that is not a parent 
directory of default-directory. Some code out there probably 
soft-depends on that.

Message #23 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 77122 <at> debbugs.gnu.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Thu, 20 Mar 2025 07:19:10 -0400

[Message part 1 (text/plain, inline)]

On Wed, Mar 19, 2025 at 10:15 PM Dmitry Gutov <dmitry <at> gutov.dev> wrote:

> Hi!
>
> On 19/03/2025 18:55, Ship Mints wrote:
> > This avoids duplicate projects accessed via symlinks that share resolved
> > directory locations.  This has bothered me for a while, so here we go.
>
> Could you describe the reasons why your setup is the way it is?
>

Many of us here do the same out of habit and convenience to normalize our
own directory trees as an overlay on the varied-structured work we need to
deal with.

Personally, I've never used symlinks at this level, and if I did, I'm
> not sure I would see the problem with project-root returning different
> strings for those.
>

> What exactly is the problem? Having the "same" project multiple times in
> project history?
>

If one loads two files from the same project but one via the symlinked
path, and another via the natural path (often, an underlying tool will
choose natural while Emacs chooses default-directory), and one has tools
that compare project-likeness between/among buffers, they report different
projects.  That's misleading and annoying.


> I can name a couple of (probably minor) downsides:
>
> * The project detection would be slower (more disk I/O).
> * project-root would more often return a string that is not a parent
> directory of default-directory. Some code out there probably
> soft-depends on that.
>

I've been running with the following for a while and haven't found any
issues yet, hence the patch to help others with similar situations.

  (defun my/project--find-in-directory-advice (args)
    (cons (file-truename (car args)) (cdr args)))
  (advice-add #'project--find-in-directory :filter-args
#'my/project--find-in-directory-advice)

We can make this a user option nil (default), t (always), 'inhibit-remote
so the user can avoid hammering remote connections.  If we could normalize
at a lower level and still retain the symmetry between default-directory
and a normalized project object, that would suggest caches like the one in
project-try-vc have to cache more than one project directory form.  Doesn't
seem worth the complexity?

-Stephane

[Message part 2 (text/html, inline)]

Message #26 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 77122 <at> debbugs.gnu.org, Ship Mints <shipmints <at> gmail.com>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Sat, 22 Mar 2025 21:47:30 -0400

Dmitry Gutov <dmitry <at> gutov.dev> writes:

> Hi!
>
> On 19/03/2025 18:55, Ship Mints wrote:
>> This avoids duplicate projects accessed via symlinks that share
>> resolved directory locations.  This has bothered me for a while, so
>> here we go.
>
> Could you describe the reasons why your setup is the way it is?
>
> Personally, I've never used symlinks at this level, and if I did, I'm
> not sure I would see the problem with project-root returning different
> strings for those.
>
> What exactly is the problem? Having the "same" project multiple times
> in project history?
>
> I can name a couple of (probably minor) downsides:
>
> * The project detection would be slower (more disk I/O).
> * project-root would more often return a string that is not a parent
>   directory of default-directory. Some code out there probably
>   soft-depends on that.

FWIW, I sometimes symlink convenient names to inconvenient ones, and it
would be a shame to use the inconvenient names due to excessive
symlink chasing. (Other packages aren't great about this.) Can't people
who want symlink chasing add something to project-find-functions?

Message #29 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: 77122 <at> debbugs.gnu.org, Dmitry Gutov <dmitry <at> gutov.dev>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Mon, 24 Mar 2025 06:40:29 -0400

[Message part 1 (text/plain, inline)]

On Sat, Mar 22, 2025 at 9:47 PM Daniel Colascione <dancol <at> dancol.org> wrote:

> Dmitry Gutov <dmitry <at> gutov.dev> writes:
>
> > Hi!
> >
> > On 19/03/2025 18:55, Ship Mints wrote:
> >> This avoids duplicate projects accessed via symlinks that share
> >> resolved directory locations.  This has bothered me for a while, so
> >> here we go.
> >
> > Could you describe the reasons why your setup is the way it is?
> >
> > Personally, I've never used symlinks at this level, and if I did, I'm
> > not sure I would see the problem with project-root returning different
> > strings for those.
> >
> > What exactly is the problem? Having the "same" project multiple times
> > in project history?
> >
> > I can name a couple of (probably minor) downsides:
> >
> > * The project detection would be slower (more disk I/O).
> > * project-root would more often return a string that is not a parent
> >   directory of default-directory. Some code out there probably
> >   soft-depends on that.
>
> FWIW, I sometimes symlink convenient names to inconvenient ones, and it
> would be a shame to use the inconvenient names due to excessive
> symlink chasing. (Other packages aren't great about this.) Can't people
> who want symlink chasing add something to project-find-functions?
>

Yeah, good question.  "Can't people" applies pretty much to every feature
and option added to Emacs that people can do themselves including
overriding functions wholesale.  The bigger picture, to me, is that if we
add options to do things like this for our users, then we can help more of
them, we can refine implementations at deeper levels should we need to, we
can prompt people to think more considerately about their working
environments.  Perhaps some never considered using symlinks as vanity paths
and they'll have the choice to treat project objects among those
paths canonically.

Since I suggested it be optional, it would be off by default, right?  If
you think you'd want to canonicalize paths to project roots in some places
and not others, perhaps we could contrive a project sentinel file ala
.project-notruename.el/d or .project-config.el/d, and for people that want
to do things in code, a list project can consult.

[Message part 2 (text/html, inline)]

Message #32 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Mon, 24 Mar 2025 15:03:02 +0200

> Cc: 77122 <at> debbugs.gnu.org, Dmitry Gutov <dmitry <at> gutov.dev>
> From: Ship Mints <shipmints <at> gmail.com>
> Date: Mon, 24 Mar 2025 06:40:29 -0400
> 
> Since I suggested it be optional, it would be off by default, right?  If you think you'd want to canonicalize
> paths to project roots in some places and not others, perhaps we could contrive a project sentinel file ala .
> project-notruename.el/d or .project-config.el/d, and for people that want to do things in code, a list project
> can consult.

AFAIR, you didn't respond to my suggest to try a different solution.
Namely, instead of changing the file name by resolving links,
something that could cause problems for some people, how about using
file-equal-p to avoid duplication of projects in these cases?  If my
proposal makes sense, it will allow use to avoid duplication without
changing the file names, because symlinks will be chased internally,
only where we decide whether two projects are different or not.

This is better than having an optional behavior, because inevitably
someone will want to use this option, but also wouldn't like his/her
project directories appear under their resolved names, and then we are
back at the same problem.

Message #35 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Mon, 24 Mar 2025 11:15:20 -0400

[Message part 1 (text/plain, inline)]

On Mon, Mar 24, 2025 at 9:03 AM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > Cc: 77122 <at> debbugs.gnu.org, Dmitry Gutov <dmitry <at> gutov.dev>
> > From: Ship Mints <shipmints <at> gmail.com>
> > Date: Mon, 24 Mar 2025 06:40:29 -0400
> >
> > Since I suggested it be optional, it would be off by default, right?  If
> you think you'd want to canonicalize
> > paths to project roots in some places and not others, perhaps we could
> contrive a project sentinel file ala .
> > project-notruename.el/d or .project-config.el/d, and for people that
> want to do things in code, a list project
> > can consult.
>
> AFAIR, you didn't respond to my suggest to try a different solution.
> Namely, instead of changing the file name by resolving links,
> something that could cause problems for some people, how about using
> file-equal-p to avoid duplication of projects in these cases?  If my
> proposal makes sense, it will allow use to avoid duplication without
> changing the file names, because symlinks will be chased internally,
> only where we decide whether two projects are different or not.
>
> This is better than having an optional behavior, because inevitably
> someone will want to use this option, but also wouldn't like his/her
> project directories appear under their resolved names, and then we are
> back at the same problem.
>

I did consider it.  The most popular project objects, project-vc, and
transient are defined as (list 'vc vcbackend dir) and (cons 'transient
dir), respectively and vc objects are cached.  If we don't canonicalize the
dir name, we can't find the vc cache entry if a probe is attempted from
another dir, even if that dir morally is equivalent.  One can't change the
embedded dir on the way out of without creating two objects, one with the
first location to create the cache entry, and a second from an equivalent
convenience dir.

I'm curious to hear more about why people would object to (project-root
project-obj) being canonicalized.  I don't think many people ever manually
enter project dirs.  The persisted known projects, I'd think, would also
benefit from no duplicates.

If you see the place in project.el where file-equal-p helps, I'll happily
hack on it.

[Message part 2 (text/html, inline)]

Message #38 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Mon, 24 Mar 2025 20:24:07 +0200

> From: Ship Mints <shipmints <at> gmail.com>
> Date: Mon, 24 Mar 2025 11:15:20 -0400
> Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> 
> 
> On Mon, Mar 24, 2025 at 9:03 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>  AFAIR, you didn't respond to my suggest to try a different solution.
>  Namely, instead of changing the file name by resolving links,
>  something that could cause problems for some people, how about using
>  file-equal-p to avoid duplication of projects in these cases?  If my
>  proposal makes sense, it will allow use to avoid duplication without
>  changing the file names, because symlinks will be chased internally,
>  only where we decide whether two projects are different or not.
> 
>  This is better than having an optional behavior, because inevitably
>  someone will want to use this option, but also wouldn't like his/her
>  project directories appear under their resolved names, and then we are
>  back at the same problem.
> 
> I did consider it.  The most popular project objects, project-vc, and transient are defined as (list 'vc
> vcbackend dir) and (cons 'transient dir), respectively and vc objects are cached.  If we don't canonicalize the
> dir name, we can't find the vc cache entry if a probe is attempted from another dir, even if that dir morally is
> equivalent.

Why can't we? what precludes that?

> If you see the place in project.el where file-equal-p helps, I'll happily hack on it. 

I'm sorry, I cannot afford looking through the project.el code to find
this.  But if the problem is that directories or files don't compare
equal, the file-equal-p is the way to go, so I don't understand why
the places where we do the comparison should be hard to find for
someone who knows their way in project.el's code and/or has enough
time to dig.

Message #41 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Mon, 24 Mar 2025 15:29:26 -0400

[Message part 1 (text/plain, inline)]

On Mon, Mar 24, 2025 at 2:24 PM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Ship Mints <shipmints <at> gmail.com>
> > Date: Mon, 24 Mar 2025 11:15:20 -0400
> > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> >
> >
> > On Mon, Mar 24, 2025 at 9:03 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
> >
> >  AFAIR, you didn't respond to my suggest to try a different solution.
> >  Namely, instead of changing the file name by resolving links,
> >  something that could cause problems for some people, how about using
> >  file-equal-p to avoid duplication of projects in these cases?  If my
> >  proposal makes sense, it will allow use to avoid duplication without
> >  changing the file names, because symlinks will be chased internally,
> >  only where we decide whether two projects are different or not.
> >
> >  This is better than having an optional behavior, because inevitably
> >  someone will want to use this option, but also wouldn't like his/her
> >  project directories appear under their resolved names, and then we are
> >  back at the same problem.
> >
> > I did consider it.  The most popular project objects, project-vc, and
> transient are defined as (list 'vc
> > vcbackend dir) and (cons 'transient dir), respectively and vc objects
> are cached.  If we don't canonicalize the
> > dir name, we can't find the vc cache entry if a probe is attempted from
> another dir, even if that dir morally is
> > equivalent.
>
> Why can't we? what precludes that?
>

Nothing.  Just a matter of programming, as always.  We need to restructure
some of project.el to make this work well.  I'll have a deeper look/think.

> If you see the place in project.el where file-equal-p helps, I'll happily
> hack on it.
>
> I'm sorry, I cannot afford looking through the project.el code to find
> this.  But if the problem is that directories or files don't compare
> equal, the file-equal-p is the way to go, so I don't understand why
> the places where we do the comparison should be hard to find for
> someone who knows their way in project.el's code and/or has enough
> time to dig.
>

You don't have to look.  The issue is that no directories are explicitly
compared.  You just have to humor that it's a bit evil that a
singular project approached from different places produces two different
project objects.

-Stephane

[Message part 2 (text/html, inline)]

Message #44 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Mon, 24 Mar 2025 21:34:38 +0200

> From: Ship Mints <shipmints <at> gmail.com>
> Date: Mon, 24 Mar 2025 15:29:26 -0400
> Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> 
>  > If you see the place in project.el where file-equal-p helps, I'll happily hack on it. 
> 
>  I'm sorry, I cannot afford looking through the project.el code to find
>  this.  But if the problem is that directories or files don't compare
>  equal, the file-equal-p is the way to go, so I don't understand why
>  the places where we do the comparison should be hard to find for
>  someone who knows their way in project.el's code and/or has enough
>  time to dig.
> 
> You don't have to look.  The issue is that no directories are explicitly compared.  You just have to humor
> that it's a bit evil that a singular project approached from different places produces two different project
> objects.

I'm confused: how does project.el know it's the same or a different
project, without comparing?

Message #47 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Mon, 24 Mar 2025 15:39:30 -0400

[Message part 1 (text/plain, inline)]

On Mon, Mar 24, 2025 at 3:34 PM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Ship Mints <shipmints <at> gmail.com>
> > Date: Mon, 24 Mar 2025 15:29:26 -0400
> > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> >
> >  > If you see the place in project.el where file-equal-p helps, I'll
> happily hack on it.
> >
> >  I'm sorry, I cannot afford looking through the project.el code to find
> >  this.  But if the problem is that directories or files don't compare
> >  equal, the file-equal-p is the way to go, so I don't understand why
> >  the places where we do the comparison should be hard to find for
> >  someone who knows their way in project.el's code and/or has enough
> >  time to dig.
> >
> > You don't have to look.  The issue is that no directories are explicitly
> compared.  You just have to humor
> > that it's a bit evil that a singular project approached from different
> places produces two different project
> > objects.
>
> I'm confused: how does project.el know it's the same or a different
> project, without comparing?
>

It looks for root markers in/below the specified directory such as .git for
project-vc.  Once found, it records a project object in a cache based on
the dir it originally searched.  If approached from another directory, it
repeats the process naively.  That's what using the canonical name solved
for--that all searches use the same key.  The resulting "issue" would be
that calling (project-root project-object) might return a directory
different than default-directory for a particular buffer.  It wouldn't be a
misrepresentation.

[Message part 2 (text/html, inline)]

Message #50 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Mon, 24 Mar 2025 22:05:06 +0200

> From: Ship Mints <shipmints <at> gmail.com>
> Date: Mon, 24 Mar 2025 15:39:30 -0400
> Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> 
> 
> On Mon, Mar 24, 2025 at 3:34 PM Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>  > From: Ship Mints <shipmints <at> gmail.com>
>  > Date: Mon, 24 Mar 2025 15:29:26 -0400
>  > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
>  > 
>  >  > If you see the place in project.el where file-equal-p helps, I'll happily hack on it. 
>  > 
>  >  I'm sorry, I cannot afford looking through the project.el code to find
>  >  this.  But if the problem is that directories or files don't compare
>  >  equal, the file-equal-p is the way to go, so I don't understand why
>  >  the places where we do the comparison should be hard to find for
>  >  someone who knows their way in project.el's code and/or has enough
>  >  time to dig.
>  > 
>  > You don't have to look.  The issue is that no directories are explicitly compared.  You just have to
>  humor
>  > that it's a bit evil that a singular project approached from different places produces two different
>  project
>  > objects.
> 
>  I'm confused: how does project.el know it's the same or a different
>  project, without comparing?
> 
> It looks for root markers in/below the specified directory such as .git for project-vc.  Once found, it records a
> project object in a cache based on the dir it originally searched.  If approached from another directory, it
> repeats the process naively.

In this description, the directory appears twice.  Assuming that the
project object either records the directory or the cache uses the
directory as the key, using file-equal-p should solve the problem, AFAIU.

> That's what using the canonical name solved for--that all searches use the
> same key.

You can use the same key when searching without canonicalizing, can't you?

> The resulting "issue" would be that calling (project-root project-object) might return a directory
> different than default-directory for a particular buffer.  It wouldn't be a misrepresentation.

As this discussion shows, it might well be a "misrepresentation" in
some cases.

Message #53 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Mon, 24 Mar 2025 16:15:14 -0400

[Message part 1 (text/plain, inline)]

On Mon, Mar 24, 2025 at 4:05 PM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Ship Mints <shipmints <at> gmail.com>
> > Date: Mon, 24 Mar 2025 15:39:30 -0400
> > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> >
> >
> > On Mon, Mar 24, 2025 at 3:34 PM Eli Zaretskii <eliz <at> gnu.org> wrote:
> >
> >  > From: Ship Mints <shipmints <at> gmail.com>
> >  > Date: Mon, 24 Mar 2025 15:29:26 -0400
> >  > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> >  >
> >  >  > If you see the place in project.el where file-equal-p helps, I'll
> happily hack on it.
> >  >
> >  >  I'm sorry, I cannot afford looking through the project.el code to
> find
> >  >  this.  But if the problem is that directories or files don't compare
> >  >  equal, the file-equal-p is the way to go, so I don't understand why
> >  >  the places where we do the comparison should be hard to find for
> >  >  someone who knows their way in project.el's code and/or has enough
> >  >  time to dig.
> >  >
> >  > You don't have to look.  The issue is that no directories are
> explicitly compared.  You just have to
> >  humor
> >  > that it's a bit evil that a singular project approached from
> different places produces two different
> >  project
> >  > objects.
> >
> >  I'm confused: how does project.el know it's the same or a different
> >  project, without comparing?
> >
> > It looks for root markers in/below the specified directory such as .git
> for project-vc.  Once found, it records a
> > project object in a cache based on the dir it originally searched.  If
> approached from another directory, it
> > repeats the process naively.
>
> In this description, the directory appears twice.  Assuming that the
> project object either records the directory or the cache uses the
> directory as the key, using file-equal-p should solve the problem, AFAIU.
>
> > That's what using the canonical name solved for--that all searches use
> the
> > same key.
>
> You can use the same key when searching without canonicalizing, can't you?
>
> > The resulting "issue" would be that calling (project-root
> project-object) might return a directory
> > different than default-directory for a particular buffer.  It wouldn't
> be a misrepresentation.
>
> As this discussion shows, it might well be a "misrepresentation" in
> some cases.
>

Maybe unexpected but not a misrepresentation.

In any case, the project object returned must be equivalent for both
directories passed in and that's not how project.el is structured.  Using
file-equal-p or any other method to find an "equivalent" project object but
substitute the "expected" directory results in two objects that don't
compare as equal and that's a misrepresentation IMO.

[Message part 2 (text/html, inline)]

Message #56 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Mon, 24 Mar 2025 16:25:44 -0400

[Message part 1 (text/plain, inline)]

On Mon, Mar 24, 2025 at 4:15 PM Ship Mints <shipmints <at> gmail.com> wrote:

> On Mon, Mar 24, 2025 at 4:05 PM Eli Zaretskii <eliz <at> gnu.org> wrote:
>
>> > From: Ship Mints <shipmints <at> gmail.com>
>> > Date: Mon, 24 Mar 2025 15:39:30 -0400
>> > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
>> >
>> >
>> > On Mon, Mar 24, 2025 at 3:34 PM Eli Zaretskii <eliz <at> gnu.org> wrote:
>> >
>> >  > From: Ship Mints <shipmints <at> gmail.com>
>> >  > Date: Mon, 24 Mar 2025 15:29:26 -0400
>> >  > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
>> >  >
>> >  >  > If you see the place in project.el where file-equal-p helps, I'll
>> happily hack on it.
>> >  >
>> >  >  I'm sorry, I cannot afford looking through the project.el code to
>> find
>> >  >  this.  But if the problem is that directories or files don't compare
>> >  >  equal, the file-equal-p is the way to go, so I don't understand why
>> >  >  the places where we do the comparison should be hard to find for
>> >  >  someone who knows their way in project.el's code and/or has enough
>> >  >  time to dig.
>> >  >
>> >  > You don't have to look.  The issue is that no directories are
>> explicitly compared.  You just have to
>> >  humor
>> >  > that it's a bit evil that a singular project approached from
>> different places produces two different
>> >  project
>> >  > objects.
>> >
>> >  I'm confused: how does project.el know it's the same or a different
>> >  project, without comparing?
>> >
>> > It looks for root markers in/below the specified directory such as .git
>> for project-vc.  Once found, it records a
>> > project object in a cache based on the dir it originally searched.  If
>> approached from another directory, it
>> > repeats the process naively.
>>
>> In this description, the directory appears twice.  Assuming that the
>> project object either records the directory or the cache uses the
>> directory as the key, using file-equal-p should solve the problem, AFAIU.
>>
>> > That's what using the canonical name solved for--that all searches use
>> the
>> > same key.
>>
>> You can use the same key when searching without canonicalizing, can't you?
>>
>> > The resulting "issue" would be that calling (project-root
>> project-object) might return a directory
>> > different than default-directory for a particular buffer.  It wouldn't
>> be a misrepresentation.
>>
>> As this discussion shows, it might well be a "misrepresentation" in
>> some cases.
>>
>
> Maybe unexpected but not a misrepresentation.
>
> In any case, the project object returned must be equivalent for both
> directories passed in and that's not how project.el is structured.  Using
> file-equal-p or any other method to find an "equivalent" project object but
> substitute the "expected" directory results in two objects that don't
> compare as equal and that's a misrepresentation IMO.
>

To concretely demonstrate the differences, the function project-name will
return different results for each project object based on buffers loaded
from different paths, despite the projects being equivalent.

project-name is defined as:

(file-name-nondirectory (directory-file-name (project-root project)))

If the root directory is determined to be different, the objects return
different names (and different roots).

[Message part 2 (text/html, inline)]

Message #59 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Ship Mints <shipmints <at> gmail.com>, Eli Zaretskii <eliz <at> gnu.org>
Cc: 77122 <at> debbugs.gnu.org, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 04:16:33 +0200

On 24/03/2025 17:15, Ship Mints wrote:
> I'm curious to hear more about why people would object to (project-root 
> project-obj) being canonicalized.  I don't think many people ever 
> manually enter project dirs.  The persisted known projects, I'd think, 
> would also benefit from no duplicates.

I wonder if you yourself would prefer for all buffer-file-name values, 
and default-directory values, to be canonicalized as well.

The same reasoning would seem to apply to them too anyway.

Message #62 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 77122 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 06:54:34 -0400

[Message part 1 (text/plain, inline)]

On Mon, Mar 24, 2025 at 10:16 PM Dmitry Gutov <dmitry <at> gutov.dev> wrote:

> On 24/03/2025 17:15, Ship Mints wrote:
> > I'm curious to hear more about why people would object to (project-root
> > project-obj) being canonicalized.  I don't think many people ever
> > manually enter project dirs.  The persisted known projects, I'd think,
> > would also benefit from no duplicates.
>
> I wonder if you yourself would prefer for all buffer-file-name values,
> and default-directory values, to be canonicalized as well.
>
> The same reasoning would seem to apply to them too anyway.
>

Ochen funny. I'll submit a separate patch for that one day (sarcasm doesn't
work in email, sorry).

In the meantime, I maintain my view that project.el needs to report uniform
project names and roots for identical projects approached from different
places, even if optional.

I just took a look at projectile.el, which I'd never looked at before
because I prefer using/improving core features.  It has a longer user
history to see what they've experienced (and it looks like some of
project.el's approach is copied almost verbatim e.g., the implementation of
project-name).  projectile seems to both have users that want symlink
chasing and those that don't (looks like ClearCase users--but out of
necessity not desire?).  As those concerns seem to be project dependent, we
could an option that is a list of paths or matchers to include/exclude from
chasing, and also a project root semaphore file or project config as I
suggested in another message in this thread.

I'd enable chasing as my default, opt out in a specific project should I
ever have one that needs it rather than the other way around, and enjoy
project-name and project-root uniformity.

-Stephane

[Message part 2 (text/html, inline)]

Message #65 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 14:15:07 +0200

> From: Ship Mints <shipmints <at> gmail.com>
> Date: Mon, 24 Mar 2025 16:15:14 -0400
> Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> 
>  As this discussion shows, it might well be a "misrepresentation" in
>  some cases.
> 
> Maybe unexpected but not a misrepresentation.
> 
> In any case, the project object returned must be equivalent for both directories passed in and that's not how
> project.el is structured.  Using file-equal-p or any other method to find an "equivalent" project object but
> substitute the "expected" directory results in two objects that don't compare as equal and that's a
> misrepresentation IMO.

Thus my suggestion to make those objects 'equal', even if they aren't
'eq'.

Message #68 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 14:16:56 +0200

> From: Ship Mints <shipmints <at> gmail.com>
> Date: Mon, 24 Mar 2025 16:25:44 -0400
> Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> 
> To concretely demonstrate the differences, the function project-name will return different results for each
> project object based on buffers loaded from different paths, despite the projects being equivalent.
> 
> project-name is defined as:
> 
> (file-name-nondirectory (directory-file-name (project-root project)))
> 
> If the root directory is determined to be different, the objects return different names (and different roots).

Unless I'm misunderstanding what Daniel wrote, the above is actually a
feature from his POV.

Message #71 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 08:26:11 -0400

[Message part 1 (text/plain, inline)]

On Tue, Mar 25, 2025 at 8:17 AM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Ship Mints <shipmints <at> gmail.com>
> > Date: Mon, 24 Mar 2025 16:25:44 -0400
> > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> >
> > To concretely demonstrate the differences, the function project-name
> will return different results for each
> > project object based on buffers loaded from different paths, despite the
> projects being equivalent.
> >
> > project-name is defined as:
> >
> > (file-name-nondirectory (directory-file-name (project-root project)))
> >
> > If the root directory is determined to be different, the objects return
> different names (and different roots).
>
> Unless I'm misunderstanding what Daniel wrote, the above is actually a
> feature from his POV.
>

Indeed, it might be, but I'm curious what tooling he's using that depends
on project-root not being "absolute" so to speak, and relative to his
ambient default-directory.

[Message part 2 (text/html, inline)]

Message #74 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 08:27:45 -0400

[Message part 1 (text/plain, inline)]

On Tue, Mar 25, 2025 at 8:15 AM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Ship Mints <shipmints <at> gmail.com>
> > Date: Mon, 24 Mar 2025 16:15:14 -0400
> > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> >
> >  As this discussion shows, it might well be a "misrepresentation" in
> >  some cases.
> >
> > Maybe unexpected but not a misrepresentation.
> >
> > In any case, the project object returned must be equivalent for both
> directories passed in and that's not how
> > project.el is structured.  Using file-equal-p or any other method to
> find an "equivalent" project object but
> > substitute the "expected" directory results in two objects that don't
> compare as equal and that's a
> > misrepresentation IMO.
>
> Thus my suggestion to make those objects 'equal', even if they aren't
> 'eq'.
>

I hear you, 100%.  But I don't see a place to perform that comparison given
the current implementation.

[Message part 2 (text/html, inline)]

Message #77 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 08:30:19 -0400

[Message part 1 (text/plain, inline)]

On Tue, Mar 25, 2025 at 8:27 AM Ship Mints <shipmints <at> gmail.com> wrote:

> On Tue, Mar 25, 2025 at 8:15 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
>
>> > From: Ship Mints <shipmints <at> gmail.com>
>> > Date: Mon, 24 Mar 2025 16:15:14 -0400
>> > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
>> >
>> >  As this discussion shows, it might well be a "misrepresentation" in
>> >  some cases.
>> >
>> > Maybe unexpected but not a misrepresentation.
>> >
>> > In any case, the project object returned must be equivalent for both
>> directories passed in and that's not how
>> > project.el is structured.  Using file-equal-p or any other method to
>> find an "equivalent" project object but
>> > substitute the "expected" directory results in two objects that don't
>> compare as equal and that's a
>> > misrepresentation IMO.
>>
>> Thus my suggestion to make those objects 'equal', even if they aren't
>> 'eq'.
>>
>
> I hear you, 100%.  But I don't see a place to perform that comparison
> given the current implementation.
>

I'll give you a concrete example I've been playing with.  I keep my Emacs
init files in a git repo.  I put the repo in ~/proj/emacs.d.  In my home
directory, I ln -s ~/proj/emacs.d .emacs.d.  In this set up, project-name
reports "emacs.d" when default-directory is relative to ~/proj/emacs.d and
".emacs.d" when relative to my home directory.  It's the same project and I
expect the name to be the same.  Also for project-root, as I've said.

[Message part 2 (text/html, inline)]

Message #80 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Ship Mints <shipmints <at> gmail.com>, Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 77122 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 08:54:53 -0400

On March 25, 2025 6:54:34 AM EDT, Ship Mints <shipmints <at> gmail.com> wrote:
>On Mon, Mar 24, 2025 at 10:16 PM Dmitry Gutov <dmitry <at> gutov.dev> wrote:
>
>> On 24/03/2025 17:15, Ship Mints wrote:
>> > I'm curious to hear more about why people would object to (project-root
>> > project-obj) being canonicalized.  I don't think many people ever
>> > manually enter project dirs.  The persisted known projects, I'd think,
>> > would also benefit from no duplicates.
>>
>> I wonder if you yourself would prefer for all buffer-file-name values,
>> and default-directory values, to be canonicalized as well.
>>
>> The same reasoning would seem to apply to them too anyway.
>>
>
>Ochen funny. I'll submit a separate patch for that one day (sarcasm doesn't
>work in email, sorry).
>
>In the meantime, I maintain my view that project.el needs to report uniform
>project names and roots for identical projects approached from different
>places, even if optional.
>
>I just took a look at projectile.el, which I'd never looked at before
>because I prefer using/improving core features.  It has a longer user
>history to see what they've experienced (and it looks like some of
>project.el's approach is copied almost verbatim e.g., the implementation of
>project-name).  projectile seems to both have users that want symlink
>chasing and those that don't (looks like ClearCase users--but out of
>necessity not desire?).  As those concerns seem to be project dependent, we
>could an option that is a list of paths or matchers to include/exclude from
>chasing, and also a project root semaphore file or project config as I
>suggested in another message in this thread.
>
>I'd enable chasing as my default, opt out in a specific project should I
>ever have one that needs it rather than the other way around, and enjoy
>project-name and project-root uniformity.
>
>-Stephane

Eli is right. You want either global path normalization or you don't. Doing it peacemeal at the project level might satisfy some small immediate need, but it won't solve the whole problem and will inevitably cause downstream problems due to the incoherence.

Also, it's just not universally true that I want to regard the "real" version of something as its realpath. People use symlinks differently. In a world in which we had ubiquitous unprivileged bind mounts *and* symlinks, the realpath as identity concept might be more powerful, but we don't.

Message #83 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: 77122 <at> debbugs.gnu.org, Dmitry Gutov <dmitry <at> gutov.dev>,
 Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 09:01:32 -0400

[Message part 1 (text/plain, inline)]

On Tue, Mar 25, 2025 at 8:54 AM Daniel Colascione <dancol <at> dancol.org> wrote:

>
>
> On March 25, 2025 6:54:34 AM EDT, Ship Mints <shipmints <at> gmail.com> wrote:
> >On Mon, Mar 24, 2025 at 10:16 PM Dmitry Gutov <dmitry <at> gutov.dev> wrote:
> >
> >> On 24/03/2025 17:15, Ship Mints wrote:
> >> > I'm curious to hear more about why people would object to
> (project-root
> >> > project-obj) being canonicalized.  I don't think many people ever
> >> > manually enter project dirs.  The persisted known projects, I'd think,
> >> > would also benefit from no duplicates.
> >>
> >> I wonder if you yourself would prefer for all buffer-file-name values,
> >> and default-directory values, to be canonicalized as well.
> >>
> >> The same reasoning would seem to apply to them too anyway.
> >>
> >
> >Ochen funny. I'll submit a separate patch for that one day (sarcasm
> doesn't
> >work in email, sorry).
> >
> >In the meantime, I maintain my view that project.el needs to report
> uniform
> >project names and roots for identical projects approached from different
> >places, even if optional.
> >
> >I just took a look at projectile.el, which I'd never looked at before
> >because I prefer using/improving core features.  It has a longer user
> >history to see what they've experienced (and it looks like some of
> >project.el's approach is copied almost verbatim e.g., the implementation
> of
> >project-name).  projectile seems to both have users that want symlink
> >chasing and those that don't (looks like ClearCase users--but out of
> >necessity not desire?).  As those concerns seem to be project dependent,
> we
> >could an option that is a list of paths or matchers to include/exclude
> from
> >chasing, and also a project root semaphore file or project config as I
> >suggested in another message in this thread.
> >
> >I'd enable chasing as my default, opt out in a specific project should I
> >ever have one that needs it rather than the other way around, and enjoy
> >project-name and project-root uniformity.
> >
> >-Stephane
>
> Eli is right. You want either global path normalization or you don't.
> Doing it peacemeal at the project level might satisfy some small immediate
> need, but it won't solve the whole problem and will inevitably cause
> downstream problems due to the incoherence.
>
> Also, it's just not universally true that I want to regard the "real"
> version of something as its realpath. People use symlinks differently. In a
> world in which we had ubiquitous unprivileged bind mounts *and* symlinks,
> the realpath as identity concept might be more powerful, but we don't.
>

Again...I'm not saying universally, I'm saying optionally, and preferably
flexibly optional.

[Message part 2 (text/html, inline)]

Message #86 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: 77122 <at> debbugs.gnu.org, Dmitry Gutov <dmitry <at> gutov.dev>,
 Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 11:42:11 -0400

[Message part 1 (text/plain, inline)]

On Tue, Mar 25, 2025 at 9:01 AM Ship Mints <shipmints <at> gmail.com> wrote:

> On Tue, Mar 25, 2025 at 8:54 AM Daniel Colascione <dancol <at> dancol.org>
> wrote:
>
>>
>>
>> On March 25, 2025 6:54:34 AM EDT, Ship Mints <shipmints <at> gmail.com> wrote:
>> >On Mon, Mar 24, 2025 at 10:16 PM Dmitry Gutov <dmitry <at> gutov.dev> wrote:
>> >
>> >> On 24/03/2025 17:15, Ship Mints wrote:
>> >> > I'm curious to hear more about why people would object to
>> (project-root
>> >> > project-obj) being canonicalized.  I don't think many people ever
>> >> > manually enter project dirs.  The persisted known projects, I'd
>> think,
>> >> > would also benefit from no duplicates.
>> >>
>> >> I wonder if you yourself would prefer for all buffer-file-name values,
>> >> and default-directory values, to be canonicalized as well.
>> >>
>> >> The same reasoning would seem to apply to them too anyway.
>> >>
>> >
>> >Ochen funny. I'll submit a separate patch for that one day (sarcasm
>> doesn't
>> >work in email, sorry).
>> >
>> >In the meantime, I maintain my view that project.el needs to report
>> uniform
>> >project names and roots for identical projects approached from different
>> >places, even if optional.
>> >
>> >I just took a look at projectile.el, which I'd never looked at before
>> >because I prefer using/improving core features.  It has a longer user
>> >history to see what they've experienced (and it looks like some of
>> >project.el's approach is copied almost verbatim e.g., the implementation
>> of
>> >project-name).  projectile seems to both have users that want symlink
>> >chasing and those that don't (looks like ClearCase users--but out of
>> >necessity not desire?).  As those concerns seem to be project dependent,
>> we
>> >could an option that is a list of paths or matchers to include/exclude
>> from
>> >chasing, and also a project root semaphore file or project config as I
>> >suggested in another message in this thread.
>> >
>> >I'd enable chasing as my default, opt out in a specific project should I
>> >ever have one that needs it rather than the other way around, and enjoy
>> >project-name and project-root uniformity.
>> >
>> >-Stephane
>>
>> Eli is right. You want either global path normalization or you don't.
>> Doing it peacemeal at the project level might satisfy some small immediate
>> need, but it won't solve the whole problem and will inevitably cause
>> downstream problems due to the incoherence.
>>
>> Also, it's just not universally true that I want to regard the "real"
>> version of something as its realpath. People use symlinks differently. In a
>> world in which we had ubiquitous unprivileged bind mounts *and* symlinks,
>> the realpath as identity concept might be more powerful, but we don't.
>>
>
> Again...I'm not saying universally, I'm saying optionally, and preferably
> flexibly optional.
>

I just got bitten by project-buffers not reporting all the buffers
"strictly" related to the project because project.el thinks there are two
different projects.  I can't imagine that me and the colleagues of mine
that use convenience symlinks are the only ones to see these kinds of
inconsistencies.  Just thought I'd leave a concrete example in this thread.

I've turned my advice back on to force truename canonicalization for the
time being.

[Message part 2 (text/html, inline)]

Message #89 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 14:54:20 -0400

Ship Mints <shipmints <at> gmail.com> writes:

> On Tue, Mar 25, 2025 at 8:17 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
>
>> > From: Ship Mints <shipmints <at> gmail.com>
>> > Date: Mon, 24 Mar 2025 16:25:44 -0400
>> > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
>> >
>> > To concretely demonstrate the differences, the function project-name
>> will return different results for each
>> > project object based on buffers loaded from different paths, despite the
>> projects being equivalent.
>> >
>> > project-name is defined as:
>> >
>> > (file-name-nondirectory (directory-file-name (project-root project)))
>> >
>> > If the root directory is determined to be different, the objects return
>> different names (and different roots).
>>
>> Unless I'm misunderstanding what Daniel wrote, the above is actually a
>> feature from his POV.
>>
>
> Indeed, it might be, but I'm curious what tooling he's using that depends
> on project-root not being "absolute" so to speak, and relative to his
> ambient default-directory.

🧠.

If I have a project under ~/foo/bar/qux/blah and I want to symlink to it
from ~/blah, and I specifically find-file ~/blah/something.txt, then I
want my project to be ~/blah-related, not ~/foo/bar/qux/blah!
The difference matters for things like project-shell.

Isn't the feature you really want a hypothetical
find-file-resolves-symlinks flag that has find-file find the absolute
version of anything visited?  Some people sincerely want this behavior.
Then you'd always be "in" your truename directory in the first place
(modulo other entry points like dired, which you could similarly
modify).

For my use case, I'd actually like to tweak file-truename to report
anything under ~/foo/bar/qux/blah as being under ~/blah, but that's a
separate and low-priority subject.

Message #92 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Tue, 25 Mar 2025 14:58:23 -0400

[Message part 1 (text/plain, inline)]

On Tue, Mar 25, 2025 at 2:54 PM Daniel Colascione <dancol <at> dancol.org> wrote:

> Ship Mints <shipmints <at> gmail.com> writes:
>
> > On Tue, Mar 25, 2025 at 8:17 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
> >
> >> > From: Ship Mints <shipmints <at> gmail.com>
> >> > Date: Mon, 24 Mar 2025 16:25:44 -0400
> >> > Cc: dancol <at> dancol.org, 77122 <at> debbugs.gnu.org, dmitry <at> gutov.dev
> >> >
> >> > To concretely demonstrate the differences, the function project-name
> >> will return different results for each
> >> > project object based on buffers loaded from different paths, despite
> the
> >> projects being equivalent.
> >> >
> >> > project-name is defined as:
> >> >
> >> > (file-name-nondirectory (directory-file-name (project-root project)))
> >> >
> >> > If the root directory is determined to be different, the objects
> return
> >> different names (and different roots).
> >>
> >> Unless I'm misunderstanding what Daniel wrote, the above is actually a
> >> feature from his POV.
> >>
> >
> > Indeed, it might be, but I'm curious what tooling he's using that depends
> > on project-root not being "absolute" so to speak, and relative to his
> > ambient default-directory.
>
> 🧠.
>
> If I have a project under ~/foo/bar/qux/blah and I want to symlink to it
> from ~/blah, and I specifically find-file ~/blah/something.txt, then I
> want my project to be ~/blah-related, not ~/foo/bar/qux/blah!
> The difference matters for things like project-shell.
>
> Isn't the feature you really want a hypothetical
> find-file-resolves-symlinks flag that has find-file find the absolute
> version of anything visited?  Some people sincerely want this behavior.
> Then you'd always be "in" your truename directory in the first place
> (modulo other entry points like dired, which you could similarly
> modify).
>
> For my use case, I'd actually like to tweak file-truename to report
> anything under ~/foo/bar/qux/blah as being under ~/blah, but that's a
> separate and low-priority subject.
>

That, too, but this all started when I got sufficiently frustrated that
project-name doesn't match for both real and symlinked directory avenues.
Making project-root's match, solves a bunch of other things where
implementations of code, including project-buffers, rely on project-root.

[Message part 2 (text/html, inline)]

Message #95 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, dancol <at> dancol.org
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Wed, 26 Mar 2025 05:02:06 +0200

On 25/03/2025 12:54, Ship Mints wrote:

> Ochen funny. I'll submit a separate patch for that one day (sarcasm 
> doesn't work in email, sorry).

I was not making a joke. Wouldn't it be easier, for your purposes, if 
symlinks were followed right away?

> In the meantime, I maintain my view that project.el needs to report 
> uniform project names and roots for identical projects approached from 
> different places, even if optional.
> 
> I just took a look at projectile.el, which I'd never looked at before 
> because I prefer using/improving core features.  It has a longer user 
> history to see what they've experienced (and it looks like some of 
> project.el's approach is copied almost verbatim e.g., the implementation 
> of project-name).  projectile seems to both have users that want symlink 
> chasing and those that don't (looks like ClearCase users--but out of 
> necessity not desire?).

Does it have that decision configurable as well? Could you point to the 
user option, or whatever else it has for that purpose?

Message #98 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Ship Mints <shipmints <at> gmail.com>, Daniel Colascione <dancol <at> dancol.org>
Cc: 77122 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Wed, 26 Mar 2025 05:11:20 +0200

On 25/03/2025 17:42, Ship Mints wrote:
> I just got bitten by project-buffers not reporting all the buffers 
> "strictly" related to the project because project.el thinks there are 
> two different projects.  I can't imagine that me and the colleagues of 
> mine that use convenience symlinks are the only ones to see these kinds 
> of inconsistencies.  Just thought I'd leave a concrete example in this 
> thread.

Thanks for the example.

> I've turned my advice back on to force truename canonicalization for the 
> time being.

Does that change really help?

In my testing, what it results in, is showing only the buffers belonging 
to the "canonical" project, both when project-switch-buffer is called 
when in it, or when inside a "symlinked" version. Buffers visited from 
the "symlinked" project are not suggested in completion anymore.

When reproducing, try using find-file (C-x C-f) to open the buffers.

Message #101 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 77122 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Daniel Colascione <dancol <at> dancol.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Wed, 26 Mar 2025 07:48:41 -0400

[Message part 1 (text/plain, inline)]

On Tue, Mar 25, 2025 at 11:11 PM Dmitry Gutov <dmitry <at> gutov.dev> wrote:

> On 25/03/2025 17:42, Ship Mints wrote:
> > I just got bitten by project-buffers not reporting all the buffers
> > "strictly" related to the project because project.el thinks there are
> > two different projects.  I can't imagine that me and the colleagues of
> > mine that use convenience symlinks are the only ones to see these kinds
> > of inconsistencies.  Just thought I'd leave a concrete example in this
> > thread.
>
> Thanks for the example.
>
> > I've turned my advice back on to force truename canonicalization for the
> > time being.
>
> Does that change really help?
>

It does for unifying project-name and project-root.

In my testing, what it results in, is showing only the buffers belonging
> to the "canonical" project, both when project-switch-buffer is called
> when in it, or when inside a "symlinked" version. Buffers visited from
> the "symlinked" project are not suggested in completion anymore.
>
> When reproducing, try using find-file (C-x C-f) to open the buffers.
>

Yes, indeed, project-buffers also needs some help.

Thanks for the continuing discussion.

WRT projectile, having only eyeballed the code (I haven't used it), it
seems truename use is not optional.  It looks like
projectile-project-buffer-p is defined in terms of truename
https://github.com/bbatsov/projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1775,
project-root
caches the truename (but I see a bug in the code which doesn't normalize
dir on the way in so users likely get odd results)
https://github.com/bbatsov/projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385,
the file cache is in terms of truename
https://github.com/bbatsov/projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1262
.

[Message part 2 (text/html, inline)]

Message #104 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Daniel Colascione <dancol <at> dancol.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Wed, 26 Mar 2025 15:00:16 +0200

On 26/03/2025 13:48, Ship Mints wrote:
> On Tue, Mar 25, 2025 at 11:11 PM Dmitry Gutov <dmitry <at> gutov.dev 
> <mailto:dmitry <at> gutov.dev>> wrote:
> 
>     On 25/03/2025 17:42, Ship Mints wrote:
>      > I just got bitten by project-buffers not reporting all the buffers
>      > "strictly" related to the project because project.el thinks there
>     are
>      > two different projects.  I can't imagine that me and the
>     colleagues of
>      > mine that use convenience symlinks are the only ones to see these
>     kinds
>      > of inconsistencies.  Just thought I'd leave a concrete example in
>     this
>      > thread.
> 
>     Thanks for the example.
> 
>      > I've turned my advice back on to force truename canonicalization
>     for the
>      > time being.
> 
>     Does that change really help?
> 
> 
> It does for unifying project-name and project-root.

Do they have corresponding scenarios that cause confusion?

>     In my testing, what it results in, is showing only the buffers
>     belonging
>     to the "canonical" project, both when project-switch-buffer is called
>     when in it, or when inside a "symlinked" version. Buffers visited from
>     the "symlinked" project are not suggested in completion anymore.
> 
>     When reproducing, try using find-file (C-x C-f) to open the buffers.
> 
> 
> Yes, indeed, project-buffers also needs some help.

We could add a user option for project-buffers specifically, as one 
approach.

> Thanks for the continuing discussion.
> 
> WRT projectile, having only eyeballed the code (I haven't used it), it 
> seems truename use is not optional.  It looks like projectile-project- 
> buffer-p is defined in terms of truename https://github.com/bbatsov/ 
> projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/ 
> projectile.el#L1775 <https://github.com/bbatsov/projectile/ 
> blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/ 
> projectile.el#L1775>, project-root caches the truename (but I see a bug 
> in the code which doesn't normalize dir on the way in so users likely 
> get odd results) https://github.com/bbatsov/projectile/ 
> blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385 
> <https://github.com/bbatsov/projectile/ 
> blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385>, the 
> file cache is in terms of truename https://github.com/bbatsov/ 
> projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/ 
> projectile.el#L1262 <https://github.com/bbatsov/projectile/ 
> blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1262>. </>

I wonder how it affects Projectile's performance over Tramp.

Message #107 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 77122 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Daniel Colascione <dancol <at> dancol.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Wed, 26 Mar 2025 11:52:26 -0400

[Message part 1 (text/plain, inline)]

On Wed, Mar 26, 2025 at 9:00 AM Dmitry Gutov <dmitry <at> gutov.dev> wrote:

> On 26/03/2025 13:48, Ship Mints wrote:
> > On Tue, Mar 25, 2025 at 11:11 PM Dmitry Gutov <dmitry <at> gutov.dev
> > <mailto:dmitry <at> gutov.dev>> wrote:
> >
> >     On 25/03/2025 17:42, Ship Mints wrote:
> >      > I just got bitten by project-buffers not reporting all the buffers
> >      > "strictly" related to the project because project.el thinks there
> >     are
> >      > two different projects.  I can't imagine that me and the
> >     colleagues of
> >      > mine that use convenience symlinks are the only ones to see these
> >     kinds
> >      > of inconsistencies.  Just thought I'd leave a concrete example in
> >     this
> >      > thread.
> >
> >     Thanks for the example.
> >
> >      > I've turned my advice back on to force truename canonicalization
> >     for the
> >      > time being.
> >
> >     Does that change really help?
> >
> >
> > It does for unifying project-name and project-root.
>
> Do they have corresponding scenarios that cause confusion?
>
> >     In my testing, what it results in, is showing only the buffers
> >     belonging
> >     to the "canonical" project, both when project-switch-buffer is called
> >     when in it, or when inside a "symlinked" version. Buffers visited
> from
> >     the "symlinked" project are not suggested in completion anymore.
> >
> >     When reproducing, try using find-file (C-x C-f) to open the buffers.
> >
> >
> > Yes, indeed, project-buffers also needs some help.
>
> We could add a user option for project-buffers specifically, as one
> approach.
>

I assume this would be the same option that canonicalizes project-roots so
it's all synchronized.

> Thanks for the continuing discussion.
> >
> > WRT projectile, having only eyeballed the code (I haven't used it), it
> > seems truename use is not optional.  It looks like projectile-project-
> > buffer-p is defined in terms of truename https://github.com/bbatsov/
> > projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
> > projectile.el#L1775 <https://github.com/bbatsov/projectile/
> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
> > projectile.el#L1775>, project-root caches the truename (but I see a bug
> > in the code which doesn't normalize dir on the way in so users likely
> > get odd results) https://github.com/bbatsov/projectile/
> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385
> > <https://github.com/bbatsov/projectile/
> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385>, the
> > file cache is in terms of truename https://github.com/bbatsov/
> > projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
> > projectile.el#L1262 <https://github.com/bbatsov/projectile/
> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1262>. </>
>
> I wonder how it affects Projectile's performance over Tramp.
>

No idea.  But I can say that I have wrapped a bunch of project.el functions
to avoid remote file operations based on a personal user option (some of my
users are heavy Trampers, some hosts are slow).  But the way I've done it
is a blanket inhibition, not project-specific or some other appropriate
criteria which would be nicer.  Perhaps a .project.eld file with variables
akin to dir-locals that project could consult.  I haven't had the time to
think that through.

I've also implemented personal 'consult' project and related
buffer-matching functions to handle project root canonicalization.  Those
functions are not in terms of project-buffers and perhaps should be as they
miss out on back-end specializations.  I've started a discussion in the
consult repo to offer to contribute PRs over there.

[Message part 2 (text/html, inline)]

Message #110 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org, Dmitry Gutov <dmitry <at> gutov.dev>,
 Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Wed, 26 Mar 2025 20:28:39 -0400

Ship Mints <shipmints <at> gmail.com> writes:

> On Wed, Mar 26, 2025 at 9:00 AM Dmitry Gutov <dmitry <at> gutov.dev> wrote:
>
>> On 26/03/2025 13:48, Ship Mints wrote:
>> > On Tue, Mar 25, 2025 at 11:11 PM Dmitry Gutov <dmitry <at> gutov.dev
>> > <mailto:dmitry <at> gutov.dev>> wrote:
>> >
>> >     On 25/03/2025 17:42, Ship Mints wrote:
>> >      > I just got bitten by project-buffers not reporting all the buffers
>> >      > "strictly" related to the project because project.el thinks there
>> >     are
>> >      > two different projects.  I can't imagine that me and the
>> >     colleagues of
>> >      > mine that use convenience symlinks are the only ones to see these
>> >     kinds
>> >      > of inconsistencies.  Just thought I'd leave a concrete example in
>> >     this
>> >      > thread.
>> >
>> >     Thanks for the example.
>> >
>> >      > I've turned my advice back on to force truename canonicalization
>> >     for the
>> >      > time being.
>> >
>> >     Does that change really help?
>> >
>> >
>> > It does for unifying project-name and project-root.
>>
>> Do they have corresponding scenarios that cause confusion?
>>
>> >     In my testing, what it results in, is showing only the buffers
>> >     belonging
>> >     to the "canonical" project, both when project-switch-buffer is called
>> >     when in it, or when inside a "symlinked" version. Buffers visited
>> from
>> >     the "symlinked" project are not suggested in completion anymore.
>> >
>> >     When reproducing, try using find-file (C-x C-f) to open the buffers.
>> >
>> >
>> > Yes, indeed, project-buffers also needs some help.
>>
>> We could add a user option for project-buffers specifically, as one
>> approach.
>>
>
> I assume this would be the same option that canonicalizes project-roots so
> it's all synchronized.
>
>> Thanks for the continuing discussion.
>> >
>> > WRT projectile, having only eyeballed the code (I haven't used it), it
>> > seems truename use is not optional.  It looks like projectile-project-
>> > buffer-p is defined in terms of truename https://github.com/bbatsov/
>> > projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
>> > projectile.el#L1775 <https://github.com/bbatsov/projectile/
>> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
>> > projectile.el#L1775>, project-root caches the truename (but I see a bug
>> > in the code which doesn't normalize dir on the way in so users likely
>> > get odd results) https://github.com/bbatsov/projectile/
>> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385
>> > <https://github.com/bbatsov/projectile/
>> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385>, the
>> > file cache is in terms of truename https://github.com/bbatsov/
>> > projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
>> > projectile.el#L1262 <https://github.com/bbatsov/projectile/
>> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1262>. </>
>>
>> I wonder how it affects Projectile's performance over Tramp.
>>
>
> No idea.  But I can say that I have wrapped a bunch of project.el functions
> to avoid remote file operations based on a personal user option (some of my
> users are heavy Trampers, some hosts are slow).  But the way I've done it
> is a blanket inhibition, not project-specific or some other appropriate
> criteria which would be nicer.  Perhaps a .project.eld file with variables
> akin to dir-locals that project could consult.  I haven't had the time to
> think that through.
>
> I've also implemented personal 'consult' project and related
> buffer-matching functions to handle project root canonicalization.  Those
> functions are not in terms of project-buffers and perhaps should be as they
> miss out on back-end specializations.  I've started a discussion in the
> consult repo to offer to contribute PRs over there.

I'm not unsympathetic to the need, but I think we need to understand
whether your request is project-specific or not.  I realized that the
Emacs features I wanted already exist.  In particular,
find-file-visit-truename should excise symlinks from every visited path,
and directory-abbrev-alist should be able to alias one directory
to another.  Can you add your symlink directory to
directory-abbrev-alist and have it rewritten to the truename?

Message #113 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: 77122 <at> debbugs.gnu.org, Dmitry Gutov <dmitry <at> gutov.dev>,
 Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Thu, 27 Mar 2025 10:41:22 -0400

[Message part 1 (text/plain, inline)]

On Wed, Mar 26, 2025 at 8:28 PM Daniel Colascione <dancol <at> dancol.org> wrote:

> Ship Mints <shipmints <at> gmail.com> writes:
>
> > On Wed, Mar 26, 2025 at 9:00 AM Dmitry Gutov <dmitry <at> gutov.dev> wrote:
> >
> >> On 26/03/2025 13:48, Ship Mints wrote:
> >> > On Tue, Mar 25, 2025 at 11:11 PM Dmitry Gutov <dmitry <at> gutov.dev
> >> > <mailto:dmitry <at> gutov.dev>> wrote:
> >> >
> >> >     On 25/03/2025 17:42, Ship Mints wrote:
> >> >      > I just got bitten by project-buffers not reporting all the
> buffers
> >> >      > "strictly" related to the project because project.el thinks
> there
> >> >     are
> >> >      > two different projects.  I can't imagine that me and the
> >> >     colleagues of
> >> >      > mine that use convenience symlinks are the only ones to see
> these
> >> >     kinds
> >> >      > of inconsistencies.  Just thought I'd leave a concrete example
> in
> >> >     this
> >> >      > thread.
> >> >
> >> >     Thanks for the example.
> >> >
> >> >      > I've turned my advice back on to force truename
> canonicalization
> >> >     for the
> >> >      > time being.
> >> >
> >> >     Does that change really help?
> >> >
> >> >
> >> > It does for unifying project-name and project-root.
> >>
> >> Do they have corresponding scenarios that cause confusion?
> >>
> >> >     In my testing, what it results in, is showing only the buffers
> >> >     belonging
> >> >     to the "canonical" project, both when project-switch-buffer is
> called
> >> >     when in it, or when inside a "symlinked" version. Buffers visited
> >> from
> >> >     the "symlinked" project are not suggested in completion anymore.
> >> >
> >> >     When reproducing, try using find-file (C-x C-f) to open the
> buffers.
> >> >
> >> >
> >> > Yes, indeed, project-buffers also needs some help.
> >>
> >> We could add a user option for project-buffers specifically, as one
> >> approach.
> >>
> >
> > I assume this would be the same option that canonicalizes project-roots
> so
> > it's all synchronized.
> >
> >> Thanks for the continuing discussion.
> >> >
> >> > WRT projectile, having only eyeballed the code (I haven't used it), it
> >> > seems truename use is not optional.  It looks like projectile-project-
> >> > buffer-p is defined in terms of truename https://github.com/bbatsov/
> >> > projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
> >> > projectile.el#L1775 <https://github.com/bbatsov/projectile/
> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
> >> > projectile.el#L1775>, project-root caches the truename (but I see a
> bug
> >> > in the code which doesn't normalize dir on the way in so users likely
> >> > get odd results) https://github.com/bbatsov/projectile/
> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385
> >> > <https://github.com/bbatsov/projectile/
> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385>,
> the
> >> > file cache is in terms of truename https://github.com/bbatsov/
> >> > projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
> >> > projectile.el#L1262 <https://github.com/bbatsov/projectile/
> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1262>.
> </>
> >>
> >> I wonder how it affects Projectile's performance over Tramp.
> >>
> >
> > No idea.  But I can say that I have wrapped a bunch of project.el
> functions
> > to avoid remote file operations based on a personal user option (some of
> my
> > users are heavy Trampers, some hosts are slow).  But the way I've done it
> > is a blanket inhibition, not project-specific or some other appropriate
> > criteria which would be nicer.  Perhaps a .project.eld file with
> variables
> > akin to dir-locals that project could consult.  I haven't had the time to
> > think that through.
> >
> > I've also implemented personal 'consult' project and related
> > buffer-matching functions to handle project root canonicalization.  Those
> > functions are not in terms of project-buffers and perhaps should be as
> they
> > miss out on back-end specializations.  I've started a discussion in the
> > consult repo to offer to contribute PRs over there.
>
> I'm not unsympathetic to the need, but I think we need to understand
> whether your request is project-specific or not.  I realized that the
> Emacs features I wanted already exist.  In particular,
> find-file-visit-truename should excise symlinks from every visited path,
> and directory-abbrev-alist should be able to alias one directory
> to another.  Can you add your symlink directory to
> directory-abbrev-alist and have it rewritten to the truename?
>

Of course people can always hack around these cases manually.

To me, what makes project.el special is that it has an implicit promise to
organize buffers related to a project.  That the existing implementation
depends on the ambient directory from which a file buffer was created to
determine what's related has become a "leaky abstraction" where symlinks to
projects are involved.  The documentation in project.el kind of alludes to
this limitation in an implied way for transient projects.  It is silent on
directory assumptions for vc projects, instead delegating to the vc back
end.

It would benefit the whole community to offer a resolution to let
project.el fulfil its buffer organization mission independent of
convenience names and that doesn't rely on management of
directory-abbrev-alist (eek).

[Message part 2 (text/html, inline)]

Message #116 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: 77122 <at> debbugs.gnu.org, Dmitry Gutov <dmitry <at> gutov.dev>,
 Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Thu, 27 Mar 2025 11:22:22 -0400

On March 27, 2025 10:41:22 AM EDT, Ship Mints <shipmints <at> gmail.com> wrote:
>On Wed, Mar 26, 2025 at 8:28 PM Daniel Colascione <dancol <at> dancol.org> wrote:
>
>> Ship Mints <shipmints <at> gmail.com> writes:
>>
>> > On Wed, Mar 26, 2025 at 9:00 AM Dmitry Gutov <dmitry <at> gutov.dev> wrote:
>> >
>> >> On 26/03/2025 13:48, Ship Mints wrote:
>> >> > On Tue, Mar 25, 2025 at 11:11 PM Dmitry Gutov <dmitry <at> gutov.dev
>> >> > <mailto:dmitry <at> gutov.dev>> wrote:
>> >> >
>> >> >     On 25/03/2025 17:42, Ship Mints wrote:
>> >> >      > I just got bitten by project-buffers not reporting all the
>> buffers
>> >> >      > "strictly" related to the project because project.el thinks
>> there
>> >> >     are
>> >> >      > two different projects.  I can't imagine that me and the
>> >> >     colleagues of
>> >> >      > mine that use convenience symlinks are the only ones to see
>> these
>> >> >     kinds
>> >> >      > of inconsistencies.  Just thought I'd leave a concrete example
>> in
>> >> >     this
>> >> >      > thread.
>> >> >
>> >> >     Thanks for the example.
>> >> >
>> >> >      > I've turned my advice back on to force truename
>> canonicalization
>> >> >     for the
>> >> >      > time being.
>> >> >
>> >> >     Does that change really help?
>> >> >
>> >> >
>> >> > It does for unifying project-name and project-root.
>> >>
>> >> Do they have corresponding scenarios that cause confusion?
>> >>
>> >> >     In my testing, what it results in, is showing only the buffers
>> >> >     belonging
>> >> >     to the "canonical" project, both when project-switch-buffer is
>> called
>> >> >     when in it, or when inside a "symlinked" version. Buffers visited
>> >> from
>> >> >     the "symlinked" project are not suggested in completion anymore.
>> >> >
>> >> >     When reproducing, try using find-file (C-x C-f) to open the
>> buffers.
>> >> >
>> >> >
>> >> > Yes, indeed, project-buffers also needs some help.
>> >>
>> >> We could add a user option for project-buffers specifically, as one
>> >> approach.
>> >>
>> >
>> > I assume this would be the same option that canonicalizes project-roots
>> so
>> > it's all synchronized.
>> >
>> >> Thanks for the continuing discussion.
>> >> >
>> >> > WRT projectile, having only eyeballed the code (I haven't used it), it
>> >> > seems truename use is not optional.  It looks like projectile-project-
>> >> > buffer-p is defined in terms of truename https://github.com/bbatsov/
>> >> > projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
>> >> > projectile.el#L1775 <https://github.com/bbatsov/projectile/
>> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
>> >> > projectile.el#L1775>, project-root caches the truename (but I see a
>> bug
>> >> > in the code which doesn't normalize dir on the way in so users likely
>> >> > get odd results) https://github.com/bbatsov/projectile/
>> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385
>> >> > <https://github.com/bbatsov/projectile/
>> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385>,
>> the
>> >> > file cache is in terms of truename https://github.com/bbatsov/
>> >> > projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
>> >> > projectile.el#L1262 <https://github.com/bbatsov/projectile/
>> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1262>.
>> </>
>> >>
>> >> I wonder how it affects Projectile's performance over Tramp.
>> >>
>> >
>> > No idea.  But I can say that I have wrapped a bunch of project.el
>> functions
>> > to avoid remote file operations based on a personal user option (some of
>> my
>> > users are heavy Trampers, some hosts are slow).  But the way I've done it
>> > is a blanket inhibition, not project-specific or some other appropriate
>> > criteria which would be nicer.  Perhaps a .project.eld file with
>> variables
>> > akin to dir-locals that project could consult.  I haven't had the time to
>> > think that through.
>> >
>> > I've also implemented personal 'consult' project and related
>> > buffer-matching functions to handle project root canonicalization.  Those
>> > functions are not in terms of project-buffers and perhaps should be as
>> they
>> > miss out on back-end specializations.  I've started a discussion in the
>> > consult repo to offer to contribute PRs over there.
>>
>> I'm not unsympathetic to the need, but I think we need to understand
>> whether your request is project-specific or not.  I realized that the
>> Emacs features I wanted already exist.  In particular,
>> find-file-visit-truename should excise symlinks from every visited path,
>> and directory-abbrev-alist should be able to alias one directory
>> to another.  Can you add your symlink directory to
>> directory-abbrev-alist and have it rewritten to the truename?
>>
>
>Of course people can always hack around these cases manually.
>
>To me, what makes project.el special is that it has an implicit promise to
>organize buffers related to a project.  That the existing implementation
>depends on the ambient directory from which a file buffer was created to
>determine what's related has become a "leaky abstraction" where symlinks to
>projects are involved.  The documentation in project.el kind of alludes to
>this limitation in an implied way for transient projects.  It is silent on
>directory assumptions for vc projects, instead delegating to the vc back
>end.
>
>It would benefit the whole community to offer a resolution to let
>project.el fulfil its buffer organization mission independent of
>convenience names and that doesn't rely on management of
>directory-abbrev-alist (eek).


I hear you, but I'm not sure the situation is so clear cut. Of course we want a single project in the user's mind to map to one process object on the Emacs heap: there's just genuine diversity in opinions of what counts as "the same" and I think project is the wrong level of abstraction for expressing your desired unification. We already have a way for the use to express a preference that the fully resolved name of a file be considered its identity, and we already have a mechanism through which the user can inform Emacs of the canonical version of an ambiguous path.

Maybe we can improve the UX. When we find ourselves with two project objects (say A and B) that refer to the same true path, we can ask the user to choose between making A canonical, making B canonical, and rejecting the unification --- then remember that choice in the existing mechanism. That'll fix the non-project path ambiguity problems too.

Message #119 received at 77122 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: 77122 <at> debbugs.gnu.org, Dmitry Gutov <dmitry <at> gutov.dev>,
 Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#77122: [PATCH] project--find-in-directory resolves symlinks
Date: Thu, 27 Mar 2025 12:04:25 -0400

[Message part 1 (text/plain, inline)]

On Thu, Mar 27, 2025 at 11:22 AM Daniel Colascione <dancol <at> dancol.org>
wrote:

>
>
> On March 27, 2025 10:41:22 AM EDT, Ship Mints <shipmints <at> gmail.com> wrote:
> >On Wed, Mar 26, 2025 at 8:28 PM Daniel Colascione <dancol <at> dancol.org>
> wrote:
> >
> >> Ship Mints <shipmints <at> gmail.com> writes:
> >>
> >> > On Wed, Mar 26, 2025 at 9:00 AM Dmitry Gutov <dmitry <at> gutov.dev>
> wrote:
> >> >
> >> >> On 26/03/2025 13:48, Ship Mints wrote:
> >> >> > On Tue, Mar 25, 2025 at 11:11 PM Dmitry Gutov <dmitry <at> gutov.dev
> >> >> > <mailto:dmitry <at> gutov.dev>> wrote:
> >> >> >
> >> >> >     On 25/03/2025 17:42, Ship Mints wrote:
> >> >> >      > I just got bitten by project-buffers not reporting all the
> >> buffers
> >> >> >      > "strictly" related to the project because project.el thinks
> >> there
> >> >> >     are
> >> >> >      > two different projects.  I can't imagine that me and the
> >> >> >     colleagues of
> >> >> >      > mine that use convenience symlinks are the only ones to see
> >> these
> >> >> >     kinds
> >> >> >      > of inconsistencies.  Just thought I'd leave a concrete
> example
> >> in
> >> >> >     this
> >> >> >      > thread.
> >> >> >
> >> >> >     Thanks for the example.
> >> >> >
> >> >> >      > I've turned my advice back on to force truename
> >> canonicalization
> >> >> >     for the
> >> >> >      > time being.
> >> >> >
> >> >> >     Does that change really help?
> >> >> >
> >> >> >
> >> >> > It does for unifying project-name and project-root.
> >> >>
> >> >> Do they have corresponding scenarios that cause confusion?
> >> >>
> >> >> >     In my testing, what it results in, is showing only the buffers
> >> >> >     belonging
> >> >> >     to the "canonical" project, both when project-switch-buffer is
> >> called
> >> >> >     when in it, or when inside a "symlinked" version. Buffers
> visited
> >> >> from
> >> >> >     the "symlinked" project are not suggested in completion
> anymore.
> >> >> >
> >> >> >     When reproducing, try using find-file (C-x C-f) to open the
> >> buffers.
> >> >> >
> >> >> >
> >> >> > Yes, indeed, project-buffers also needs some help.
> >> >>
> >> >> We could add a user option for project-buffers specifically, as one
> >> >> approach.
> >> >>
> >> >
> >> > I assume this would be the same option that canonicalizes
> project-roots
> >> so
> >> > it's all synchronized.
> >> >
> >> >> Thanks for the continuing discussion.
> >> >> >
> >> >> > WRT projectile, having only eyeballed the code (I haven't used
> it), it
> >> >> > seems truename use is not optional.  It looks like
> projectile-project-
> >> >> > buffer-p is defined in terms of truename
> https://github.com/bbatsov/
> >> >> > projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
> >> >> > projectile.el#L1775 <https://github.com/bbatsov/projectile/
> >> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
> >> >> > projectile.el#L1775>, project-root caches the truename (but I see a
> >> bug
> >> >> > in the code which doesn't normalize dir on the way in so users
> likely
> >> >> > get odd results) https://github.com/bbatsov/projectile/
> >> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385
> >> >> > <https://github.com/bbatsov/projectile/
> >> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1385>,
> >> the
> >> >> > file cache is in terms of truename https://github.com/bbatsov/
> >> >> > projectile/blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/
> >> >> > projectile.el#L1262 <https://github.com/bbatsov/projectile/
> >> >> > blob/55db082cdf7b849335ccf24b7ba5aa2607d6fe93/projectile.el#L1262>.
> >> </>
> >> >>
> >> >> I wonder how it affects Projectile's performance over Tramp.
> >> >>
> >> >
> >> > No idea.  But I can say that I have wrapped a bunch of project.el
> >> functions
> >> > to avoid remote file operations based on a personal user option (some
> of
> >> my
> >> > users are heavy Trampers, some hosts are slow).  But the way I've
> done it
> >> > is a blanket inhibition, not project-specific or some other
> appropriate
> >> > criteria which would be nicer.  Perhaps a .project.eld file with
> >> variables
> >> > akin to dir-locals that project could consult.  I haven't had the
> time to
> >> > think that through.
> >> >
> >> > I've also implemented personal 'consult' project and related
> >> > buffer-matching functions to handle project root canonicalization.
> Those
> >> > functions are not in terms of project-buffers and perhaps should be as
> >> they
> >> > miss out on back-end specializations.  I've started a discussion in
> the
> >> > consult repo to offer to contribute PRs over there.
> >>
> >> I'm not unsympathetic to the need, but I think we need to understand
> >> whether your request is project-specific or not.  I realized that the
> >> Emacs features I wanted already exist.  In particular,
> >> find-file-visit-truename should excise symlinks from every visited path,
> >> and directory-abbrev-alist should be able to alias one directory
> >> to another.  Can you add your symlink directory to
> >> directory-abbrev-alist and have it rewritten to the truename?
> >>
> >
> >Of course people can always hack around these cases manually.
> >
> >To me, what makes project.el special is that it has an implicit promise to
> >organize buffers related to a project.  That the existing implementation
> >depends on the ambient directory from which a file buffer was created to
> >determine what's related has become a "leaky abstraction" where symlinks
> to
> >projects are involved.  The documentation in project.el kind of alludes to
> >this limitation in an implied way for transient projects.  It is silent on
> >directory assumptions for vc projects, instead delegating to the vc back
> >end.
> >
> >It would benefit the whole community to offer a resolution to let
> >project.el fulfil its buffer organization mission independent of
> >convenience names and that doesn't rely on management of
> >directory-abbrev-alist (eek).
>
>
> I hear you, but I'm not sure the situation is so clear cut. Of course we
> want a single project in the user's mind to map to one process object on
> the Emacs heap: there's just genuine diversity in opinions of what counts
> as "the same" and I think project is the wrong level of abstraction for
> expressing your desired unification. We already have a way for the use to
> express a preference that the fully resolved name of a file be considered
> its identity, and we already have a mechanism through which the user can
> inform Emacs of the canonical version of an ambiguous path.
>
> Maybe we can improve the UX. When we find ourselves with two project
> objects (say A and B) that refer to the same true path, we can ask the user
> to choose between making A canonical, making B canonical, and rejecting the
> unification --- then remember that choice in the existing mechanism.
> That'll fix the non-project path ambiguity problems too.


I don't think it matters to me which A or B (or C or D) becomes the
"master" but I'd still prefer the master to be rooted in the true path.
The one that gets recorded in projects.eld, I think should be the master
and if we have a user option where the true name always becomes master,
that's cool.

If we're willing to prompt, then we'd be willing to have a user option to
default to a selection and not prompt.  That would be fine with me.

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.