GNU bug report logs - #19636
[TRAMP] global minor mode hangs connection when accessing files in :lighter

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19636 in the body.
You can then email your comments to 19636 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philippe Vaucher <philippe.vaucher <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [TRAMP] global minor mode hangs connection when accessing files in
 :lighter
Date: Tue, 20 Jan 2015 18:49:31 +0100

[Message part 1 (text/plain, inline)]

Hello,

TRAMP hangs the connection for any new processes in both emacs 24.4
and emacs from the master branch when a global minor mode uses a
:lighter which evals `(file-truename default-directory)`.

To reproduce:

M-x find-file buggy-tramp-mode.el
M-x eval-buffer
M-x global-buggy-tramp-mode
M-x find-file /scpx:user <at> host:/tmp/foo.txt
M-x async-shell-command ls

The last command (new process) doens't complete and the TRAMP buffer
shows "Are you awake?". My understanding of the problem is that
`file-truename` tries to use a not-yet-ready TRAMP connection. To work
around that was tried is to use `tramp-connectable-p` or even
`file-remote-p` with the appropriate flags, but they both
(incorrectly?) return true.

The "real world" issue that it affects is
https://github.com/bbatsov/projectile/issues/523

Please ask if I'm unclear or you need more clarifications.

Thanks,
Philippe

[buggy-tramp-mode.el (text/x-emacs-lisp, attachment)]

Message #8 received at 19636 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Philippe Vaucher <philippe.vaucher <at> gmail.com>
Cc: 19636 <at> debbugs.gnu.org
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Wed, 21 Jan 2015 17:15:14 +0100

Philippe Vaucher <philippe.vaucher <at> gmail.com> writes:

> Hello,

Hi Philippe,

> TRAMP hangs the connection for any new processes in both emacs 24.4
> and emacs from the master branch when a global minor mode uses a
> :lighter which evals `(file-truename default-directory)`.
>
> To reproduce:
>
> M-x find-file buggy-tramp-mode.el
> M-x eval-buffer
> M-x global-buggy-tramp-mode
> M-x find-file /scpx:user <at> host:/tmp/foo.txt
> M-x async-shell-command ls
>
> The last command (new process) doens't complete and the TRAMP buffer
> shows "Are you awake?". My understanding of the problem is that
> `file-truename` tries to use a not-yet-ready TRAMP connection.

I tried it seriously with several combinations of Tramp parameter
settings; I couldn't reproduce. However, it is dangerous to run
an asynchronous process in parallel to retrieve information via
Tramp. While you run an asynchronous process, Tramp opens a
second connection and remembers this channel. In parallel,
file-truename needs also to retrieve information fron the remote
machine, on the other connection channel. Likely, Tramp might
confuse which connection channel to use.

Maybe you can set tramp-verbose to 6, and rerun your test. The resulting
Tramp trace buffer shall tell us what's up.

> To work around that was tried is to use `tramp-connectable-p` or even
> `file-remote-p` with the appropriate flags, but they both
> (incorrectly?) return true.

(setq buggy-tramp-mode-lighter
      '(:eval (format " Projectile[%s]"
		      (if (file-remote-p default-directory)
			  default-directory
			(file-truename default-directory)))))

> Thanks,
> Philippe

Best regards, Michael.

Message #11 received at 19636 <at> debbugs.gnu.org (full text, mbox):

From: Philippe Vaucher <philippe.vaucher <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 19636 <at> debbugs.gnu.org
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Wed, 21 Jan 2015 18:40:40 +0100

[Message part 1 (text/plain, inline)]

>> The last command (new process) doens't complete and the TRAMP buffer
>> shows "Are you awake?". My understanding of the problem is that
>> `file-truename` tries to use a not-yet-ready TRAMP connection.
>
> I tried it seriously with several combinations of Tramp parameter
> settings; I couldn't reproduce. However, it is dangerous to run
> an asynchronous process in parallel to retrieve information via
> Tramp. While you run an asynchronous process, Tramp opens a
> second connection and remembers this channel. In parallel,
> file-truename needs also to retrieve information fron the remote
> machine, on the other connection channel. Likely, Tramp might
> confuse which connection channel to use.

Thank you for your answer.

Yes, it would be nice to be able to avoid that. The real world problem
where this happens is for `projectile-mode`, a global minor mode that
tries to detect wether you are in a project for every buffers. The
final solution will probably involve avoiding detection in buffers
where there's a process, even tho it'd make sense for the mode to work
in these buffers as well. Basically the mode displays in which project
you're in and sets up keybindings for naviguating files in this
project, hence the use of :lighter in the mode, which checks for
project root files (e.g .git/Gemfile/etc) in `default-directory` (and
parents) for the current buffer in order to determinate the project
name.


> Maybe you can set tramp-verbose to 6, and rerun your test. The resulting
> Tramp trace buffer shall tell us what's up.

I attached the TRAMP debug buffer (debug-tramp.txt) of the updated
version of my testcase. You can see that at 18:06:15.613967 it says
"are you awake" after 10 seconds of waiting. It then tries to reopen
the connection and I have to `C-g` furiously in order to recover from
it. I run my testcase in the terminal like so:

emacs -Q -l buggy-tramp-mode.el

I ran my tests on the master branch at f769566, then I updated the git
repo to 41efcf4, recompiled and ran the tests again and it failed in
the same way. The commits related to TRAMP in that range (git log
--oneline f769566..41efcf4 -- lisp/net) don't seem to be relevant, but
if you want I can give you the updated TRAMP debug buffer log.


>> To work around that was tried is to use `tramp-connectable-p` or even
>> `file-remote-p` with the appropriate flags, but they both
>> (incorrectly?) return true.
>
> (setq buggy-tramp-mode-lighter
>       '(:eval (format " Projectile[%s]"
>                       (if (file-remote-p default-directory)
>                           default-directory
>                         (file-truename default-directory)))))

That's an interesting workaround, can I assume that
`default-directory` over a TRAMP connection is always absolute? what
about symlinks?

Philippe

[debug-tramp.txt (text/plain, attachment)]

[buggy-tramp-mode.el (text/x-emacs-lisp, attachment)]

Message #14 received at 19636 <at> debbugs.gnu.org (full text, mbox):

From: Philippe Vaucher <philippe.vaucher <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 19636 <19636 <at> debbugs.gnu.org>
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Thu, 22 Jan 2015 12:07:46 +0100

> Yes, it would be nice to be able to avoid that. The real world problem
> where this happens is for `projectile-mode`, a global minor mode that
> tries to detect wether you are in a project for every buffers. The
> final solution will probably involve avoiding detection in buffers
> where there's a process, even tho it'd make sense for the mode to work
> in these buffers as well. Basically the mode displays in which project
> you're in and sets up keybindings for naviguating files in this
> project, hence the use of :lighter in the mode, which checks for
> project root files (e.g .git/Gemfile/etc) in `default-directory` (and
> parents) for the current buffer in order to determinate the project
> name.

Just for clarifications, the new processes involved where this bug
happens is when you invoke "git fetch" when using magit or when doing
things like "M-& service nginx restart" over TRAMP.

Philippe

Message #17 received at 19636 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Philippe Vaucher <philippe.vaucher <at> gmail.com>
Cc: 19636 <at> debbugs.gnu.org
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Sun, 25 Jan 2015 20:40:42 +0100

Philippe Vaucher <philippe.vaucher <at> gmail.com> writes:

>> (setq buggy-tramp-mode-lighter
>>       '(:eval (format " Projectile[%s]"
>>                       (if (file-remote-p default-directory)
>>                           default-directory
>>                         (file-truename default-directory)))))
>
> That's an interesting workaround, can I assume that
> `default-directory` over a TRAMP connection is always absolute? what
> about symlinks?

You can always assume, that `default-directory' is absolute, being it
local or remote. Symlinks are not expanded, 'tho.

However, there is always the buffer local variable
`buffer-file-truename'. Maybe you could use it somehow, when setting the
lighter? Something like

(if buffer-file-truename (file-name-directory buffer-file-truename) default-directory)

Short analysis of the debug buffer:

> 18:06:05.604340 tramp-maybe-open-connection (3) # Opening connection for silex <at> unitedsoft.ch using scpx...
> 18:06:05.604996 tramp-maybe-open-connection (6) # /bin/sh -i
> 18:06:05.607758 tramp-wait-for-regexp (6) # 
> #$ 

Here Tramp opens a second connection in order to call git.

> 18:06:05.610783 tramp-sh-handle-file-truename (4) # Finding true name for `/scpx:silex <at> unitedsoft.ch:/tmp/'
> 18:06:05.610902 tramp-send-command (6) # echo are you awake
> 18:06:15.613967 tramp-wait-for-regexp (6) # 
> are you awake
> #$ 
> 18:06:15.614566 tramp-wait-for-regexp (1) # File error: [[Regexp `\(^\|\)[^#$
> ]*///06fdc8c8f4a1778883b636f13ae5468c#\$?$' not found in 10 secs]]

And here Tramp tries to use the primary connection for getting the
truename of the default directory. Tramp is not able to accept those
requests while preparing the second connection for
start-file-process. Maybe I need to add something in order to make it
more robust. This is not such easy, it might take time.

> Philippe

Best regards, Michael.

Message #20 received at 19636 <at> debbugs.gnu.org (full text, mbox):

From: Philippe Vaucher <philippe.vaucher <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 19636 <19636 <at> debbugs.gnu.org>
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Sun, 25 Jan 2015 21:15:44 +0100

> > That's an interesting workaround, can I assume that
> > `default-directory` over a TRAMP connection is always absolute? what
> > about symlinks?
>
> You can always assume, that `default-directory' is absolute, being it
> local or remote. Symlinks are not expanded, 'tho.
>
> However, there is always the buffer local variable
> `buffer-file-truename'. Maybe you could use it somehow, when setting the
> lighter? Something like
>
> (if buffer-file-truename (file-name-directory buffer-file-truename) default-directory)

Thanks for mentionning `buffer-file-truename', I didn't know about it.
Unfortunately my problem is more complex than that because at the
moment this package does many files existence checks in order to find
the project root directory (e.g it tries to find any of "Gemfile",
".git", "Makefile" in the current directory, then if not found in the
parent one, etc). I know that all these filesystem checks doesn't
scale very well when not done locally, but I'm kinda stuck with the
current design. It is planned to refactor it all into leveraging
caching and possibly doing these checks by spawning a little bash
script remotely.


> Short analysis of the debug buffer:
(snip)
> Here Tramp opens a second connection in order to call git.
(snip)
> And here Tramp tries to use the primary connection for getting the
> truename of the default directory. Tramp is not able to accept those
> requests while preparing the second connection for
> start-file-process. Maybe I need to add something in order to make it
> more robust. This is not such easy, it might take time.

Ah, that's very helpful to understand what's going on. What is weird
is that there seems to be some kind of deadlocking happening, because
"git fetch" takes less than 10 seconds to finish, and in the log we
see that it just waits for 10 seconds until the "Are you awake"
message (I tried increasing the timeout but it just waits until the
timeout, not processing either the "git fetch" or the `file-truename'.
It seems that when the second connection starts, if you request
something that happens to use the first connection, the second
connection will also be stuck and nothing will happen until the
timeout is reached.

So, basically I can see 3 ways to tackle the problem:

- Don't try to do project detection when there's more than one TRAMP
connection to the same host. Can you point me at a way to do that?
Something like `(tramp-connections-count default-directory)'
- Find a way to detect wether the 2nd connection is "ready", which is
my attempt with `tramp-connectable-p' or `file-remote-p', but I
believe this way is a dead-end because TRAMP will tell me about the
1st connection and not the 2nd one.
- Fix TRAMP so it better handles multiple connections, but it's likely
to involve quite some work. Maybe a simple idea would be that calls to
`file-truename` always use the most recent connection? I'm not sure I
make sense so feel free to ignore it :)

Thank you!
Philippe

Message #23 received at 19636 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Philippe Vaucher <philippe.vaucher <at> gmail.com>
Cc: 19636 <19636 <at> debbugs.gnu.org>
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Sun, 25 Jan 2015 22:10:30 +0100

Philippe Vaucher <philippe.vaucher <at> gmail.com> writes:

> Thanks for mentionning `buffer-file-truename', I didn't know about it.
> Unfortunately my problem is more complex than that because at the
> moment this package does many files existence checks in order to find
> the project root directory (e.g it tries to find any of "Gemfile",
> ".git", "Makefile" in the current directory, then if not found in the
> parent one, etc). I know that all these filesystem checks doesn't
> scale very well when not done locally, but I'm kinda stuck with the
> current design. It is planned to refactor it all into leveraging
> caching and possibly doing these checks by spawning a little bash
> script remotely.

Out of the Tramp problem, do you know `locate-dominating-file'? It's
designed for that purpose; see for example how it is used in `vc-find-root'.

> Ah, that's very helpful to understand what's going on. What is weird
> is that there seems to be some kind of deadlocking happening, because
> "git fetch" takes less than 10 seconds to finish, and in the log we
> see that it just waits for 10 seconds until the "Are you awake"
> message (I tried increasing the timeout but it just waits until the
> timeout, not processing either the "git fetch" or the `file-truename'.
> It seems that when the second connection starts, if you request
> something that happens to use the first connection, the second
> connection will also be stuck and nothing will happen until the
> timeout is reached.

In the current Tramp design, there can be only up to 2 parallel
connections. The primary one is always used for all file operations, and
the second one is used exclusively for `start-file-process'. So we must
teach `file-truename' and friends to use only the primary connection,
and not the one which is just active.

> So, basically I can see 3 ways to tackle the problem:
>
> - Don't try to do project detection when there's more than one TRAMP
> connection to the same host. Can you point me at a way to do that?
> Something like `(tramp-connections-count default-directory)'

All tests I could tell you are internal to Tramp. Likely, they will
change when I reimplement the connection handling; it's not worth to use
them in your case, therefore.

> - Find a way to detect wether the 2nd connection is "ready", which is
> my attempt with `tramp-connectable-p' or `file-remote-p', but I
> believe this way is a dead-end because TRAMP will tell me about the
> 1st connection and not the 2nd one.

No, not this way. `file-truename' must use the first connection, nothing
else.

> - Fix TRAMP so it better handles multiple connections, but it's likely
> to involve quite some work. Maybe a simple idea would be that calls to
> `file-truename` always use the most recent connection? I'm not sure I
> make sense so feel free to ignore it :)

I will think about. Not sure, whether there will be results very soon.

> Thank you!
> Philippe

Best regards, Michael.

Message #26 received at 19636 <at> debbugs.gnu.org (full text, mbox):

From: Philippe Vaucher <philippe.vaucher <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 19636 <19636 <at> debbugs.gnu.org>
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Sun, 25 Jan 2015 22:47:25 +0100

> Out of the Tramp problem, do you know `locate-dominating-file'? It's
> designed for that purpose; see for example how it is used in `vc-find-root'.

Yes, actually they implement their own version, see
https://github.com/bbatsov/projectile/blob/master/projectile.el#L583
The issue that I'm trying to solve is
https://github.com/bbatsov/projectile/issues/523

There's many problems with the current implementation, namely that
they call `file-truename' way too much.

I'll follow your suggestion and take a look at `vc-find-root', maybe I
can propose a rewrite that is much simpler and TRAMP friendly.


> In the current Tramp design, there can be only up to 2 parallel
> connections. The primary one is always used for all file operations, and
> the second one is used exclusively for `start-file-process'. So we must
> teach `file-truename' and friends to use only the primary connection,
> and not the one which is just active.

Okay, good to know.


>> - Fix TRAMP so it better handles multiple connections, but it's likely
>> to involve quite some work. Maybe a simple idea would be that calls to
>> `file-truename` always use the most recent connection? I'm not sure I
>> make sense so feel free to ignore it :)
>
> I will think about. Not sure, whether there will be results very soon.

No worries. Understanding the problem helps a lot already, now I know
what can/should be done when we refactor this package.

Thanks,
Philippe

Message #29 received at 19636 <at> debbugs.gnu.org (full text, mbox):

From: Philippe Vaucher <philippe.vaucher <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 19636 <19636 <at> debbugs.gnu.org>
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Sat, 25 Mar 2017 21:43:56 +0100

[Message part 1 (text/plain, inline)]

>
> (setq buggy-tramp-mode-lighter
>       '(:eval (format " Projectile[%s]"
>                       (if (file-remote-p default-directory)
>                           default-directory
>                         (file-truename default-directory)))))
>

Hello,

I just wanted to notice you that I made progress about this issue at
https://github.com/bbatsov/projectile/pull/1129

To remind you a little bit the issue: TRAMP blocks or forgot to ask the
password to the user if a minor's mode lighter tries to query the buffer's
default-directory file properties.

My workaround is the following:

 (let* ((dir default-directory)
         (is-local (not (file-remote-p dir)))
         (is-connected (file-remote-p dir nil t)))
    (if (or is-local is-connected)
        (do-the-thing)
      (do-nothing)))

Basically, do nothing if there's no reliable way of getting the information
yet (we are remote and not connected yet).

This seems to repair the issue almost always, except for TRAMP 2.2.11 (the
one in emacs 24.5).

There is a recapitulary table at
https://github.com/bbatsov/projectile/pull/1129#issuecomment-289237057

I believe this workaround seems future proof, given all the information
comes from `file-remote-p` which is used to detect remoteness and
connectivity.

Hope it helps,
Philippe

[Message part 2 (text/html, inline)]

Message #32 received at 19636 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Philippe Vaucher <philippe.vaucher <at> gmail.com>
Cc: 19636 <19636 <at> debbugs.gnu.org>
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Mon, 27 Mar 2017 15:36:37 +0200

Philippe Vaucher <philippe.vaucher <at> gmail.com> writes:

> Hello,

Hi Philippe,

> Basically, do nothing if there's no reliable way of getting the
> information yet (we are remote and not connected yet).

Sounds reasonable for me, thanks for reporting the status. I also
recommend to test with the development version of Emacs (aka 26.0.50),
because there have been serious changes in Tramp.

> This seems to repair the issue almost always, except for TRAMP 2.2.11
> (the one in emacs 24.5).

Well, I don't see what could be done else here on emacs-devel. What do
you expect us to do else?

> Hope it helps,
> Philippe

Best regards, Michael.

Message #35 received at 19636 <at> debbugs.gnu.org (full text, mbox):

From: Philippe Vaucher <philippe.vaucher <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 19636 <19636 <at> debbugs.gnu.org>
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Mon, 27 Mar 2017 17:46:58 +0200

[Message part 1 (text/plain, inline)]

>
> > Basically, do nothing if there's no reliable way of getting the
> > information yet (we are remote and not connected yet).
>
> Sounds reasonable for me, thanks for reporting the status. I also
> recommend to test with the development version of Emacs (aka 26.0.50),
> because there have been serious changes in Tramp.
>

Good idea.


> > This seems to repair the issue almost always, except for TRAMP 2.2.11
> > (the one in emacs 24.5).
>
> Well, I don't see what could be done else here on emacs-devel. What do
> you expect us to do else?
>

Oh, I was just reporting my findings. Not much you can do except keep in
mind people *do* call various emacs api on buffer-file inside :lighter or
even inside the function that has to decide wether a mode should be on or
off, so the emacs api should be resilient to errors reguarding this.
Currently it fails kinda catastrophically when you don't check with
`file-remote-p` beforehand (emacs hang and one has to furiously press C-g
or C-c to get emacs back, and often this is not sufficient as any command
you want to type hangs again, so one has to quit).

Philippe

[Message part 2 (text/html, inline)]

Message #40 received at 19636-done <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Philippe Vaucher <philippe.vaucher <at> gmail.com>
Cc: 19636-done <at> debbugs.gnu.org
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Fri, 14 Jul 2017 15:12:18 +0200

Version: 26.1

Philippe Vaucher <philippe.vaucher <at> gmail.com> writes:

Hi Philippe,

>     > Basically, do nothing if there's no reliable way of getting the
>     > information yet (we are remote and not connected yet).
>
>     Sounds reasonable for me, thanks for reporting the status. I also
>     recommend to test with the development version of Emacs (aka
>     26.0.50), because there have been serious changes in Tramp.
>
> Good idea.
>
>     Well, I don't see what could be done else here on emacs-devel.
>     What do you expect us to do else?
>
> Oh, I was just reporting my findings. Not much you can do except keep
> in mind people *do* call various emacs api on buffer-file inside :
> lighter or even inside the function that has to decide wether a mode
> should be on or off, so the emacs api should be resilient to errors
> reguarding this. Currently it fails kinda catastrophically when you
> don't check with `file-remote-p` beforehand (emacs hang and one has to
> furiously press C-g or C-c to get emacs back, and often this is not
> sufficient as any command you want to type hangs again, so one has to
> quit).

I'm closing this bug, because there's nothing left to do.

Tramp 2.3.2 of Emacs 26.0.50 has seen some patches for improving
asynchronous processes. Maybe they help you.

> Philippe

Best regards, Michael.

Message #43 received at 19636-done <at> debbugs.gnu.org (full text, mbox):

From: Philippe Vaucher <philippe.vaucher <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 19636-done <at> debbugs.gnu.org
Subject: Re: bug#19636: [TRAMP] global minor mode hangs connection when
 accessing files in :lighter
Date: Fri, 21 Jul 2017 14:56:39 +0200

[Message part 1 (text/plain, inline)]

>
> I'm closing this bug, because there's nothing left to do.
>
> Tramp 2.3.2 of Emacs 26.0.50 has seen some patches for improving
> asynchronous processes. Maybe they help you.
>

Hello! Alright, thanks for letting me know.

Kind regards,
Philippe

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.