GNU bug report logs - #60534
28.2; Forbidden reentrant call of Tramp

Previous Next

To reply to this bug, email your comments to 60534 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Georgi Danov <georgi.danov <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Cc: michael.albinus <at> gmx.de
Subject: 28.2; Forbidden reentrant call of Tramp
Date: Tue, 3 Jan 2023 21:52:36 +0100

As discussed in github issue
https://github.com/joaotavora/eglot/issues/859# eglot over tramp throws
"reentrant call" error. This is sympthom of underlying design issue -
 aux libraries use files without being aware of the tramp conection, the
 connection itself has no concurrency control, pooling or other
 multi-user features and without that guard error different tramp clients
 corrupt each other's communication with the language server.

The problem is reproducible by using company mode, probably the language
is not important. In my case it's typescript. Takes 15-30 seconds of
editing and autocomplete to trigger it.

The main congestions seems to happen in file-truename() -- after
displaying candidate company mode goes on to do refresh in the background
and calls file-truename repeatedly while eglot (or someone else) is occupying the tramp
connection. Point is -- the same tramp connection is used by eglot and
any other minor mode that queries file name or state, but they have no
means to share or coordinate this use other than the exception above.

Tramp used to throw more insignificant error in the past and tolerate
the behavior (which corrupts the data exchanged between eglot and the
lsp) and now throws this error. See `with-tramp-locked-connection`
macro. 

I got this problem with other minor modes too, but did not capture the
stack as I lowered the error prio via (add-to-list 'debug-ignored-errors
'remote-file-error). Reading the code would've suggested corruption
won't happen but still garbage was let in the stdio pipe and eglot lost track of the
buffer state.

--------------------------
In GNU Emacs 28.2 (build 1, x86_64-apple-darwin22.1.0, Carbon Version 169 AppKit 2299)
 of 2022-11-24 built on GDMBP.local
Windowing system distributor 'Apple Inc.', version 13.1.0
System Description:  macOS 13.1

Configured using:
 'configure --enable-locallisppath=/usr/local/share/emacs/site-lisp
 --infodir=/usr/local/Cellar/emacs-mac/emacs-28.2-mac-9.1/share/info/emacs
 --prefix=/usr/local/Cellar/emacs-mac/emacs-28.2-mac-9.1 --with-mac
 --enable-mac-app=/usr/local/Cellar/emacs-mac/emacs-28.2-mac-9.1
 --with-gnutls --with-modules'

Configured features:
ACL GMP GNUTLS JSON LIBXML2 MODULES NOTIFY KQUEUE PDUMPER THREADS
TOOLKIT_SCROLL_BARS XIM ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: TSX

Minor modes in effect:
  rainbow-delimiters-mode: t
  electric-pair-mode: t
  guru-mode: t
  subword-mode: t
  eglot--managed-mode: t
  flymake-mode: t
  popper-echo-mode: t
  popper-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  joro-mode: t
  doom-modeline-mode: t
  minions-mode: t
  override-global-mode: t
  global-flycheck-mode: t
  flycheck-mode: t
  global-hl-todo-mode: t
  hl-todo-mode: t
  which-function-mode: t
  global-company-mode: t
  company-mode: t
  helm-mode: t
  helm-minibuffer-history-mode: t
  helm-descbinds-mode: t
  helm--remap-mouse-mode: t
  async-bytecomp-package-mode: t
  ivy-mode: t
  editorconfig-mode: t
  winner-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  global-anzu-mode: t
  anzu-mode: t
  projectile-mode: t
  volatile-highlights-mode: t
  global-hl-line-mode: t
  super-save-mode: t
  windmove-mode: t
  recentf-mode: t
  savehist-mode: t
  save-place-mode: t
  show-smartparens-global-mode: t
  show-smartparens-mode: t
  smartparens-mode: t
  global-auto-revert-mode: t
  delete-selection-mode: t
  prelude-mode: t
  shell-dirtrack-mode: t
  which-key-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mac-mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t

Load-path shadows:
/Users/gdanov/.emacs.d/elpa/cmake-mode-20221130.1357/cmake-mode hides /usr/local/share/emacs/site-lisp/cmake/cmake-mode
/Users/gdanov/.emacs.d/elpa/transient-20221119.1503/transient hides /usr/local/Cellar/emacs-mac/emacs-28.2-mac-9.1/share/emacs/28.2/lisp/transient

Features:
(shadow sort mail-extr emacsbug sendmail typescript-mode
cc-mode-expansions smartparens-c cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs eieio-opt speedbar
ezimage dframe helm-command bug-reference helm-x-files elisp-slime-nav
rainbow-delimiters elec-pair guru-mode subword-mode-expansions cap-words
superword subword eglot array jsonrpc ert debug flymake-proc flymake
ffap tramp-archive tramp-gvfs dbus helm-for-files helm-bookmark
helm-adaptive helm-external helm-net codespaces
use-package-ensure-system-package system-packages use-package
use-package-delight use-package-diminish tramp-theme em-dirs esh-var
esh-mode poly-markdown smartparens-markdown markdown-mode polymode
poly-lock polymode-base polymode-weave polymode-export polymode-compat
polymode-methods polymode-core polymode-classes eieio-custom eieio-base
graphql-mode let-alist org-jira org-jira-sdk jiralib request soap-client
url-http url-auth url-gw nsm rng-xsd xsd-regexp xml org-clock
the-org-mode-expansions org-element avl-tree smartparens-org org ob
ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src
ob-comint org-pcomplete org-list org-faces org-entities noutline outline
org-version ob-emacs-lisp ob-core ob-eval org-table oc-basic bibtex ol
org-keys oc org-compat org-macs org-loaddefs cal-menu calendar
cal-loaddefs git-link popper-echo popper magit-bookmark magit-submodule
magit-obsolete magit-blame magit-stash magit-reflog magit-bisect
magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge
magit-branch magit-reset magit-files magit-refs magit-status magit
magit-repos magit-apply magit-wip magit-log magit-diff smerge-mode
git-commit log-edit message rmc puny rfc822 mml mml-sec epa epg rfc6068
epg-config gnus-util rmail rmail-loaddefs mailabbrev mail-utils
gmm-utils mailheader add-log magit-core magit-autorevert magit-margin
magit-transient magit-process with-editor server magit-mode transient
magit-git magit-base magit-section crm compat-27 compat-26 prettier-js
web-mode-expansions web-mode disp-table smart-shift indent-tools
yafolding hydra lv indent-tools-indentation-of highlight-indentation
helm-cider-history helm-cider helm-cider-spec helm-cider-repl
helm-cider-cheatsheet helm-cider-util cider-cheatsheet cider cider-debug
cider-browse-ns cider-mode cider-find cider-inspector cider-completion
cider-profile cider-eval cider-jar arc-mode archive-mode
cider-repl-history pulse cider-repl cider-resolve cider-test
cider-overlays cider-stacktrace cider-doc cider-browse-spec
cider-clojuredocs cider-eldoc cider-client cider-common cider-connection
cider-util cider-popup sesman-browser nrepl-client nrepl-dict spinner
clojure-mode-expansions smartparens-clojure sesman clojure-mode align
parseedn parseclj-parser parseclj-lex parseclj-alist posframe
doom-modeline doom-modeline-segments doom-modeline-env
doom-modeline-core all-the-icons all-the-icons-faces data-material
data-weathericons data-octicons data-fileicons data-faicons
data-alltheicons shrink-path f f-shortdoc shortdoc s minions
doom-themes-ext-org doom-themes-ext-treemacs doom-themes-ext-neotree
doom-themes-ext-visual-bell face-remap doom-one-theme doom-themes
doom-themes-base nlinum linum helm-ring helm-ls-git derived vc-git
helm-ag helm-xref use-package-bind-key bind-key rainbow-mode
quelpa-use-package quelpa mm-decode mm-bodies mm-encode help-fns
radix-tree use-package-ensure use-package-core prelude-modules
prelude-xml nxml-mode-expansions rng-nxml rng-valid rng-loc rng-uri
rng-parse nxml-parse rng-match rng-dt rng-util rng-pttrn nxml-ns
nxml-mode nxml-outln nxml-rap html-mode-expansions smartparens-html
sgml-mode facemenu dom nxml-util nxml-enc xmltok prelude-shell sh-script
smie executable prelude-emacs-lisp prelude-lisp prelude-programming
flycheck hl-todo compat compat-macs which-func prelude-css
prelude-company company-oddmuse company-keywords company-etags
company-gtags company-dabbrev-code company-dabbrev company-files
company-clang company-capf company-cmake company-semantic
company-template company-bbdb company prelude-helm-everywhere helm-mode
helm-misc helm-descbinds helm-eshell helm-elisp helm-eval edebug
backtrace find-func helm-info prelude-helm helm-projectile helm-files
image-dired image-mode exif helm-tags helm-buffers helm-occur helm-grep
helm-regexp helm-locate helm-types helm-imenu helm-utils helm-help helm
helm-global-bindings helm-easymenu helm-core async-bytecomp helm-source
helm-multi-match helm-lib async helm-config prelude-ivy two-column ivy
ivy-faces ivy-overlay colir color prelude-macos exec-path-from-shell
prelude-global-keybindings warnings prelude-editor editorconfig
editorconfig-core editorconfig-core-handle editorconfig-fnmatch
operate-on-number calc-bin calc-ext calc calc-loaddefs calc-macs diff-hl
smartrep log-view pcvs-util vc-dir ewoc vc vc-dispatcher diff-mode
winner undo-tree diff queue eshell esh-cmd esh-ext esh-opt esh-proc
esh-io esh-arg esh-module esh-groups esh-util re-builder whitespace
tabify browse-kill-ring midnight ediff ediff-merg ediff-mult ediff-wind
ediff-diff ediff-help ediff-init ediff-util dired-x anzu avy tramp-sh
projectile skeleton ibuf-macs find-dired dired dired-loaddefs lisp-mnt
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr grep
compile ibuf-ext ibuffer ibuffer-loaddefs bookmark text-property-search
pp expand-region text-mode-expansions er-basic-expansions
expand-region-core expand-region-custom flyspell ispell rect etags
fileloop generator xref project volatile-highlights hl-line super-save
windmove recentf tree-widget wid-edit savehist saveplace diminish
smartparens-config smartparens-text smartparens advice dash autorevert
filenotify delsel prelude-mode easy-mmode crux docker-tramp tramp-cache
tramp tramp-loaddefs trampver tramp-integration files-x tramp-compat
shell pcomplete comint ansi-color ring parse-time iso8601 time-date
ls-lisp format-spec thingatpt imenu-anywhere imenu pcase jka-compr
prelude-core epl prelude-ui which-key prelude-custom prelude-packages
cl-extra help-mode finder-inf rx info edmacro kmacro package browse-url
url url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util mailcap url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
url-vars seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/mac-win mac-win term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads kqueue mac
multi-tty make-network-process emacs)

Memory information:
((conses 16 1024998 582937)
 (symbols 48 64969 19)
 (strings 32 287822 163197)
 (string-bytes 1 9937920)
 (vectors 16 113430)
 (vector-slots 8 2200185 552812)
 (floats 8 1234 2058)
 (intervals 56 14261 3055)
 (buffers 992 43))

Message #8 received at 60534 <at> debbugs.gnu.org (full text, mbox):

From: Georgi Danov <georgi.danov <at> gmail.com>
To: 60534 <at> debbugs.gnu.org
Subject: Re: bug#60534: Acknowledgement (28.2; Forbidden reentrant call of
 Tramp)
Date: Wed, 4 Jan 2023 10:33:47 +0100

additional context: with the docker tramp the problem is reproduced much easier than with ssh tramp.

> On 04.01.2023, at 00:57, GNU bug Tracking System <help-debbugs <at> gnu.org> wrote:
> 
> Thank you for filing a new bug report with debbugs.gnu.org.
> 
> This is an automatically generated reply to let you know your message
> has been received.
> 
> Your message is being forwarded to the package maintainers and other
> interested parties for their attention; they will reply in due course.
> 
> Your message has been sent to the package maintainer(s):
> bug-gnu-emacs <at> gnu.org
> 
> If you wish to submit further information on this problem, please
> send it to 60534 <at> debbugs.gnu.org.
> 
> Please do not send mail to help-debbugs <at> gnu.org unless you wish
> to report a problem with the Bug-tracking system.
> 
> -- 
> 60534: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=60534
> GNU Bug Tracking System
> Contact help-debbugs <at> gnu.org with problems

Message #11 received at 60534 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Georgi Danov <georgi.danov <at> gmail.com>
Cc: 60534 <at> debbugs.gnu.org
Subject: Re: bug#60534: 28.2; Forbidden reentrant call of Tramp
Date: Fri, 17 Feb 2023 10:42:17 +0100

Georgi Danov <georgi.danov <at> gmail.com> writes:

Hi Georgi,

> As discussed in github issue
> https://github.com/joaotavora/eglot/issues/859# eglot over tramp throws
> "reentrant call" error. This is sympthom of underlying design issue -
>  aux libraries use files without being aware of the tramp conection, the
>  connection itself has no concurrency control, pooling or other
>  multi-user features and without that guard error different tramp clients
>  corrupt each other's communication with the language server.

Yes. This is a long standing problem in Tramp I have no solution
yet. Likely we need threading in Tramp for this, but this causes other
problems.

For the time being I merge this with bug#49954, which points to the same
problem (although with different symptoms).

Best regards, Michael.

Message #16 received at 60534 <at> debbugs.gnu.org (full text, mbox):

From: James Thomas <jimjoe <at> gmx.net>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Dima Kogan <dima <at> secretsauce.net>, 49954 <at> debbugs.gnu.org,
 60534 <at> debbugs.gnu.org
Subject: Re: bug#60534: 28.2; Forbidden reentrant call of Tramp
Date: Tue, 17 Dec 2024 04:05:22 +0530

Michael Albinus wrote:

> In order to understand the problem, let's assume the following scenario:
>
> - You have connected to a remote host, say "/ssh:host:". Tramp uses
>   internally the process *tramp/ssh host* for communicating with that
>   process.
>
> - You have also started another asynchronous process to that remote
>   host.
>
> - Now, while normal use of Emacs, the function (file-attributes
>   "/ssh:host:/path/to/file") is called. Tramp sends a command to the
>   process *tramp/ssh host*, like "stat /path/to/file".
>
> - While Tramp waits for the answer of the "stat ..." command, your other
>   process has finished. It might have a process sentinel, which is
>   called exactly at this time, because Tramp is in a loop
>   (accept-process-output ...).
>
> - This process filter might trigger another file operation, like
>   (delete-file "/ssh:host:/tmp/tmpfile"). This would require to send
>   another command to the *tramp/ssh host* process like "rm -f /tmp/tmpfile".
>
> - Since the first command, "stat ...", hasn't been finished, this would
>   result in inconsistencies. Tramp detects this situation, and raises
>   the "Forbidden reentrant call of Tramp" error.
>
> Not so easy to solve. Ideally, remote file name functions initiated in
> process filters, process sentinels, timers and alike shall wait, until
> the currently executed remote command has finished. Don't know how to
> achieve this.

(This may be an ignorant question, but) if that's so, is it possible to
open a separate connection (perhaps with a ControlPath suffix, and
ControlPersist-ed) in the place of the "Forbidden reentrant call"?

Regards,
James

Message #19 received at 60534 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: James Thomas <jimjoe <at> gmx.net>
Cc: Dima Kogan <dima <at> secretsauce.net>, 49954 <at> debbugs.gnu.org,
 60534 <at> debbugs.gnu.org
Subject: Re: bug#60534: 28.2; Forbidden reentrant call of Tramp
Date: Tue, 17 Dec 2024 09:29:36 +0100

James Thomas <jimjoe <at> gmx.net> writes:

Hi James,

>> Not so easy to solve. Ideally, remote file name functions initiated in
>> process filters, process sentinels, timers and alike shall wait, until
>> the currently executed remote command has finished. Don't know how to
>> achieve this.
>
> (This may be an ignorant question, but) if that's so, is it possible to
> open a separate connection (perhaps with a ControlPath suffix, and
> ControlPersist-ed) in the place of the "Forbidden reentrant call"?

Not so simple. There is a serious overhead when opening a new
connection, due to the handshaking actions. And it isn't clear to me how
to keep two connections in sync, if (for example) the environment
changes in one of the connections. Be it an environment variable, the
current directory, the availability of a temp file, you name it.

But yes, nobody has tried it yet. My preference is to use threads, so
that one command in a process filter could wait until another command in
the main thread has finished, as example. But the crucial point is, that
you must activate threads in the beginning of a connection. When you
detect, that there is a forbidded reentrant call, it is too late to
activate threads.

> Regards,
> James

Best regards, Michael.

Message #22 received at 60534 <at> debbugs.gnu.org (full text, mbox):

From: James Thomas <jimjoe <at> gmx.net>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Dima Kogan <dima <at> secretsauce.net>, 49954 <at> debbugs.gnu.org,
 60534 <at> debbugs.gnu.org
Subject: Re: bug#60534: 28.2; Forbidden reentrant call of Tramp
Date: Tue, 17 Dec 2024 18:41:53 +0530

Michael Albinus wrote:

> Not so simple.

> ...

> And it isn't clear to me how to keep two connections in sync, if (for
> example) the environment changes in one of the connections. Be it an
> environment variable, the current directory, the availability of a
> temp file, you name it.

And here was I thinking that Tramp was mostly remote-state-less beyond
the handshake...

> There is a serious overhead when opening a new connection, due to the
> handshaking actions.

OK; I misremembered for a moment that it was beyond any ControlPersist.

> But yes, nobody has tried it yet. My preference is to use threads, so
> that one command in a process filter could wait until another command
> in the main thread has finished, as example. But the crucial point is,
> that you must activate threads in the beginning of a connection. When
> you detect, that there is a forbidded reentrant call, it is too late
> to activate threads.

Ah, crucial; but not a blocker, I suppose.

Thank you for the effort in explaining, Michael.

Regards,
James

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.