Subject: 27.1; eshell over tramp-mode cannot locate /bin/ executables on the remote machine

From: Novak Boškov <[email-masked]>
Date: Thu, 29 Oct 2020 11:29:34 -0400

Steps to reproduce:
1. Use regular C-x C-f /ssh:remote:/path/on/remote. The remote machine is a
GNU Linux machine too.

2. Then open eshell using M-x eshell on the remote machine.

3. Type `top` in the eshell prompt. You see the following error in a *top*
buffer:

..: 1: exec: /ssh:remote:/bin/top: not found

Process /ssh:remote:/bin/top exited abnormally with code 127

Expected behavior:
Since /bin/top is present on the remote (which can be verified if you do
the same in M-x shell), I'd expect top to work there via eshell.

I've verified that top in eshell does work if the eshell is run on the
local machine (no ssh and tramp-mode).

In GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30,
cairo version 1.15.10)
 of 2020-08-26 built on lgw01-amd64-030
Repository revision: a2fc5f02361a34ff7322032ce027c8bc4cc3b25a
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.5 LTS

Recent messages:
(no useful messages)

Configured using:
 'configure --prefix= --prefix=/snap/emacs/current/usr --with-xwidgets
 --with-x-toolkit=gtk3 --without-xaw3d --with-modules --with-cairo
 'CFLAGS= -isystem/build/emacs/parts/emacs/install/usr/include -O2'
 'CPPFLAGS= -isystem/build/emacs/parts/emacs/install/usr/include'
 'LDFLAGS= -L/build/emacs/parts/emacs/install/lib
 -L/build/emacs/parts/emacs/install/usr/lib
 -L/build/emacs/parts/emacs/install/lib/x86_64-linux-gnu
 -L/build/emacs/parts/emacs/install/usr/lib/x86_64-linux-gnu''

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS XWIDGETS
LIBSYSTEMD JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Go

Minor modes in effect:
  subword-mode: t
  global-magit-file-mode: t
  magit-file-mode: t
  magit-auto-revert-mode: t
  global-git-commit-mode: t
  lsp-diagnostics-mode: t
  lsp-modeline-workspace-status-mode: t
  lsp-modeline-diagnostics-mode: t
  lsp-modeline-code-actions-mode: t
  dap-tooltip-mode: t
  dap-ui-many-windows-mode: t
  dap-ui-controls-mode: t
  dap-ui-mode: t
  treemacs-filewatch-mode: t
  treemacs-follow-mode: t
  treemacs-git-mode: deferred
  treemacs-fringe-indicator-mode: t
  dap-auto-configure-mode: t
  dap-mode: t
  lsp-completion-mode: t
  guru-mode: t
  pdf-occur-global-minor-mode: t
  lsp-ui-mode: t
  lsp-ui-doc-mode: t
  lsp-ui-sideline-mode: t
  lsp-managed-mode: t
  lsp-mode: t
  TeX-PDF-mode: t
  TeX-source-correlate-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  global-flycheck-mode: t
  flycheck-mode: t
  global-hl-todo-mode: t
  hl-todo-mode: t
  which-function-mode: t
  global-company-mode: t
  company-mode: t
  helm-mode: t
  helm-descbinds-mode: t
  helm-ff-cache-mode: t
  helm--remap-mouse-mode: t
  async-bytecomp-package-mode: t
  ivy-mode: t
  erc-truncate-mode: t
  erc-track-mode: t
  erc-spelling-mode: t
  pixel-scroll-mode: t
  editorconfig-mode: t
  global-diff-hl-mode: t
  diff-hl-mode: t
  winner-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  whitespace-mode: t
  global-anzu-mode: t
  anzu-mode: t
  projectile-mode: t
  flyspell-mode: t
  volatile-highlights-mode: t
  global-hl-line-mode: t
  super-save-mode: t
  recentf-mode: t
  savehist-mode: t
  save-place-mode: t
  show-smartparens-global-mode: t
  show-smartparens-mode: t
  smartparens-mode: t
  global-auto-revert-mode: t
  delete-selection-mode: t
  prelude-mode: t
  shell-dirtrack-mode: t
  which-key-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug sendmail eieio-opt speedbar sb-image
ezimage dframe help-fns radix-tree go-mode find-file hackernews
mm-archive wdired move-text pyimport shut-up url-cache url-http url-gw
url-auth dired-aux iso-transl pdf-sync pdf-annot pdf-outline pdf-links
pdf-history mule-util cal-move helm-ag conf-mode man dockerfile-mode
helm-sys misearch multi-isearch swiper pcmpl-unix pcmpl-gnu finder
lisp-mnt helm-x-files helm-for-files helm-bookmark helm-adaptive
easy-kill novice em-unix em-term term ehelp em-script em-prompt em-ls
em-hist em-pred em-glob em-dirs esh-var em-cmpl em-basic em-banner
em-alias esh-mode helm-command vc-mtn vc-hg vc-bzr vc-src vc-sccs vc-svn
vc-cvs vc-rcs diff-hl-dired image-file helm-external ffap
magit-gitignore virtualenvwrapper subword-mode-expansions cap-words
superword subword bug-reference magit-bookmark magit-submodule
magit-obsolete magit-popup magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log magit-diff smerge-mode
magit-core magit-autorevert magit-margin magit-transient magit-process
magit-mode git-commit transient magit-git magit-section magit-utils
log-edit pcvs-util with-editor ox-md ol-eww ol-rmail ol-mhe ol-irc
ol-info ol-gnus nnir gnus-sum shr svg gnus-group gnus-undo gnus-start
gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo gnus-spec gnus-int
gnus-range message rfc822 mml mml-sec epa derived epg epg-config
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader gnus-win gnus nnheader gnus-util rmail rmail-loaddefs rfc2047
rfc2045 ietf-drums mm-util mail-prsvr ol-docview doc-view ol-bibtex
ol-bbdb ol-w3m lsp-diagnostics lsp-modeline add-log vc-git dap-mouse
dap-ui lsp-treemacs lsp-treemacs-themes treemacs treemacs-header-line
treemacs-compatibility treemacs-mode treemacs-bookmarks
treemacs-interface treemacs-extensions treemacs-persistence
treemacs-mouse-interface treemacs-tag-follow-mode
treemacs-filewatch-mode treemacs-tags treemacs-follow-mode
treemacs-rendering treemacs-async treemacs-workspaces treemacs-dom
treemacs-visuals treemacs-fringe-indicator treemacs-scope pulse
treemacs-faces treemacs-icons treemacs-themes treemacs-core-utils
pfuture ace-window treemacs-logging treemacs-customization
treemacs-macros gdb-mi gud bui bui-list bui-info bui-entry bui-core
bui-history bui-button bui-utils lsp-lens view lsp-svelte lsp-sqls
lsp-yaml lsp-xml lsp-vimscript lsp-vhdl lsp-vetur lsp-html lsp-verilog
lsp-terraform lsp-tex lsp-solargraph lsp-rust lsp-rf lsp-r
lsp-purescript lsp-pyls lsp-pwsh lsp-php lsp-perl lsp-ocaml lsp-nix
lsp-nim lsp-lua lsp-kotlin lsp-json lsp-javascript dap-java dap-mode
dap-launch dap-variables posframe dap-overlays lsp-java request
mail-utils lsp-haxe lsp-groovy lsp-hack lsp-go lsp-completion
lsp-gdscript lsp-fsharp lsp-fortran lsp-eslint lsp-erlang lsp-elixir
lsp-elm lsp-dockerfile lsp-dhall lsp-css lsp-csharp gnutls lsp-crystal
lsp-cmake lsp-clojure lsp-clangd lsp-bash lsp-angular lsp-ada ccls
ccls-member-hierarchy ccls-inheritance-hierarchy ccls-call-hierarchy
ccls-tree ccls-code-lens ccls-semantic-highlight ccls-common
editorconfig-core editorconfig-core-handle editorconfig-fnmatch tide
tide-lv company-anaconda anaconda-mode pythonic
python-el-fgallina-expansions smartparens-python python tramp-sh
rainbow-mode xterm-color rainbow-delimiters elisp-slime-nav guru-mode
server novak-key-bindings novak-personal-customization
prelude-more-yasnippet prelude-more-java prelude-more-pdf-tools
pdf-occur tablist tablist-filter semantic/wisent/comp semantic/wisent
semantic/wisent/wisent semantic/util-modes semantic/util semantic
semantic/tag semantic/lex semantic/fw mode-local cedet pdf-isearch
pdf-misc pdf-tools pdf-view jka-compr pdf-cache pdf-info tq pdf-util
prelude-yaml prelude-xml nxml-mode-expansions rng-nxml rng-valid
nxml-mode nxml-outln nxml-rap html-mode-expansions sgml-mode dom
prelude-web web-mode-expansions smartparens-html web-mode disp-table
prelude-ts typescript-mode prelude-scss prelude-shell sh-script smie
executable prelude-scheme geiser prelude-ruby prelude-python
prelude-perl prelude-org ox-odt rng-loc rng-uri rng-parse rng-match
rng-dt rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util
ox-latex ox-icalendar ox-html table ox-ascii ox-publish ox org-element
avl-tree ob-python ob-ditaa ob-plantuml prelude-lsp lsp-ui
lsp-ui-flycheck lsp-ui-doc xwidget image-mode exif goto-addr
lsp-ui-imenu lsp-ui-peek lsp-ui-sideline face-remap lsp-mode spinner
network-stream puny nsm rmc smartparens-markdown markdown-mode lv inline
dash-functional bindat lsp-protocol ht prelude-latex company-auctex
latex-mode-expansions latex latex-flymake flymake-proc flymake warnings
tex-ispell tex-style tex dbus ido-completing-read+ memoize minibuf-eldef
crm texmathp helm-bibtex bibtex-completion org-capture
the-org-mode-expansions smartparens-org org ob ob-tangle ob-ref ob-lob
ob-table ob-exp org-macro org-footnote org-src ob-comint org-pcomplete
org-list org-faces org-entities noutline outline org-version
ob-emacs-lisp ob-core ob-eval org-table ol org-keys org-compat org-macs
org-loaddefs cal-menu calendar cal-loaddefs biblio biblio-download
biblio-dissemin biblio-ieee biblio-hal biblio-dblp biblio-crossref
biblio-arxiv timezone biblio-doi biblio-core let-alist url-queue ido f
parsebib bibtex helm-net xml smartparens-latex prelude-js indium
indium-list-sources indium-scratch indium-interaction indium-chrome
indium-nodejs url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf url-util mailcap indium-repl
indium-debugger indium-debugger-litable js2-refactor js2r-paredit
js2r-conveniences js2r-conditionals js2r-wrapping js2r-functions
yasnippet-snippets yasnippet js2r-vars mc-hide-unmatched-lines-mode
mc-mark-more mc-cycle-cursors multiple-cursors-core js2r-iife
js2r-formatting js2r-helpers s indium-debugger-locals indium-breakpoint
indium-inspector indium-render indium-faces cus-edit cus-start cus-load
indium-seq-fix indium-client indium-structs json-process-client
js2-mode-expansions js2-mode js-mode-expansions smartparens-javascript
js cc-mode-expansions smartparens-c cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs prelude-go
prelude-emacs-lisp prelude-css prelude-clojure prelude-lisp prelude-c
prelude-programming flycheck hl-todo which-func prelude-key-chord
key-chord prelude-company company-oddmuse company-keywords company-etags
company-gtags company-dabbrev-code company-dabbrev company-files
company-clang company-capf company-cmake company-semantic
company-template company-bbdb company pcase prelude-helm-everywhere
helm-mode helm-descbinds helm-eshell helm-elisp helm-eval edebug
backtrace find-func helm-info prelude-helm helm-projectile helm-files
helm-tags helm-buffers helm-occur helm-grep helm-regexp helm-locate
helm-types helm-imenu helm-utils helm-help helm async-bytecomp
helm-global-bindings helm-easymenu helm-source helm-multi-match helm-lib
async helm-config prelude-ivy two-column ivy flx ivy-faces ivy-overlay
colir color prelude-erc erc-truncate erc-track erc-match erc-autoaway
erc-spelling erc-notify erc-networks erc-log erc-goodies erc erc-backend
erc-compat erc-loaddefs prelude-linux novak-personal-premodules
exec-path-from-shell prelude-global-keybindings pixel-scroll
prelude-editor editorconfig operate-on-number calc-bin calc-ext calc
calc-loaddefs calc-macs diff-hl smartrep vc-dir ewoc vc vc-dispatcher
diff-mode winner undo-tree diff eshell esh-cmd esh-ext esh-opt esh-proc
esh-io esh-arg esh-module esh-groups esh-util re-builder whitespace
tabify browse-kill-ring midnight ediff ediff-merg ediff-mult ediff-wind
ediff-diff ediff-help ediff-init ediff-util dired-x dired dired-loaddefs
anzu avy projectile grep compile ibuf-ext ibuffer ibuffer-loaddefs
bookmark text-property-search pp expand-region text-mode-expansions
er-basic-expansions expand-region-core expand-region-custom flyspell
ispell rect etags fileloop generator xref project volatile-highlights
hl-line super-save windmove recentf tree-widget wid-edit savehist
saveplace diminish smartparens-config smartparens-text smartparens dash
autorevert filenotify delsel prelude-mode easy-mmode crux docker-tramp
tramp-cache tramp tramp-loaddefs trampver tramp-integration files-x
tramp-compat shell pcomplete comint ansi-color ring parse-time iso8601
time-date ls-lisp format-spec thingatpt imenu-anywhere imenu
prelude-core epl prelude-ui which-key advice zenburn-theme
prelude-custom prelude-packages cl-extra help-mode finder-inf tex-site
gh-common marshal eieio-compat rx edmacro kmacro
realgud-recursive-autoloads slime-autoloads w3m-load info package
easymenu browse-url url-handlers url-parse auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
url-vars seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
xwidget-internal cairo move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 4065706 537288)
 (symbols 48 111678 3)
 (strings 32 596944 94542)
 (string-bytes 1 19459691)
 (vectors 16 204511)
 (vector-slots 8 3321598 557608)
 (floats 8 979 1543)
 (intervals 56 718451 5959)
 (buffers 1000 161))