Subject: bug#64546: 30.0.50; [PATCH] Add support for explicitly-remote commands in Eshell

From: Jim Porter <[email-masked]>
Date: Mon, 10 Jul 2023 09:53:46 -0700

On 7/10/2023 12:24 AM, Michael Albinus wrote:
> Jim Porter <[email-masked]> writes:
>> This patch adds the ability to run a command in Eshell from any host,
>> no matter your current directory. For example, you could run
>> "/ssh:user@remote:whoami" from a local dir, which would run "whoami"
>> over the SSH connection for "user@remote".
> 
> Looks nice. But what if I want to run a command on another remote host
> with an absolute path? Would "/ssh:user@remote:/usr/bin/whoami" also be
> possible?

Yes, the local part should let you type any command name that would work 
if you were in the home directory[1] for that connection. I'll also add 
a note about that to the manual (and a regression test).

> The same question. What about calling "/:/usr/bin/whoami"?

Ditto.

>> +By default, commands like @code{ssh} and @code{sudo} use the external
>> +programs by those names, so if you ran @samp{ssh
>> +@var{user}@@@var{remote}}, you would end up in the default shell
>> +program for @var{user} on @var{remote}, @emph{not} in Eshell.  If you
>> +prefer to use commands like @code{ssh} but remain in Eshell
>> +afterwards, you can enable the optional Tramp extensions (@pxref{Tramp
>> +extensions}).
> 
> This surprises me. I thought, that only "doas", "su" and "sudo" are built-ins.

Oops! Somehow, I got it into my head that we had an 'eshell/ssh' 
builtin, but that's not the case. I'll just remove this paragraph.

[1] Well, whatever the default directory is if you ran "cd 
/method:user@host:", anyway. Personally, I'd avoid trying to use 
relative paths though; I think that's a bit confusing.

From 8c03abcdb81205594cb900333ddbbcebfad29ae0 Mon Sep 17 00:00:00 2001
From: Jim Porter <[email-masked]>
Date: Sun, 9 Jul 2023 12:04:01 -0700
Subject: [PATCH 1/2] Add documentation about remote access in Eshell

* doc/misc/eshell.texi
(Invocation): Mention the '*' prefix.
(Remote Access): New section...
(Commands): ... link to it.
---
 doc/misc/eshell.texi | 31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/doc/misc/eshell.texi b/doc/misc/eshell.texi
index c6376882542..cd0aaf69add 100644
--- a/doc/misc/eshell.texi
+++ b/doc/misc/eshell.texi
@@ -207,6 +207,7 @@ Commands
 * Built-ins::
 * Variables::
 * Aliases::
+* Remote Access::
 * History::
 * Completion::
 * Control Flow::
@@ -266,7 +267,10 @@ Invocation
 @vindex eshell-prefer-lisp-functions
 If you would prefer to use ordinary Lisp functions over external
 programs, set the option @code{eshell-prefer-lisp-functions} to
-@code{t}.  This will swap the lookup order of the last two items.
+@code{t}.  This will swap the lookup order of the last two items. You
+can also force Eshell to look for a command as an external program by
+prefixing its name with @kbd{*}, like @code{*@var{command}}
+(@pxref{Built-ins}).
 
 You can also group command forms together into a subcommand with curly
 braces (@code{@{@}}).  This lets you use the output of a subcommand as
@@ -1090,6 +1094,31 @@ Aliases
 
 @end table
 
+@node Remote Access
+@section Remote Access
+@cmindex remote access
+
+Since Eshell uses Emacs facilities for most of its functionality, you
+can access remote hosts transparently.  To connect to a remote host,
+simply @code{cd} into it:
+
+@example
+~ $ cd /ssh:user@@remote:
+/ssh:user@@remote:~ $
+@end example
+
+Additionally, built-in Eshell commands (@pxref{Built-ins}) and
+ordinary Lisp functions accept remote file names, so you can access
+them even without explicitly connecting first.  For example, to print
+the contents of a remote file, you could type @samp{cat
+/ssh:user@@remote:~/output.log}.  However, this means that when using
+built-in commands or Lisp functions from a remote directory, you must
+be careful about specifying absolute file names: @samp{cat
+/var/output.log} will always print the contents of your @emph{local}
+@file{/var/output.log}, even from a remote directory.  If you find
+this behavior annoying, you can enable the optional electric forward
+slash module (@pxref{Electric forward slash}).
+
 @node History
 @section History
 @cmindex history
-- 
2.25.1

From c7c431803df9b60ab29509e890f6c2cde3b38ae5 Mon Sep 17 00:00:00 2001
From: Jim Porter <[email-masked]>
Date: Sun, 9 Jul 2023 12:06:13 -0700
Subject: [PATCH 2/2] Add support for explicitly-remote commands in Eshell

* lisp/files.el (file-remote-p):
* doc/lispref/files.texi (Magic File Names): Document 'never' for
CONNECTED argument.

* lisp/net/tramp.el (tramp-handle-file-remote-p): Handle CONNECTED
value of 'never'.

* lisp/eshell/esh-ext.el (eshell-explicit-remote-commands): New
option.
(eshell-ext-initialize): Apply 'eshell-handle-remote-command' when
requested.
(eshell-handle-remote-command): New function.
(eshell-remote-command): Reimplement this function and dispatch to
'eshell-external-command', which can handle remote processes on its
own.

* test/lisp/eshell/esh-ext-tests.el
(esh-ext-test/explicitly-remote-command)
(esh-ext-test/explicitly-local-command): New tests.

* doc/misc/eshell.texi (Remote Access): Document explicitly-remote
commands.

* etc/NEWS: Announce this change.
---
 doc/lispref/files.texi            |  6 ++-
 doc/misc/eshell.texi              | 12 ++++++
 etc/NEWS                          |  9 +++++
 lisp/eshell/esh-ext.el            | 62 ++++++++++++++++++++-----------
 lisp/files.el                     |  4 +-
 lisp/net/tramp.el                 |  5 ++-
 test/lisp/eshell/esh-ext-tests.el | 32 ++++++++++++++++
 7 files changed, 104 insertions(+), 26 deletions(-)

diff --git a/doc/lispref/files.texi b/doc/lispref/files.texi
index 66de0f036c4..31d4aaca507 100644
--- a/doc/lispref/files.texi
+++ b/doc/lispref/files.texi
@@ -3630,7 +3630,11 @@ Magic File Names
 If @var{connected} is non-@code{nil}, this function returns @code{nil}
 even if @var{filename} is remote, if Emacs has no network connection
 to its host.  This is useful when you want to avoid the delay of
-making connections when they don't exist.
+making connections when they don't exist.  If @var{connected} is
+@code{never}, @emph{never} use an existing connection to return the
+identification, even if one is already present (this is otherwise like
+a value of @code{nil}).  This lets you prevent any connection-specific
+logic, such as expanding the local part of the file name.
 @end defun
 
 @defun unhandled-file-name-directory filename
diff --git a/doc/misc/eshell.texi b/doc/misc/eshell.texi
index cd0aaf69add..ecc12035650 100644
--- a/doc/misc/eshell.texi
+++ b/doc/misc/eshell.texi
@@ -1119,6 +1119,18 @@ Remote Access
 this behavior annoying, you can enable the optional electric forward
 slash module (@pxref{Electric forward slash}).
 
+@vindex eshell-explicit-remote-commands
+When running commands, you can also make them explicitly remote by
+prefixing the command name with a remote identifier, e.g.@:
+@samp{/ssh:user@@remote:whoami}.  This runs the command @code{whoami}
+over the SSH connection for @code{user@@remote}, no matter your
+current directory.  If you want to explicitly run a @emph{local}
+command even when in a remote directory, you can prefix the command
+name with @kbd{/:}, like @samp{/:whoami}.  In either case, you can
+also specify the absolute path to the program, e.g.@:
+@samp{/ssh:user@@remote:/usr/bin/whoami}.  To disable this syntax, set
+the option @code{eshell-explicit-remote-commands} to @code{nil}.
+
 @node History
 @section History
 @cmindex history
diff --git a/etc/NEWS b/etc/NEWS
index 246e6b21838..5d5ea990b92 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -202,6 +202,15 @@ or get a sublist of elements 2 through 4 with '$my-list[2..5]'.  For
 more information, see the "(eshell) Dollars Expansion" node in the
 Eshell manual.
 
++++
+*** Eshell commands can now be explicitly-remote (or local).
+By prefixing a command name in Eshell with a remote identifier, like
+"/ssh:user@remote:whoami", you can now runs commands on a particular
+host no matter your current directory.  Likewise, you can run a
+command on your local system no matter your current directory via
+"/:whoami".  For more information, see the "(eshell) Remote Access"
+node in the Eshell manual.
+
 +++
 *** Eshell's '$UID' and '$GID' variables are now connection-aware.
 Now, when expanding '$UID' or '$GID' in a remote directory, the value
diff --git a/lisp/eshell/esh-ext.el b/lisp/eshell/esh-ext.el
index f350622e78c..38579d7ef1c 100644
--- a/lisp/eshell/esh-ext.el
+++ b/lisp/eshell/esh-ext.el
@@ -168,11 +168,23 @@ eshell-explicit-command-char
   :type 'character
   :group 'eshell-ext)
 
+(defcustom eshell-explicit-remote-commands t
+  "If non-nil, support explicitly-remote commands.
+These are commands with a full remote file name, such as
+\"/ssh:host:whoami\".  If this is enabled, you can also run
+explicitly-local commands by using a quoted file name, like
+\"/:whoami\"."
+  :type 'boolean
+  :group 'eshell-ext)
+
 ;;; Functions:
 
 (defun eshell-ext-initialize ()     ;Called from `eshell-mode' via intern-soft!
   "Initialize the external command handling code."
-  (add-hook 'eshell-named-command-hook #'eshell-explicit-command nil t))
+  (add-hook 'eshell-named-command-hook #'eshell-explicit-command nil t)
+  (when eshell-explicit-remote-commands
+    (add-hook 'eshell-named-command-hook
+              #'eshell-handle-remote-command nil t)))
 
 (defun eshell-explicit-command (command args)
   "If a command name begins with `*', call it externally always.
@@ -186,30 +198,36 @@ eshell-explicit-command
 	(error "%s: external command not found"
 	       (substring command 1))))))
 
+(defun eshell-handle-remote-command (command args)
+  "Handle remote (or quoted) COMMAND names, using ARGS.
+This calls the appropriate function for commands that aren't on
+the connection associated with `default-directory'.  (See
+`eshell-explicit-remote-commands'.)"
+  (if (file-name-quoted-p command)
+      (let ((default-directory (if (file-remote-p default-directory)
+                                   (expand-file-name "~")
+                                 default-directory)))
+        (eshell-external-command (file-name-unquote command) args))
+    (when (file-remote-p command)
+      (eshell-remote-command command args))))
+
 (defun eshell-remote-command (command args)
   "Insert output from a remote COMMAND, using ARGS.
 A remote command is something that executes on a different machine.
-An external command simply means external to Emacs.
-
-Note that this function is very crude at the moment.  It gathers up
-all the output from the remote command, and sends it all at once,
-causing the user to wonder if anything's really going on..."
-  (let ((outbuf (generate-new-buffer " *eshell remote output*"))
-	(errbuf (generate-new-buffer " *eshell remote error*"))
-	(command (file-local-name command))
-	(exitcode 1))
-    (unwind-protect
-	(progn
-	  (setq exitcode
-		(shell-command
-		 (mapconcat #'shell-quote-argument
-			    (append (list command) args) " ")
-		 outbuf errbuf))
-	  (eshell-print (with-current-buffer outbuf (buffer-string)))
-	  (eshell-error (with-current-buffer errbuf (buffer-string))))
-      (eshell-close-handles exitcode 'nil)
-      (kill-buffer outbuf)
-      (kill-buffer errbuf))))
+An external command simply means external to Emacs."
+  (let* ((cwd-connection (file-remote-p default-directory))
+         (command-connection (file-remote-p command))
+         (default-directory (if (equal cwd-connection command-connection)
+                                default-directory
+                              command-connection))
+         ;; Never use the remote connection here.  We don't want to
+         ;; expand the local name!  Instead, we want it as the user
+         ;; typed, so that if COMMAND is "/ssh:host:cat", we just get
+         ;; "cat" as the result.
+         (command-localname (file-remote-p command 'localname 'never)))
+    (unless command-connection
+      (error "%s: not a remote command" command))
+    (eshell-external-command command-localname args)))
 
 (defun eshell-external-command (command args)
   "Insert output from an external COMMAND, using ARGS."
diff --git a/lisp/files.el b/lisp/files.el
index 2fffd2e6c35..377ed1b8a0b 100644
--- a/lisp/files.el
+++ b/lisp/files.el
@@ -1270,7 +1270,9 @@ file-remote-p
 
 If CONNECTED is non-nil, return an identification only if FILE is
 located on a remote system and a connection is established to
-that remote system.
+that remote system.  If CONNECTED is `never', never use an
+existing connection to return the identification (this is
+otherwise like a value of nil).
 
 Tip: You can use this expansion of remote identifier components
      to derive a new remote file name from an existing one.  For
diff --git a/lisp/net/tramp.el b/lisp/net/tramp.el
index 8c9023d004a..538cc4252db 100644
--- a/lisp/net/tramp.el
+++ b/lisp/net/tramp.el
@@ -4336,13 +4336,14 @@ tramp-handle-file-remote-p
   (let ((tramp-verbose (min tramp-verbose 3)))
     (when (tramp-tramp-file-p filename)
       (let* ((o (tramp-dissect-file-name filename))
-	     (p (tramp-get-connection-process o))
+	     (p (and (not (eq connected 'never))
+                     (tramp-get-connection-process o)))
 	     (c (and (process-live-p p)
 		     (tramp-get-connection-property p "connected"))))
 	;; We expand the file name only, if there is already a connection.
 	(with-parsed-tramp-file-name
 	    (if c (expand-file-name filename) filename) nil
-	  (and (or (not connected) c)
+	  (and (or (memq connected '(nil never)) c)
 	       (cond
 		((eq identification 'method) method)
 		;; Domain and port are appended to user and host,
diff --git a/test/lisp/eshell/esh-ext-tests.el b/test/lisp/eshell/esh-ext-tests.el
index ef073d3487d..aae297cd413 100644
--- a/test/lisp/eshell/esh-ext-tests.el
+++ b/test/lisp/eshell/esh-ext-tests.el
@@ -23,6 +23,7 @@
 
 ;;; Code:
 
+(require 'tramp)
 (require 'ert)
 (require 'esh-mode)
 (require 'esh-ext)
@@ -73,4 +74,35 @@ esh-ext-test/addpath/set-locally
      (eshell-match-command-output "echo $PATH"
                                   (concat original-path "\n")))))
 
+(ert-deftest esh-ext-test/explicitly-remote-command ()
+  "Test that an explicitly-remote command is remote no matter the current dir."
+  (skip-unless (and (eshell-tests-remote-accessible-p)
+                    (executable-find "sh")))
+  (dolist (default-directory (list default-directory
+                                   ert-remote-temporary-file-directory))
+    (dolist (cmd (list "sh" (executable-find "sh")))
+      (ert-info ((format "Directory: %s; executable: %s" default-directory cmd))
+        (with-temp-eshell
+         ;; Check the value of $INSIDE_EMACS using `sh' in order to
+         ;; delay variable expansion.
+         (eshell-match-command-output
+          (format "%s%s -c 'echo $INSIDE_EMACS'"
+                  (file-remote-p ert-remote-temporary-file-directory) cmd)
+          "eshell,tramp"))))))
+
+(ert-deftest esh-ext-test/explicitly-local-command ()
+  "Test that an explicitly-local command is local no matter the current dir."
+  (skip-unless (and (eshell-tests-remote-accessible-p)
+                    (executable-find "sh")))
+  (dolist (default-directory (list default-directory
+                                   ert-remote-temporary-file-directory))
+    (dolist (cmd (list "sh" (executable-find "sh")))
+      (ert-info ((format "In directory: %s" default-directory))
+        (with-temp-eshell
+         ;; Check the value of $INSIDE_EMACS using `sh' in order to
+         ;; delay variable expansion.
+         (eshell-match-command-output
+          (format "/:%s -c 'echo $INSIDE_EMACS'" cmd)
+          "eshell\n"))))))
+
 ;; esh-ext-tests.el ends here
-- 
2.25.1