GNU bug report logs - #54487
29.0.50; connection-local value for `shell-file-name' not set in Dired buffers over Tramp

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 54487 in the body.
You can then email your comments to 54487 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.0.50; connection-local value for `shell-file-name' not set in
 Dired buffers over Tramp
Date: Sun, 20 Mar 2022 21:58:56 -0700

Hopefully I've summarized the issue correctly in the bug title. To see 
this in action, run the following from `emacs -Q' on an MS-Windows 
system ("host" in this example is a remote GNU/Linux system):

  C-x C-f /ssh:host:~
  M-x rgrep RET
  some text RET RET RET

The rgrep output will look something like:

  find [...] --null -e "some text" "{}" +
  find: paths must precede expression: `^^!^'

You can click the "[...]" to see the full invocation. However, even 
without doing that, if you look carefully, you'll notice that the 
shell-quoting uses the MS-Windows rules, not that of /bin/sh. For the 
MS-Windows shell, spaces are quoted by wrapping the entire argument in 
double-quotes ("like this"); for /bin/sh, spaces are escaped via a 
backslash (like\ this).

Presumably, that's because if you eval `shell-file-name' in the Dired 
buffer, it reports ".../path/to/cmdproxy.exe". When in a remote *file*, 
`shell-file-name' is correctly set to "/bin/sh".

This also comes up in other (non-Dired) situations. For example:

  C-x C-f /ssh:host:~/some-file.txt
  M-x rgrep RET
  some text RET RET RET
  ;; everything looks ok

  ;; now, from the rgrep buffer...
  M-x rgrep
  some text RET RET RET
  ;; same error as in the original case above

Message #8 received at 54487 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Jim Porter <jporterbugs <at> gmail.com>
Cc: 54487 <at> debbugs.gnu.org
Subject: Re: bug#54487: 29.0.50; connection-local value for
 `shell-file-name' not set in Dired buffers over Tramp
Date: Mon, 21 Mar 2022 11:25:28 +0100

[Message part 1 (text/plain, inline)]

Jim Porter <jporterbugs <at> gmail.com> writes:

Hi Jim,

> Hopefully I've summarized the issue correctly in the bug title. To see
> this in action, run the following from `emacs -Q' on an MS-Windows
> system ("host" in this example is a remote GNU/Linux system):
>
>   C-x C-f /ssh:host:~
>   M-x rgrep RET
>   some text RET RET RET
>
> The rgrep output will look something like:
>
>   find [...] --null -e "some text" "{}" +
>   find: paths must precede expression: `^^!^'

I confirm the bug, it happens also for me.

> You can click the "[...]" to see the full invocation. However, even
> without doing that, if you look carefully, you'll notice that the
> shell-quoting uses the MS-Windows rules, not that of /bin/sh. For the
> MS-Windows shell, spaces are quoted by wrapping the entire argument in
> double-quotes ("like this"); for /bin/sh, spaces are escaped via a
> backslash (like\ this).
>
> Presumably, that's because if you eval `shell-file-name' in the Dired
> buffer, it reports ".../path/to/cmdproxy.exe". When in a remote
> *file*, `shell-file-name' is correctly set to "/bin/sh".

It is not a problem of shell-file-name, if you check the Tramp debug
buffer you'll see, that a proper shell ("/bin/sh" in my case) is
applied.

The problem is rather quoting the arguments with shell-quote-argument. It
applies the quoting according to the value of system-type. If this is
'ms-dos or 'windows-nt, MS Windows quoting rules are applied.

The appended patch fixes this for me, could you pls check?

Best regards, Michael.

[Message part 2 (text/x-patch, attachment)]

Message #11 received at 54487 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: jporterbugs <at> gmail.com, 54487 <at> debbugs.gnu.org
Subject: Re: bug#54487: 29.0.50;
 connection-local value for `shell-file-name' not set in Dired buffers
 over Tramp
Date: Mon, 21 Mar 2022 14:40:39 +0200

> From: Michael Albinus <michael.albinus <at> gmx.de>
> Date: Mon, 21 Mar 2022 11:25:28 +0100
> Cc: 54487 <at> debbugs.gnu.org
> 
> The problem is rather quoting the arguments with shell-quote-argument. It
> applies the quoting according to the value of system-type. If this is
> 'ms-dos or 'windows-nt, MS Windows quoting rules are applied.
> 
> The appended patch fixes this for me, could you pls check?

Is it really a good idea to solve this only for Grep?  Shouldn't shell
quoting always use this logic (with some variable that callers could
bind in exceptional cases, which I presume will be rare)?  Or am I
missing something?

Message #14 received at 54487 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jporterbugs <at> gmail.com, 54487 <at> debbugs.gnu.org
Subject: Re: bug#54487: 29.0.50; connection-local value for
 `shell-file-name' not set in Dired buffers over Tramp
Date: Mon, 21 Mar 2022 15:06:28 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

Hi Eli,

> Is it really a good idea to solve this only for Grep?  Shouldn't shell
> quoting always use this logic (with some variable that callers could
> bind in exceptional cases, which I presume will be rare)?  Or am I
> missing something?

I had the same feeling after sending the patch, so I've started to
rework this. I came out with the following solution:

--8<---------------cut here---------------start------------->8---
shell-quote-argument is a compiled Lisp function in
‘../../../src/emacs/lisp/subr.el’.

(shell-quote-argument ARGUMENT &optional POSIX)

Quote ARGUMENT for passing as argument to an inferior shell.

This function is designed to work with the syntax of your system’s
standard shell, and might produce incorrect results with unusual shells.
See Info node ‘(elisp)Security Considerations’.

If the optional POSIX argument is non-nil, ARGUMENT is quoted
according to POSIX rules.
--8<---------------cut here---------------end--------------->8---

I'll wait until Jim confirms that this works in general, then I would
apply a patch along this spec.

Best regards, Michael.

Message #17 received at 54487 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: jporterbugs <at> gmail.com, 54487 <at> debbugs.gnu.org
Subject: Re: bug#54487: 29.0.50; connection-local value for
 `shell-file-name' not set in Dired buffers over Tramp
Date: Mon, 21 Mar 2022 16:52:21 +0200

> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: jporterbugs <at> gmail.com,  54487 <at> debbugs.gnu.org
> Date: Mon, 21 Mar 2022 15:06:28 +0100
> 
> (shell-quote-argument ARGUMENT &optional POSIX)
> 
> Quote ARGUMENT for passing as argument to an inferior shell.
> 
> This function is designed to work with the syntax of your system’s
> standard shell, and might produce incorrect results with unusual shells.
> See Info node ‘(elisp)Security Considerations’.
> 
> If the optional POSIX argument is non-nil, ARGUMENT is quoted
> according to POSIX rules.

Thanks.

Please augment the last sentence by using "according to POSIX shell
quoting rules, regardless of the system's shell."  Or something
similar.

Message #20 received at 54487 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jporterbugs <at> gmail.com, 54487 <at> debbugs.gnu.org
Subject: Re: bug#54487: 29.0.50; connection-local value for
 `shell-file-name' not set in Dired buffers over Tramp
Date: Mon, 21 Mar 2022 16:02:09 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

Hi Eli,

>> If the optional POSIX argument is non-nil, ARGUMENT is quoted
>> according to POSIX rules.
>
> Thanks.
>
> Please augment the last sentence by using "according to POSIX shell
> quoting rules, regardless of the system's shell."  Or something
> similar.

Sure. I'm just testing the full patch, and it looks promising.

While I'm at this, I'm thinking whether I shall change computing of grep
defaults to connection-local variables. This would be more in line with
handling such local variables in recent Emacsen. WDYT?

Best regards, Michael.

Message #23 received at 54487 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: jporterbugs <at> gmail.com, 54487 <at> debbugs.gnu.org
Subject: Re: bug#54487: 29.0.50; connection-local value for
 `shell-file-name' not set in Dired buffers over Tramp
Date: Mon, 21 Mar 2022 17:05:36 +0200

> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: jporterbugs <at> gmail.com,  54487 <at> debbugs.gnu.org
> Date: Mon, 21 Mar 2022 16:02:09 +0100
> 
> While I'm at this, I'm thinking whether I shall change computing of grep
> defaults to connection-local variables. This would be more in line with
> handling such local variables in recent Emacsen. WDYT?

Probably.  But I don't have enough experience with remote processes,
so it is probably best to wait for others to chime in.

Message #26 received at 54487 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jporterbugs <at> gmail.com, 54487 <at> debbugs.gnu.org
Subject: Re: bug#54487: 29.0.50; connection-local value for
 `shell-file-name' not set in Dired buffers over Tramp
Date: Mon, 21 Mar 2022 16:09:50 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

Hi Eli,

>> While I'm at this, I'm thinking whether I shall change computing of grep
>> defaults to connection-local variables. This would be more in line with
>> handling such local variables in recent Emacsen. WDYT?
>
> Probably.  But I don't have enough experience with remote processes,
> so it is probably best to wait for others to chime in.

OK. Anyway, I will push first a patch for the given bug. Other changes,
if any, will be different patches.

Best regards, Michael.

Message #29 received at 54487 <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>, Eli Zaretskii <eliz <at> gnu.org>
Cc: 54487 <at> debbugs.gnu.org
Subject: Re: bug#54487: 29.0.50; connection-local value for `shell-file-name'
 not set in Dired buffers over Tramp
Date: Mon, 21 Mar 2022 11:04:32 -0700

On 3/21/2022 7:06 AM, Michael Albinus wrote:
> I'll wait until Jim confirms that this works in general, then I would
> apply a patch along this spec.

The patch you posted works for me. Setting `shell-file-name' to 
"/bin/sh" worked in my tests because it makes the function 
`w32-shell-dos-semantics' return nil, so this condition in 
`shell-quote-argument' isn't matched:

  ((and (eq system-type 'windows-nt) (w32-shell-dos-semantics))

That makes the shell-quoting use POSIX-style rules instead, which is 
what we want if the default-directory is remote. Reading that code, I 
think the `w32-shell-dos-semantics' part of that condition is there to 
handle things like Cygwin builds, so maybe it's not quite right to rely 
on that for the case I described in the original report. (That said, I 
think it would only be an issue for some truly esoteric configurations.)

On the other hand, I think I like the idea of having grep be aware of 
connection-local variables even better. That's more flexible, and also 
should work for the reverse case: if you call rgrep from a Tramp file 
buffer, but change the search directory to a local path, rgrep uses 
POSIX shell-quoting. It should use MS-Windows shell-quoting in that case 
(since it's running the command on the local Windows system).

Message #32 received at 54487 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Jim Porter <jporterbugs <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 54487 <at> debbugs.gnu.org
Subject: Re: bug#54487: 29.0.50; connection-local value for
 `shell-file-name' not set in Dired buffers over Tramp
Date: Tue, 22 Mar 2022 10:44:04 +0100

Jim Porter <jporterbugs <at> gmail.com> writes:

Hi Jim,

> On 3/21/2022 7:06 AM, Michael Albinus wrote:
>> I'll wait until Jim confirms that this works in general, then I would
>> apply a patch along this spec.

I've pushed a fix to master. It is different from what I have shown
before, but shall serve as well.

> The patch you posted works for me. Setting `shell-file-name' to
> "/bin/sh" worked in my tests because it makes the function
> `w32-shell-dos-semantics' return nil, so this condition in
> `shell-quote-argument' isn't matched:
>
>   ((and (eq system-type 'windows-nt) (w32-shell-dos-semantics))
>
> That makes the shell-quoting use POSIX-style rules instead, which is
> what we want if the default-directory is remote. Reading that code, I
> think the `w32-shell-dos-semantics' part of that condition is there to
> handle things like Cygwin builds, so maybe it's not quite right to
> rely on that for the case I described in the original report. (That
> said, I think it would only be an issue for some truly esoteric
> configurations.)

Fiddling with shell-file-name doesn't help in this case, because
connection-local variables are not applied in every remote buffer, like
in dired buffers. The more general collection of Tramp-aware
connection-local variables could damage something else, that's why it is
appled on programmatic request only.

> On the other hand, I think I like the idea of having grep be aware of
> connection-local variables even better. That's more flexible, and also
> should work for the reverse case: if you call rgrep from a Tramp file
> buffer, but change the search directory to a local path, rgrep uses
> POSIX shell-quoting. It should use MS-Windows shell-quoting in that
> case (since it's running the command on the local Windows system).

Yep, I'll start now to work on this. The plan is to collect these
specific connection-local variables in an own :application, so that they
are set only with the given "grep" scope.

For the scope of this bug report, it could be closed. But I'll like to
keep it open for now in order to discuss possible problems with the
connection-local variables approach.

Best regards, Michael.

Message #37 received at 54487-done <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Jim Porter <jporterbugs <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 54487-done <at> debbugs.gnu.org
Subject: Re: bug#54487: 29.0.50; connection-local value for
 `shell-file-name' not set in Dired buffers over Tramp
Date: Wed, 23 Mar 2022 12:53:08 +0100

Version: 29.1

Michael Albinus <michael.albinus <at> gmx.de> writes:

Hi Jim,

> For the scope of this bug report, it could be closed. But I'll like to
> keep it open for now in order to discuss possible problems with the
> connection-local variables approach.

I've checked the needed changes, and it would be too invasive. In
grep.el, host specific settings are computed by global variables.
Furthermore, the compilation buffer is created only after computing
these settings.

This would require a larger rewrite of grep.el. I believe it isn't worth then.

Closing the bug.

Best regards, Michael.

Message #40 received at 54487 <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: 54487 <at> debbugs.gnu.org, michael.albinus <at> gmx.de
Subject: Re: bug#54487: 29.0.50; connection-local value for `shell-file-name'
 not set in Dired buffers over Tramp
Date: Wed, 23 Mar 2022 09:55:58 -0700

On 3/23/2022 4:53 AM, Michael Albinus wrote:
> I've checked the needed changes, and it would be too invasive. In
> grep.el, host specific settings are computed by global variables.
> Furthermore, the compilation buffer is created only after computing
> these settings.

Thanks for looking into it. I'm looking at doing this for a similar 
package, so if I can come up with a solution there, maybe I can see 
about porting it to grep.el too. If I come up with a simple way to do 
this, I'll just file another bug with a patch though.

Message #43 received at 54487 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Jim Porter <jporterbugs <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 54487 <at> debbugs.gnu.org
Subject: Re: bug#54487: 29.0.50; connection-local value for
 `shell-file-name' not set in Dired buffers over Tramp
Date: Wed, 23 Mar 2022 19:58:44 +0100

Jim Porter <jporterbugs <at> gmail.com> writes:

Hi Jim,

> On the other hand, I think I like the idea of having grep be aware of
> connection-local variables even better. That's more flexible, and also
> should work for the reverse case: if you call rgrep from a Tramp file
> buffer, but change the search directory to a local path, rgrep uses
> POSIX shell-quoting. It should use MS-Windows shell-quoting in that
> case (since it's running the command on the local Windows system).

With commit ef0a0d30c5 this shall work now, even w/o connection-local
variables.

Best regards, Michael.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.