Subject: bug#21702: closed (Re: shell-quote-argument semantics and safety)

From: [email-masked] (GNU bug Tracking System)
Date: Thu, 22 Oct 2015 03:51:02 +0000

Your bug report

#21702: shell-quote-argument semantics and safety

which was filed against the emacs package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
21702: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21702
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: shell-quote-argument semantics and safety
From: Paul Eggert <[email-masked]>
To: [email-masked]
Cc: Taylan Ulrich Bayırlı/Kammer<[email-masked]>
Date: Wed, 21 Oct 2015 20:49:37 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

I installed a patch to the Emacs manual that attempts to address the 
documentation problem, and am boldly closing the bug. The bug report can 
be reopened if more work is needed re shell-quote-argument's 
documentation. For the patch, please see:

http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=f373e812d95e1822833f88db024e011a769998b4

Subject: shell-quote-argument semantics and safety
From: [email-masked] (Taylan Ulrich Bayırlı/Kammer)
To: [email-masked]
Date: Sun, 18 Oct 2015 14:36:03 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

The documentation of shell-quote-argument only says

    Quote ARGUMENT for passing as argument to an inferior shell.

It's unclear for which shells this is supposed to work.  In a recent
thread in emacs-devel, it has been demonstrated that if the result is
passed to csh, it can allow an attacker to execute an arbitrary shell
command, although without arguments:

    (let ((argument (read untrusted-source)))
      (assert (stringp argument))
      (call-process "csh" nil t nil "-c"
                    (concat "echo " (shell-quote-argument argument))))

    ;; If untrusted-source gives us "\nevil-command\n", we get:
    ;; evil-command: Command not found.

The function should clearly document

    1) for which shells will the quoting work absolutely, i.e. lead to
    the given string to appear *verbatim* in an element of the ARGV of
    the called command,

    2) optionally, for which shells will the quoting at least prevent
    code injection,

    3) optionally, for which shells and character sets for ARGUMENT will
    the quoting work absolutely,

    4) optionally, for which shells and character sets for ARGUMENT will
    the quoting at least prevent code injection,

    5) optionally, for which shells will the quoting work at all even if
    it provides no clear semantics, such that one can at least use it
    with data coming from trusted sources (e.g. other parts of Emacs's
    source code, or the user sitting in front of Emacs), where it's the
    user's/programmer's responsibility to stick to values for ARGUMENT
    that are intuitively known to be unproblematic even if the character
    set isn't well-defined.

Currently #5 seems to be implied for all shells, for lack of further
documentation.  Possibly, the function was never meant to be used with
untrusted data, but there's no warning against doing so either.

I stress-tested the strategy it uses for POSIX shells with the following
horrible hack; the results are positive, i.e. the strategy seems to meet
the criteria #1 above for POSIX shells.

for i in {0..999}
do
    dd if=/dev/urandom of=/dev/stdout bs=1K count=1 2>/dev/null |
        tr -d '\000' > randomfile  # NULL bytes in ARGV are impossible

    emacs -q --batch --eval \
          "(with-temp-buffer 
             (insert-file-contents-literally \"randomfile\")
             (let ((data
                    (replace-regexp-in-string
                      \"\\n\" \"'\\n'\"
                      (replace-regexp-in-string 
                        \"[^-0-9a-zA-Z_./\\n]\" 
                        \"\\\\\\\\\\\\&\" 
                        (buffer-substring (point-min) (point-max))))))
               (erase-buffer)
               (insert \"printf %s \")
               (insert data)
               (write-region (point-min) (point-max) \"commandfile\")))"

    sh - < commandfile > output  # tested with bash, dash, and ksh

    diff randomfile output || exit
done

There's also wording in POSIX which seems to guarantee the safety of the
strategy:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_02_01

"A <backslash> that is not quoted shall preserve the literal value of
the following character, with the exception of a <newline>. [...]"

For now, here's a trivial patch improving the docstring.  If anyone is
confident in the safety of the function for shells other than those
conforming to POSIX sh, feel free to change the docstring accordingly.

>From dedcb603da981dcab8f576dea2f36d58fd2ddcfa Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?=
 <[email-masked]>
Date: Sun, 18 Oct 2015 14:23:35 +0200
Subject: [PATCH] * lisp/subr.el (shell-quote-argument): Improve documentation.

---
 lisp/subr.el | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/lisp/subr.el b/lisp/subr.el
index e176907..940ebe6 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -2711,7 +2711,11 @@ Note: :data and :device are currently not supported on Windows."
 (declare-function w32-shell-dos-semantics "w32-fns" nil)
 
 (defun shell-quote-argument (argument)
-  "Quote ARGUMENT for passing as argument to an inferior shell."
+  "Quote ARGUMENT for passing as argument to an inferior shell.
+
+This is safe for shells conforming to POSIX sh.  No guarantees
+regarding code injection are made for other shells, but csh,
+MS-DOS and Windows NT are supported for simple cases as well."
   (cond
    ((eq system-type 'ms-dos)
     ;; Quote using double quotes, but escape any existing quotes in
-- 
2.5.0