Subject: Re: bug#30794: Cannot produce backslash through hexadecimal - sed (GNU sed) 4.2.2

From: Assaf Gordon <[email-masked]>
Date: Tue, 13 Mar 2018 20:45:32 -0600

Hello,

On Tue, Mar 13, 2018 at 06:02:05PM +0800, 0xddaa wrote:
> There is the Proof-of-Concept:
> 
> # echo z | sed 's/z/\x5c/' # \x5c == '\'
> \ # success
> # echo z | sed 's/z/\x5c\x61/'
> a # backslash is gone
> 
> According the manual, *\xxx* should produce an ascii character. sed should
> not treat *\x5c* as an escape symbol.

Thank you for reporting this issue with such clear details
and providing a bug fix - much appreciated!

I expanded your fix by adding tests and news item, see attached patch.

There is a small issue that this is a backwards incompatible change,
and might break some exotic (bad) scripts.

I'll wait for other opinions before pushing this.

Jim,
What do you think?

regards,
 - assaf
>From 9946f6d4177d65ac62b963af391c1c8ad022f7fa Mon Sep 17 00:00:00 2001
From: 0xddaa <[email-masked]>
Date: Tue, 13 Mar 2018 22:34:26 -0400
Subject: [PATCH] sed: treat '\x5c' as literal backslash

Sed incorrectly treated '\x5c' (ASCII 92, backslash) as an escape character.

Old behavior:
  $ echo z | sed -E 's/(z)/\x5c1/' # identical to 's/(z)/\1/'
  z

New behavior:
  $ echo z | sed -E 's/(z)/\x5c1/'
  \1

Reported in https://bugs.gnu.org/30794.

* NEWS: Mention bug fix.
* sed/compile.c (normalize_text): Pass backslash as literal character.
* testsuite/misc.pl: Add tests.
---
 NEWS              | 10 ++++++++++
 sed/compile.c     |  2 +-
 testsuite/misc.pl |  7 +++++++
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/NEWS b/NEWS
index e762b2d..f9618e6 100644
--- a/NEWS
+++ b/NEWS
@@ -15,6 +15,16 @@ GNU sed NEWS                                    -*- outline -*-
   instead of the symlink target. [Bug present since at least sed-4.2]
   sed -i --follow-symlinks remains unchanged.
 
+  sed now treats the sequence '\x5c' (ASCII 92, backslash) as literal
+  backslash character, not as an escape prefix character.
+  [Bug present since at least sed-4.0.6]
+  Old behavior:
+     $ echo z | sed -E 's/(z)/\x5c1/' # identical to 's/(z)/\1/'
+     z
+  New behavior:
+     $ echo z | sed -E 's/(z)/\x5c1/'
+     \1
+
 
 * Noteworthy changes in release 4.4 (2017-02-03) [stable]
 
diff --git a/sed/compile.c b/sed/compile.c
index 26bd18a..25baf5e 100644
--- a/sed/compile.c
+++ b/sed/compile.c
@@ -1481,7 +1481,7 @@ convert:
             p = convert_number(&ch, p, bufend, base);
 
             /* for an ampersand in a replacement, pass the \ up one level */
-            if (buftype == TEXT_REPLACEMENT && ch == '&')
+            if (buftype == TEXT_REPLACEMENT && (ch == '&' || ch == '\\'))
               *q++ = '\\';
             *q++ = ch;
             continue;
diff --git a/testsuite/misc.pl b/testsuite/misc.pl
index 92838cb..3fa5b84 100644
--- a/testsuite/misc.pl
+++ b/testsuite/misc.pl
@@ -1190,6 +1190,13 @@ s,.*[^\/],,
            . "baaaacx\n"}
      ],
 
+
+     # Four backslashes (2 pairs of "\\") to pass through two interpolations:
+     # once in Perl, then the shell command line argument.
+     # sed will see one backslash character in the s/// command.
+     ['bug30794_1', "s/z/\\\\x5cA/",  {IN=>'z'}, {OUT => "\\A"}],
+     ['bug30794_2', "s/z/\\\\x5c/",   {IN=>'z'}, {OUT => "\\"}],
+     ['bug30794_3', "s/z/\\\\x5c1/",  {IN=>'z'}, {OUT => "\\1"}],
     );
 
 my $save_temps = $ENV{SAVE_TEMPS};
-- 
2.7.4