Subject: [PATCH] sed: do not elide an invalid byte in a substitution RHS
From: Jim Meyering <[email-masked]>
To: [email-masked]
Date: Sun, 27 Dec 2015 22:02:24 -0800
List-Id: <debbugs-submit.debbugs.gnu.org>

Just after pushing the fix for that heap-clobbering bug,
http://debbugs.gnu.org/22127, I noticed there was another minor
problem: sed would silently elide any invalid byte specified in a
substitution replacement string.

Here's a patch that's not quite complete (no NEWS entry, and I haven't
yet tried to determine when it was introduced).
From 74f6081358bfd271f679b87600367975111c7436 Mon Sep 17 00:00:00 2001
From: Jim Meyering <[email-masked]>
Date: Sun, 27 Dec 2015 21:56:04 -0800
Subject: [PATCH] sed: do not elide an invalid byte in a substitution RHS

Now, sed copies such bytes into the output.
* sed/execute.c (str_append_modified): Copy each invalid byte
into the result string, as the comment suggests.
* testsuite/invalid-mb-seq-UMR.sh: Adjust this test to conform.
Perl does the same thing.  Also, remove an unnecessary "-e" option.
---
 sed/execute.c                   | 1 +
 testsuite/invalid-mb-seq-UMR.sh | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/sed/execute.c b/sed/execute.c
index 5bebfc2..93b5650 100644
--- a/sed/execute.c
+++ b/sed/execute.c
@@ -225,6 +225,7 @@ str_append_modified(struct line *to, const char *string, size_t length,
               return;
             }

+          str_append (to, string, 1);
           memset (&to->mbstate, 0, sizeof (from_stat));
           n = 1;
           string += n, length -= n;
diff --git a/testsuite/invalid-mb-seq-UMR.sh b/testsuite/invalid-mb-seq-UMR.sh
index f631e24..0c17b9a 100755
--- a/testsuite/invalid-mb-seq-UMR.sh
+++ b/testsuite/invalid-mb-seq-UMR.sh
@@ -24,9 +24,9 @@ require_valgrind_
 test "$LOCALE_JA" = none && skip_ found no Japanese EUC locale

 echo a > in || framework_failure_
-echo bC > exp || framework_failure_
+printf 'b\262C\n' > exp || framework_failure_
 LC_ALL=$LOCALE_JA valgrind --quiet --error-exitcode=1 \
-  sed -e 's/a/b\U\xb2c/' in > out 2> err || fail=1
+  sed 's/a/b\U\xb2c/' in > out 2> err || fail=1

 # Work around a bug in CentOS 5.10's valgrind
 # FIXME: remove in 2018 or when CentOS 5 is no longer officially supported
-- 
2.6.2