From unknown Fri Jun 20 18:21:43 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#32082 <32082@debbugs.gnu.org> To: bug#32082 <32082@debbugs.gnu.org> Subject: Status: heap buffer overflow in sed/execute.c, line 992 Reply-To: bug#32082 <32082@debbugs.gnu.org> Date: Sat, 21 Jun 2025 01:21:43 +0000 retitle 32082 heap buffer overflow in sed/execute.c, line 992 reassign 32082 sed submitter 32082 bugs@feusi.co severity 32082 normal tag 32082 fixed thanks From debbugs-submit-bounces@debbugs.gnu.org Sat Jul 07 10:00:21 2018 Received: (at submit) by debbugs.gnu.org; 7 Jul 2018 14:00:21 +0000 Received: from localhost ([127.0.0.1]:50222 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fbnl4-0004t4-Or for submit@debbugs.gnu.org; Sat, 07 Jul 2018 10:00:19 -0400 Received: from eggs.gnu.org ([208.118.235.92]:52253) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fbkxC-0006ru-3W for submit@debbugs.gnu.org; Sat, 07 Jul 2018 07:00:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fbkx5-0007qS-T9 for submit@debbugs.gnu.org; Sat, 07 Jul 2018 07:00:32 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:33045) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fbkx5-0007qL-NS for submit@debbugs.gnu.org; Sat, 07 Jul 2018 07:00:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43219) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fbkx2-0006ls-Tr for bug-sed@gnu.org; Sat, 07 Jul 2018 07:00:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fbkwy-0007m8-3K for bug-sed@gnu.org; Sat, 07 Jul 2018 07:00:29 -0400 Received: from sender-of-o52.zoho.eu ([31.186.226.248]:21396) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fbkwx-0007kG-Hp for bug-sed@gnu.org; Sat, 07 Jul 2018 07:00:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1530961215; s=zoho; d=feusi.co; i=bugs@feusi.co; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; l=3336; bh=19F8JUWASYJIEeDHYyWKeTbDy5SZ0LCYLv6sraF+RAw=; b=EcG5CTfYyoV3PEQBtSe/2rVd0NE0WEC2Qn8HGgPFOkI9uuJmVwseyZOYdZMyaQow BpDDAo57/Iqs5+KZZP2+7DFoh/JvXUzPPuwBz6OTmwwq85ulAmE768oUN01pbOcX2xl BkeL6gH8GaFX9J8xH2EVrG8n8ovDoIoZl6fp+SD8= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=zoho; d=feusi.co; h=date:from:to:subject:message-id:mime-version:content-type:user-agent; b=iO2xOYXZJagpcMY3jMVZGYIbPk9JKRkc5uJ9CDPUb+p6buFAB3esI75rRWe5IU00IpFU0smYLhqi qisaeeqwr0KYRNe6R/ji4KbooqKi9f13xVjIwfxiN7YkQll/tw1PUOE1EM35wxsKP5frTFJ0hSG/ ugziLbreBwIfizDLyyY= Received: from feusi.co (194.191.249.240 [194.191.249.240]) by mx.zoho.eu with SMTPS id 1530961214954550.7420637175951; Sat, 7 Jul 2018 13:00:14 +0200 (CEST) Date: Sat, 7 Jul 2018 13:01:14 +0200 From: bugs@feusi.co To: bug-sed@gnu.org Subject: heap buffer overflow in sed/execute.c, line 992 Message-ID: <20180707110031.GA19713@feusi.co> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="GID0FwUMdk1T2AWN" Content-Disposition: inline User-Agent: Mutt/1.10.0 (2018-05-17) X-Zoho-Virus-Status: 1 X-ZohoMailClient: External X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Sat, 07 Jul 2018 10:00:18 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) --GID0FwUMdk1T2AWN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, I am working on a project in which I use the afl fuzzer to fuzz different open-source software. In doing so, I discovered a heap buffer overflow in sed/execute.c, line 992. Following is a detailed backtrace: ================================================================= ==18674==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000fb8 at pc 0x55d4f61c3909 bp 0x7ffc4fa90580 sp 0x7ffc4fa90578 READ of size 8 at 0x602000000fb8 thread T0 #0 0x55d4f61c3908 in append_replacement sed/execute.c:992 #1 0x55d4f61c3908 in do_subst sed/execute.c:1071 #2 0x55d4f61c3908 in execute_program sed/execute.c:1507 #3 0x55d4f61c516a in process_files sed/execute.c:1677 #4 0x55d4f619c649 in main sed/sed.c:377 #5 0x7f15904d2a86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21a86) #6 0x55d4f619d119 in _start (/home/jefeus/sed/sed/sed+0xc119) Address 0x602000000fb8 is a wild pointer. SUMMARY: AddressSanitizer: heap-buffer-overflow sed/execute.c:992 in append_replacement Shadow bytes around the buggy address: 0x0c047fff81a0: fa fa 00 00 fa fa 00 00 fa fa 01 fa fa fa fd fd 0x0c047fff81b0: fa fa 00 00 fa fa 00 00 fa fa 01 fa fa fa 00 00 0x0c047fff81c0: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00 0x0c047fff81d0: fa fa 00 00 fa fa 00 00 fa fa fd fd fa fa fd fa 0x0c047fff81e0: fa fa 00 00 fa fa 00 00 fa fa 00 fa fa fa 00 fa =>0x0c047fff81f0: fa fa fa fa fa fa fa[fa]fa fa fa fa fa fa fa fa 0x0c047fff8200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff8210: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff8220: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff8230: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff8240: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==18674==ABORTING To reproduce this bug run sed with: sed -f Where and are the two files attached. These files demonstrate how this bug can be used to print undefined memory. By replacing the first line in with a long string of "A"s, a segmentation fault can be produced. I was able to reproduce this bug with the current git version, the debian version and on arch linux. cheers, project-repo --GID0FwUMdk1T2AWN Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=script-file 10s11\91 \0^0s1011 \00a \0^0s111w0 --GID0FwUMdk1T2AWN Content-Type: application/octet-stream Content-Disposition: attachment; filename=input-file Content-Transfer-Encoding: quoted-printable =00=00=00=00'=10=00=00=0A=0A=0A=0A=0A=0A=0A=0A=0AAAAAAAAAAAAAAAAAAAAAAAAAAA= AAAAAAAAAAAAAAAAAAAAAAAA=0A=0A --GID0FwUMdk1T2AWN-- From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 08 00:28:58 2018 Received: (at 32082) by debbugs.gnu.org; 8 Jul 2018 04:28:58 +0000 Received: from localhost ([127.0.0.1]:50603 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fc1Jb-0008Q5-DH for submit@debbugs.gnu.org; Sun, 08 Jul 2018 00:28:58 -0400 Received: from mail-pl0-f53.google.com ([209.85.160.53]:33719) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fc1JV-0008Pl-0s for 32082@debbugs.gnu.org; Sun, 08 Jul 2018 00:28:48 -0400 Received: by mail-pl0-f53.google.com with SMTP id 6-v6so4554048plb.0 for <32082@debbugs.gnu.org>; Sat, 07 Jul 2018 21:28:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language; bh=1SaKQudKmEU2TiHYajTlHQz9ojUswddplKvDTlsI9js=; b=biE7B4/PvJWVQ6FzNbWE8MTELDDaFIAG5uZi1Z1Z+GyhL6kO9JNKoDYmrhGF9XNht6 xGYUYpL0gYzKnn593cPg0ju6mhGERbUY3L04p2fZeNg9TKO/cExLC1NpQvP2dwltf38R wCvtJTaNMG6FkktKmEcRIzMs7SZJ/9c4fJhZPCeTaDjbP+oSJSEpKo9Lj4w1Ouuh/Jas l+ps8v2tsCUR0WxK5ETuirz3ZK3LKcO4TPGZpejQinZtsvP+T94UUV28b7h31rB/ttLB XQ9IPd2nsRcIbTwY4urBlbDpSjTH4AEbLKZcwx2duyH33eYSlRqA2LGAvxIlZumhaoV/ emqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=1SaKQudKmEU2TiHYajTlHQz9ojUswddplKvDTlsI9js=; b=gxsZzofCxDFq5u4/orIfIyKI2LmnpAAj6pUfJyp1u3ecpD0Sg3c3OCpVjbqAA9GSiC XUPg7RR2mSXvKf9zPCFIvBf+L54fNmrYSRlMeaP5FGybE+cZTjPH2FNA0ayLN7kXWLWu XVEUbXa6qlWrnuS3ihlXCccGfSqmrBeehv5tNdPEH2dIY9dgQUA9tBAbTmOBONFc/bXd ARdTQACmis/h5APzF1o642m0ZgqIAXnihkpArelnhtxXL/NcI3tgfY52iBOnJew4tnM6 sSSACTtZ5pFwH6dhTFOB+wdhessSARwn282Zg31LyIk/CFUIWnYY+InjP9uNf/SqkNLS hLFA== X-Gm-Message-State: APt69E3pxZuj+BFNF+gHuLlv+EHqMT1Y9t43FviElzM0LLBGrL1F1wdV BBoH7oLJ32IsUwpZvfvSTEhCoNjf X-Google-Smtp-Source: AAOMgpdf2ShhBMhj25U9WXYmFtOiGnFTPjTiZHaXlZxXlcb2KfXNz7ELqwN9kBgezT5RfaC0+4k73A== X-Received: by 2002:a17:902:8497:: with SMTP id c23-v6mr16202008plo.124.1531024118708; Sat, 07 Jul 2018 21:28:38 -0700 (PDT) Received: from tomato.housegordon.com (moose.housegordon.com. [184.68.105.38]) by smtp.googlemail.com with ESMTPSA id b73-v6sm548051pfl.152.2018.07.07.21.28.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 07 Jul 2018 21:28:37 -0700 (PDT) Subject: Re: bug#32082: heap buffer overflow in sed/execute.c, line 992 To: bugs@feusi.co, 32082@debbugs.gnu.org References: <20180707110031.GA19713@feusi.co> From: Assaf Gordon Message-ID: <916db1d1-f158-fb30-76ef-e9c6f76c40f2@gmail.com> Date: Sat, 7 Jul 2018 22:28:36 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <20180707110031.GA19713@feusi.co> Content-Type: multipart/mixed; boundary="------------F61D2B4EC4D9F53231AD1D1A" Content-Language: en-US X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 32082 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) This is a multi-part message in MIME format. --------------F61D2B4EC4D9F53231AD1D1A Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hello, On 07/07/18 05:01 AM, bugs@feusi.co wrote: > I am working on a project in which I use the afl fuzzer to fuzz > different open-source software. In doing so, I discovered a > heap buffer overflow in sed/execute.c, line 992. Thank you for this interesting bug report, and for providing such easy way to reproduce. I can confirm this is reproducible, and in fact is a very old bug! fantastic work. It took some time and lots of squinting to track it down, so I'll write it here in details for others. Your sed script file was: ==== 10s11\91 \0^0s1011 \00a \0^0s111w0 ==== The input file content doesn't matter for the bug, so I won't focus on it. Interested readers - If you like challenges, stop reading this email and spend couple of minutes trying to understand the above script. fun a-plenty :) I assume that all the 1's and 0's are because AFL mutated bit by bit until something was triggered. They aren't critical for the bug, so here's a simpler and more concise buggy program: ==== seq 2 | valgrind sed -e '/^/s///p ; 2s//\9/' ==== It might still not be immediately clear why there is a bug here. An even simpler version of the above program does not seem buggy at first, because the invalid backref is detected: === $ seq 2 | sed -e '/^/p; 2s//\9/' 1 1 2 sed: -e expression #1, char 0: invalid reference \9 on `s' command === However, this detection suppressed under "--posix", so the bug is actually there: ==== $ seq 2 | valgrind sed --posix -e '/^/p; 2s//\9/' 1 1 2 ==19663== Invalid read of size 4 ==19663== at 0x10FD51: ??? (in /bin/sed) ==19663== by 0x110402: ??? (in /bin/sed) ==19663== by 0x10B106: ??? (in /bin/sed) ==19663== by 0x50802E0: (below main) (libc-start.c:291) ==19663== Address 0x5a9df74 is 20 bytes after a block of size 16 in [....] ==== The novelty of this bug report is that using few more "previous regexes" and the ^/$ optimization [1], the safety check is bypassed even when not using "--posix". [1] https://git.savannah.gnu.org/cgit/sed.git/commit/?id=6dea75e7 Attached is a suggested fix. It seems very simple, but I encourage everyone to double-check my logic and perhaps detect more problems. comments very welcomed, - assaf P.S. 1. You haven't provided a name, so I credited you as "bugs@feusi.co" in the commit message. If you'd like something else, let me know. 2. Out of curiosity, are you using stock AFL or a modified one? If stock AFL, how long did you run it before something was triggered, and how was it configured (and which input files) ? I used AFL on previous version of sed for 24 hours without any bug detected. --------------F61D2B4EC4D9F53231AD1D1A Content-Type: text/x-patch; name="0001-sed-fix-heap-buffer-overflow-from-invalid-references.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0001-sed-fix-heap-buffer-overflow-from-invalid-references.pa"; filename*1="tch" >From 1644a080b8a22aa36ff187218f9895a06127efdd Mon Sep 17 00:00:00 2001 From: Assaf Gordon Date: Sat, 7 Jul 2018 22:03:38 -0600 Subject: [PATCH] sed: fix heap buffer overflow from invalid references Under certain conditions sed would access invalid memory based on the requested back-reference (e.g. "s//\9/" would access the 9th element in the regex registers without checking it is at least 9 element in size). The following examples would trigger valgrind errors: seq 2 | valgrind sed -e '/^/s///p ; 2s//\9/' seq 2 | valgrind sed --posix -e '/2/p ; 2s//\9/' Reported by bugs@feusi.co in https://lists.gnu.org/r/bug-sed/2018-07/msg00004.html . * NEWS: Mention the bugfix. * sed/execute.c (append_replacement): Check number of allocated regex replacement registers before accessing the array. * sed/testsuite/bug32082.sh: Test sed for this behaviour under valgrind. * sed/testsuite/local.mk (T): Add new test. --- NEWS | 5 ++++ sed/execute.c | 2 +- testsuite/bug32082.sh | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++ testsuite/local.mk | 1 + 4 files changed, 89 insertions(+), 1 deletion(-) create mode 100755 testsuite/bug32082.sh diff --git a/NEWS b/NEWS index 6c2fbaf..0c50526 100644 --- a/NEWS +++ b/NEWS @@ -2,6 +2,11 @@ GNU sed NEWS -*- outline -*- * Noteworthy changes in release ?.? (????-??-??) [?] +** Bug fixes + + sed no longer accesses invalid memory (heap overflow) when given invalid + references in 's' command [bug#32082, present at least since sed-4.0.6]. + * Noteworthy changes in release 4.5 (2018-03-31) [stable] diff --git a/sed/execute.c b/sed/execute.c index 2804c5e..7a4850f 100644 --- a/sed/execute.c +++ b/sed/execute.c @@ -987,7 +987,7 @@ static void append_replacement (struct line *buf, struct replacement *p, curr_type &= ~REPL_MODIFIERS; } - if (0 <= i) + if (0 <= i && i < regs->num_regs) { if (regs->end[i] == regs->start[i] && p->repl_type & REPL_MODIFIERS) /* Save this modifier, we shall apply it later. diff --git a/testsuite/bug32082.sh b/testsuite/bug32082.sh new file mode 100755 index 0000000..6a77656 --- /dev/null +++ b/testsuite/bug32082.sh @@ -0,0 +1,82 @@ +#!/bin/sh +# sed may access to uninitialized memory whe invalid backreference +# are used under certain circumstances. +# Before sed 4.6 these would result in "Invalid read size of 4" reported +# by valgrind from execute.c:992 + +# Copyright (C) 2018 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +. "${srcdir=.}/testsuite/init.sh"; path_prepend_ ./sed +print_ver_ sed + +require_valgrind_ + +printf '1\n2\n' > in || framework_failure_ +printf '1\n2\n\n' > exp-posix || framework_failure_ +printf '1\n1\n2\n2\n' > exp-no-posix || framework_failure_ + +# +# Test 1: with "--posix" +# +# using "--posix" disables the backref safety check in +# regexp.c:compile_regex_1(), which is reported as: +# "invalid reference \\%d on `s' command's RHS" + +valgrind --quiet --error-exitcode=1 \ + sed --posix -e '/2/p ; 2s//\9/' in > out-posix 2> err-posix || fail=1 + +echo "valgrind report for 'posix' test:" +echo "==================================" +cat err-posix +echo "==================================" + + +# Work around a bug in CentOS 5.10's valgrind +# FIXME: remove in 2018 or when CentOS 5 is no longer officially supported +grep 'valgrind: .*Assertion.*failed' err-posix > /dev/null \ + && skip_ 'you seem to have a buggy version of valgrind' + +compare exp-posix out-posix || fail=1 +compare /dev/null err || fail=1 + + + +# +# Test 2: without "--posix" +# +# When not using "--posix", using a backref to a non-existing group. +# would be caught in compile_regex_1. +# As reported in bugs.gnu.org/32082 by bugs@feusi.co, +# using the recent begline/endline optimization with few "previous regex" +# tricks bypasses this check. + +valgrind --quiet --error-exitcode=1 \ + sed -e '/^/s///p ; 2s//\9/' in > out-no-posix 2> err-no-posix || fail=1 + +echo "valgrind report for 'no-posix' test:" +echo "====================================" +cat err-no-posix +echo "====================================" + +# Work around a bug in CentOS 5.10's valgrind +# FIXME: remove in 2018 or when CentOS 5 is no longer officially supported +grep 'valgrind: .*Assertion.*failed' err-no-posix > /dev/null \ + && skip_ 'you seem to have a buggy version of valgrind' + +compare exp-no-posix out-no-posix || fail=1 +compare /dev/null err || fail=1 + + +Exit $fail diff --git a/testsuite/local.mk b/testsuite/local.mk index b4a4f5a..bf4559f 100644 --- a/testsuite/local.mk +++ b/testsuite/local.mk @@ -43,6 +43,7 @@ LOG_COMPILER = false T = \ testsuite/misc.pl \ + testsuite/bug32082.sh \ testsuite/cmd-l.sh \ testsuite/cmd-R.sh \ testsuite/colon-with-no-label.sh \ -- 2.11.0 --------------F61D2B4EC4D9F53231AD1D1A-- From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 08 03:40:46 2018 Received: (at 32082) by debbugs.gnu.org; 8 Jul 2018 07:40:46 +0000 Received: from localhost ([127.0.0.1]:50629 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fc4JB-0004i3-FC for submit@debbugs.gnu.org; Sun, 08 Jul 2018 03:40:46 -0400 Received: from sender-of-o52.zoho.eu ([31.186.226.248]:21307) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fc4J6-0004hs-6u for 32082@debbugs.gnu.org; Sun, 08 Jul 2018 03:40:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1531035627; s=zoho; d=feusi.co; i=bugs@feusi.co; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; l=10150; bh=y2bzz76XpChm6tCdU91GBTYjYGUjscK5/kassgPNc1w=; b=GQJBMrH7hDw9NA6QltmEe3QX9VRkZIHd3XoWJfPbftfZbPPeMYx9XA9gxap/3gzA naSPlJJ8tNkF7FRUV7L1PZPn3hWn5+nsv09etVctueWXtFVpJZ7VuQNUDAnsdzvMHk8 xB0hWUBERh9wqbsLrtQ5YrPXAbg3BxQcN1ZlDZK0= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=zoho; d=feusi.co; h=date:from:to:subject:message-id:mime-version:content-type:user-agent; b=gKUqgcH0g8+QbJEy/fajys2ntENWYSHikbsn2HNjONvBhsCMgo6+tABN79Nkx2HPltUOJ12R1x3g r16Tw7V5Om4CtPzy0l6b8WooJhdiU7A6vp4i8cra9paGb6vQbMkdxq5ep5yIZYuFLsfWoIcgKxub Cy6xpW1yvyJbD0xDFyc= Received: from feusi.co (194.191.249.240 [194.191.249.240]) by mx.zoho.eu with SMTPS id 1531035627557771.6259121265664; Sun, 8 Jul 2018 09:40:27 +0200 (CEST) Date: Sun, 8 Jul 2018 09:41:26 +0200 From: project-repo To: Assaf Gordon , 32082@debbugs.gnu.org Subject: Re: bug#32082: heap buffer overflow in sed/execute.c, line 992 Message-ID: <20180708074126.GA24781@feusi.co> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.10.0 (2018-05-17) X-ZohoMailClient: External X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 32082 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, Thanks for the quick response. that patch seems to work at first glance. I'll keep fuzzing and tell you if I find anything interesting. Oh yes, about my name: I had some problems once when I reported a bug using my real name, with people sending me phishing emails and things, so I set up this email address which I could use for reporting bugs. Crediting this bug to "bugs@feusi.co" is perfect, thanks. Then, to your second question: I actually used an off-the-shelf afl fuzzer for this fuzzing process. The first crashes started to appear about 6 hours after I started. The setup was as follows: I ran 6 parallel fuzzers which each called sed as "sed -f /dev/stdin testfile". The file "testfile" was a rather large file piped from /dev/urandom, which aided the fuzzing process, as the bug I reported only causes a segmentation fault when the "testfile" has specific properties. I didn't compile sed with address sanitizer as it produced a bunch of false positives when I tried to use it. The fuzzing input file was a complex regex string which I pulled off the internet somewhere, although it does not bear much resemblance to the buggy file, so I don't think it mattered so much. Thanks again for fixing this so quickly and analysing this bug in such detail. This will be very helpful for my project. :) cheers, project-repo In-Reply-To: <916db1d1-f158-fb30-76ef-e9c6f76c40f2@gmail.com> On Sat, Jul 07, 2018 at 10:28:36PM -0600, Assaf Gordon wrote: > Hello, > > On 07/07/18 05:01 AM, bugs@feusi.co wrote: > > I am working on a project in which I use the afl fuzzer to fuzz > > different open-source software. In doing so, I discovered a > > heap buffer overflow in sed/execute.c, line 992. > > Thank you for this interesting bug report, and for providing such easy > way to reproduce. I can confirm this is reproducible, and in fact is > a very old bug! fantastic work. > > It took some time and lots of squinting to track it down, so I'll write > it here in details for others. > > Your sed script file was: > ==== > 10s11\91 > \0^0s1011 > \00a > \0^0s111w0 > ==== > > The input file content doesn't matter for the bug, so I won't focus on it. > > Interested readers - If you like challenges, stop reading this email and > spend couple of minutes trying to understand the above script. > fun a-plenty :) > > I assume that all the 1's and 0's are because AFL mutated bit by bit > until something was triggered. They aren't critical for the bug, so > here's a simpler and more concise buggy program: > > ==== > seq 2 | valgrind sed -e '/^/s///p ; 2s//\9/' > ==== > > It might still not be immediately clear why there is a bug here. > An even simpler version of the above program does not seem buggy at > first, because the invalid backref is detected: > === > $ seq 2 | sed -e '/^/p; 2s//\9/' > 1 > 1 > 2 > sed: -e expression #1, char 0: invalid reference \9 on `s' command > === > > However, this detection suppressed under "--posix", so the bug is > actually there: > ==== > $ seq 2 | valgrind sed --posix -e '/^/p; 2s//\9/' > > 1 > 1 > 2 > ==19663== Invalid read of size 4 > ==19663== at 0x10FD51: ??? (in /bin/sed) > ==19663== by 0x110402: ??? (in /bin/sed) > ==19663== by 0x10B106: ??? (in /bin/sed) > ==19663== by 0x50802E0: (below main) (libc-start.c:291) > ==19663== Address 0x5a9df74 is 20 bytes after a block of size 16 in > [....] > ==== > > The novelty of this bug report is that using few more "previous regexes" > and the ^/$ optimization [1], the safety check is bypassed even when not > using "--posix". > [1] https://git.savannah.gnu.org/cgit/sed.git/commit/?id=6dea75e7 > > > Attached is a suggested fix. > It seems very simple, but I encourage everyone to double-check my logic > and perhaps detect more problems. > > comments very welcomed, > - assaf > > > P.S. > 1. You haven't provided a name, so I credited you as "bugs@feusi.co" in > the commit message. If you'd like something else, let me know. > > 2. Out of curiosity, are you using stock AFL or a modified one? > If stock AFL, how long did you run it before something was triggered, > and how was it configured (and which input files) ? > I used AFL on previous version of sed for 24 hours without any bug detected. > > > > > > > > From 1644a080b8a22aa36ff187218f9895a06127efdd Mon Sep 17 00:00:00 2001 > From: Assaf Gordon > Date: Sat, 7 Jul 2018 22:03:38 -0600 > Subject: [PATCH] sed: fix heap buffer overflow from invalid references > > Under certain conditions sed would access invalid memory based on > the requested back-reference (e.g. "s//\9/" would access the 9th element > in the regex registers without checking it is at least 9 element in > size). > > The following examples would trigger valgrind errors: > seq 2 | valgrind sed -e '/^/s///p ; 2s//\9/' > seq 2 | valgrind sed --posix -e '/2/p ; 2s//\9/' > > Reported by bugs@feusi.co in > https://lists.gnu.org/r/bug-sed/2018-07/msg00004.html . > > * NEWS: Mention the bugfix. > * sed/execute.c (append_replacement): Check number of allocated regex > replacement registers before accessing the array. > * sed/testsuite/bug32082.sh: Test sed for this behaviour under valgrind. > * sed/testsuite/local.mk (T): Add new test. > --- > NEWS | 5 ++++ > sed/execute.c | 2 +- > testsuite/bug32082.sh | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++ > testsuite/local.mk | 1 + > 4 files changed, 89 insertions(+), 1 deletion(-) > create mode 100755 testsuite/bug32082.sh > > diff --git a/NEWS b/NEWS > index 6c2fbaf..0c50526 100644 > --- a/NEWS > +++ b/NEWS > @@ -2,6 +2,11 @@ GNU sed NEWS -*- outline -*- > > * Noteworthy changes in release ?.? (????-??-??) [?] > > +** Bug fixes > + > + sed no longer accesses invalid memory (heap overflow) when given invalid > + references in 's' command [bug#32082, present at least since sed-4.0.6]. > + > > * Noteworthy changes in release 4.5 (2018-03-31) [stable] > > diff --git a/sed/execute.c b/sed/execute.c > index 2804c5e..7a4850f 100644 > --- a/sed/execute.c > +++ b/sed/execute.c > @@ -987,7 +987,7 @@ static void append_replacement (struct line *buf, struct replacement *p, > curr_type &= ~REPL_MODIFIERS; > } > > - if (0 <= i) > + if (0 <= i && i < regs->num_regs) > { > if (regs->end[i] == regs->start[i] && p->repl_type & REPL_MODIFIERS) > /* Save this modifier, we shall apply it later. > diff --git a/testsuite/bug32082.sh b/testsuite/bug32082.sh > new file mode 100755 > index 0000000..6a77656 > --- /dev/null > +++ b/testsuite/bug32082.sh > @@ -0,0 +1,82 @@ > +#!/bin/sh > +# sed may access to uninitialized memory whe invalid backreference > +# are used under certain circumstances. > +# Before sed 4.6 these would result in "Invalid read size of 4" reported > +# by valgrind from execute.c:992 > + > +# Copyright (C) 2018 Free Software Foundation, Inc. > + > +# This program is free software: you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation, either version 3 of the License, or > +# (at your option) any later version. > + > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > + > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see . > +. "${srcdir=.}/testsuite/init.sh"; path_prepend_ ./sed > +print_ver_ sed > + > +require_valgrind_ > + > +printf '1\n2\n' > in || framework_failure_ > +printf '1\n2\n\n' > exp-posix || framework_failure_ > +printf '1\n1\n2\n2\n' > exp-no-posix || framework_failure_ > + > +# > +# Test 1: with "--posix" > +# > +# using "--posix" disables the backref safety check in > +# regexp.c:compile_regex_1(), which is reported as: > +# "invalid reference \\%d on `s' command's RHS" > + > +valgrind --quiet --error-exitcode=1 \ > + sed --posix -e '/2/p ; 2s//\9/' in > out-posix 2> err-posix || fail=1 > + > +echo "valgrind report for 'posix' test:" > +echo "==================================" > +cat err-posix > +echo "==================================" > + > + > +# Work around a bug in CentOS 5.10's valgrind > +# FIXME: remove in 2018 or when CentOS 5 is no longer officially supported > +grep 'valgrind: .*Assertion.*failed' err-posix > /dev/null \ > + && skip_ 'you seem to have a buggy version of valgrind' > + > +compare exp-posix out-posix || fail=1 > +compare /dev/null err || fail=1 > + > + > + > +# > +# Test 2: without "--posix" > +# > +# When not using "--posix", using a backref to a non-existing group. > +# would be caught in compile_regex_1. > +# As reported in bugs.gnu.org/32082 by bugs@feusi.co, > +# using the recent begline/endline optimization with few "previous regex" > +# tricks bypasses this check. > + > +valgrind --quiet --error-exitcode=1 \ > + sed -e '/^/s///p ; 2s//\9/' in > out-no-posix 2> err-no-posix || fail=1 > + > +echo "valgrind report for 'no-posix' test:" > +echo "====================================" > +cat err-no-posix > +echo "====================================" > + > +# Work around a bug in CentOS 5.10's valgrind > +# FIXME: remove in 2018 or when CentOS 5 is no longer officially supported > +grep 'valgrind: .*Assertion.*failed' err-no-posix > /dev/null \ > + && skip_ 'you seem to have a buggy version of valgrind' > + > +compare exp-no-posix out-no-posix || fail=1 > +compare /dev/null err || fail=1 > + > + > +Exit $fail > diff --git a/testsuite/local.mk b/testsuite/local.mk > index b4a4f5a..bf4559f 100644 > --- a/testsuite/local.mk > +++ b/testsuite/local.mk > @@ -43,6 +43,7 @@ LOG_COMPILER = false > > T = \ > testsuite/misc.pl \ > + testsuite/bug32082.sh \ > testsuite/cmd-l.sh \ > testsuite/cmd-R.sh \ > testsuite/colon-with-no-label.sh \ > -- > 2.11.0 > From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 08 12:36:56 2018 Received: (at 32082) by debbugs.gnu.org; 8 Jul 2018 16:36:56 +0000 Received: from localhost ([127.0.0.1]:51299 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fcCgB-0002d9-RQ for submit@debbugs.gnu.org; Sun, 08 Jul 2018 12:36:56 -0400 Received: from mail-wm0-f49.google.com ([74.125.82.49]:36168) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fcCg9-0002cv-WA for 32082@debbugs.gnu.org; Sun, 08 Jul 2018 12:36:55 -0400 Received: by mail-wm0-f49.google.com with SMTP id s14-v6so18410222wmc.1 for <32082@debbugs.gnu.org>; Sun, 08 Jul 2018 09:36:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=zP3kuyGVxxe36bPCUl5vIs4vO2IZUVcAUkwL7wpX+Bw=; b=pucljhVuYxaO2eXEmBOjmGl81n0DB2gIjBZZyZXQbtbEwOt3RPlORaUHf+Xu/iK12X wi0QdRokU2+XPCW5Z1b3JKtaVahkiXcuaLP3OTZqxmy/V8f0Vv6GHF6JstNh8w2/NoDk 5ogqAJasiOCIfR6OWEC7hLRn85sPN7mo7lEQrrVT9Oe968aXIFDNzx/7GYK186Zjahzn cCXQsOdCmq/3XbixdqQjqUQMG+g9Ts/GvMoZtnW/opXioQlJ5iWRAiM6XB2fKepHXR0x 1UStQqxJU5r7SjEP0fOKQ2sJ4xOQTVz3NhHrg7UqHKwmXN5Ki514/yZnU9s8ijn/GpH3 yaCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=zP3kuyGVxxe36bPCUl5vIs4vO2IZUVcAUkwL7wpX+Bw=; b=fTN3gL9fR5khfsApoykxQ9fXr5xn//2aw/q0jRf7dRbh0qYhwxBrbMDQXxKn8KvMN3 AS2a+L/+smu0Z0cOSBYSwy37IOmubYX9M+aoG6/UjhvA6ENFwzsfyAGYGV6Fm/5S0BwQ ruxJs5RFjkF28YNm4c5jSEYieurDQJx2wyldBIC/WqN89FaKUn/2rRPTjh9OgznTe17A ZoFF2U+VLGMinSP3d5R20m5TVbRZId2CfdrxTDceSvYazStlgVfe8g5oULFxKNxYQBIV ajqebZDafP9D7dpQRu26caoq2sKar60tH0o6gMywugWetuZ6l8hRgbpbi6m/Us5AsP53 hbGQ== X-Gm-Message-State: AOUpUlEcwI4xBhMjO4o8a9eOzYY08xGJ36TzS3UbvL7xlCP+MgGNq5Pb cD6go+Npaf1CJbqOFKAdGw8tdAFUmHXmnk1sK1k= X-Google-Smtp-Source: AAOMgpclH/MSrq2o1HTZHZsGR0R7/R9Ngg57yoJdQ93zApQoYcDOhgNBE7mYQD5m6wZXquc9L+kGYbpA5trdlqE7E5g= X-Received: by 2002:a1c:ac45:: with SMTP id v66-v6mr1839355wme.136.1531067808125; Sun, 08 Jul 2018 09:36:48 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:adf:ec4e:0:0:0:0:0 with HTTP; Sun, 8 Jul 2018 09:36:27 -0700 (PDT) In-Reply-To: <916db1d1-f158-fb30-76ef-e9c6f76c40f2@gmail.com> References: <20180707110031.GA19713@feusi.co> <916db1d1-f158-fb30-76ef-e9c6f76c40f2@gmail.com> From: Jim Meyering Date: Sun, 8 Jul 2018 09:36:27 -0700 X-Google-Sender-Auth: DSgSNzmKcjOhHQu4wNabEhmHyO4 Message-ID: Subject: Re: bug#32082: heap buffer overflow in sed/execute.c, line 992 To: Assaf Gordon Content-Type: multipart/mixed; boundary="000000000000d4c6fa05707f8008" X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 32082 Cc: bugs@feusi.co, 32082@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) --000000000000d4c6fa05707f8008 Content-Type: text/plain; charset="UTF-8" On Sat, Jul 7, 2018 at 9:28 PM, Assaf Gordon wrote: > Hello, > > On 07/07/18 05:01 AM, bugs@feusi.co wrote: >> >> I am working on a project in which I use the afl fuzzer to fuzz >> different open-source software. In doing so, I discovered a >> heap buffer overflow in sed/execute.c, line 992. > > > Thank you for this interesting bug report, and for providing such easy > way to reproduce. I can confirm this is reproducible, and in fact is > a very old bug! fantastic work. > > It took some time and lots of squinting to track it down, so I'll write > it here in details for others. > > Your sed script file was: > ==== > 10s11\91 > \0^0s1011 > \00a > \0^0s111w0 > ==== > > The input file content doesn't matter for the bug, so I won't focus on it. > > Interested readers - If you like challenges, stop reading this email and > spend couple of minutes trying to understand the above script. > fun a-plenty :) > > I assume that all the 1's and 0's are because AFL mutated bit by bit > until something was triggered. They aren't critical for the bug, so > here's a simpler and more concise buggy program: > > ==== > seq 2 | valgrind sed -e '/^/s///p ; 2s//\9/' > ==== > > It might still not be immediately clear why there is a bug here. > An even simpler version of the above program does not seem buggy at > first, because the invalid backref is detected: > === > $ seq 2 | sed -e '/^/p; 2s//\9/' > 1 > 1 > 2 > sed: -e expression #1, char 0: invalid reference \9 on `s' command > === > > However, this detection suppressed under "--posix", so the bug is > actually there: > ==== > $ seq 2 | valgrind sed --posix -e '/^/p; 2s//\9/' > 1 > 1 > 2 > ==19663== Invalid read of size 4 > ==19663== at 0x10FD51: ??? (in /bin/sed) > ==19663== by 0x110402: ??? (in /bin/sed) > ==19663== by 0x10B106: ??? (in /bin/sed) > ==19663== by 0x50802E0: (below main) (libc-start.c:291) > ==19663== Address 0x5a9df74 is 20 bytes after a block of size 16 in > [....] > ==== > > The novelty of this bug report is that using few more "previous regexes" > and the ^/$ optimization [1], the safety check is bypassed even when not > using "--posix". > [1] https://git.savannah.gnu.org/cgit/sed.git/commit/?id=6dea75e7 > > > Attached is a suggested fix. > It seems very simple, but I encourage everyone to double-check my logic > and perhaps detect more problems. > > comments very welcomed, Fine work! Thank you, Assaf. Here are some suggested comment adjustments: --000000000000d4c6fa05707f8008 Content-Type: application/octet-stream; name="sed-touchups.diff" Content-Disposition: attachment; filename="sed-touchups.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_jjd27oqk1 ZGlmZiAtLWdpdCBhL05FV1MgYi9ORVdTCmluZGV4IDBjNTA1MjYuLmFjMTY2ZGQgMTAwNjQ0Ci0t LSBhL05FV1MKKysrIGIvTkVXUwpAQCAtNSw3ICs1LDcgQEAgR05VIHNlZCBORVdTICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgLSotIG91dGxpbmUgLSotCiAqKiBCdWcgZml4ZXMK CiAgIHNlZCBubyBsb25nZXIgYWNjZXNzZXMgaW52YWxpZCBtZW1vcnkgKGhlYXAgb3ZlcmZsb3cp IHdoZW4gZ2l2ZW4gaW52YWxpZAotICByZWZlcmVuY2VzIGluICdzJyBjb21tYW5kIFtidWcjMzIw ODIsIHByZXNlbnQgYXQgbGVhc3Qgc2luY2Ugc2VkLTQuMC42XS4KKyAgYmFja3JlZmVyZW5jZXMg aW4gJ3MnIGNvbW1hbmQgW2J1ZyMzMjA4MiwgcHJlc2VudCBhdCBsZWFzdCBzaW5jZSBzZWQtNC4w LjZdLgoKCiAqIE5vdGV3b3J0aHkgY2hhbmdlcyBpbiByZWxlYXNlIDQuNSAoMjAxOC0wMy0zMSkg W3N0YWJsZV0KZGlmZiAtLWdpdCBhL3Rlc3RzdWl0ZS9idWczMjA4Mi5zaCBiL3Rlc3RzdWl0ZS9i dWczMjA4Mi5zaAppbmRleCA2YTc3NjU2Li5kNWQ0ZTkyIDEwMDc1NQotLS0gYS90ZXN0c3VpdGUv YnVnMzIwODIuc2gKKysrIGIvdGVzdHN1aXRlL2J1ZzMyMDgyLnNoCkBAIC0xLDYgKzEsNSBAQAog IyEvYmluL3NoCi0jIHNlZCBtYXkgYWNjZXNzIHRvIHVuaW5pdGlhbGl6ZWQgbWVtb3J5IHdoZSBp bnZhbGlkIGJhY2tyZWZlcmVuY2UKLSMgYXJlIHVzZWQgdW5kZXIgY2VydGFpbiBjaXJjdW1zdGFu Y2VzLgorIyBzZWQgd291bGQgYWNjZXNzIHVuaW5pdGlhbGl6ZWQgbWVtb3J5IGZvciBjZXJ0YWlu IGludmFsaWQgYmFja3JlZmVyZW5jZSB1c2VzLgogIyBCZWZvcmUgc2VkIDQuNiB0aGVzZSB3b3Vs ZCByZXN1bHQgaW4gIkludmFsaWQgcmVhZCBzaXplIG9mIDQiIHJlcG9ydGVkCiAjIGJ5IHZhbGdy aW5kIGZyb20gZXhlY3V0ZS5jOjk5MgoKQEAgLTU2LDEwICs1NSwxMCBAQCBjb21wYXJlIC9kZXYv bnVsbCBlcnIgfHwgZmFpbD0xCiAjCiAjIFRlc3QgMjogd2l0aG91dCAiLS1wb3NpeCIKICMKLSMg V2hlbiBub3QgdXNpbmcgIi0tcG9zaXgiLCB1c2luZyBhIGJhY2tyZWYgdG8gYSBub24tZXhpc3Rp bmcgZ3JvdXAuCisjIFdoZW4gbm90IHVzaW5nICItLXBvc2l4IiwgdXNpbmcgYSBiYWNrcmVmIHRv IGEgbm9uLWV4aXN0aW5nIGdyb3VwCiAjIHdvdWxkIGJlIGNhdWdodCBpbiBjb21waWxlX3JlZ2V4 XzEuCiAjIEFzIHJlcG9ydGVkIGluIGJ1Z3MuZ251Lm9yZy8zMjA4MiBieSBidWdzQGZldXNpLmNv LAotIyB1c2luZyB0aGUgcmVjZW50IGJlZ2xpbmUvZW5kbGluZSBvcHRpbWl6YXRpb24gd2l0aCBm ZXcgInByZXZpb3VzIHJlZ2V4IgorIyB1c2luZyB0aGUgcmVjZW50IGJlZ2xpbmUvZW5kbGluZSBv cHRpbWl6YXRpb24gd2l0aCBhIGZldyAicHJldmlvdXMgcmVnZXgiCiAjIHRyaWNrcyBieXBhc3Nl cyB0aGlzIGNoZWNrLgoKIHZhbGdyaW5kIC0tcXVpZXQgLS1lcnJvci1leGl0Y29kZT0xIFwK --000000000000d4c6fa05707f8008-- From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 08 22:14:40 2018 Received: (at 32082) by debbugs.gnu.org; 9 Jul 2018 02:14:40 +0000 Received: from localhost ([127.0.0.1]:51587 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fcLh9-0003F6-24 for submit@debbugs.gnu.org; Sun, 08 Jul 2018 22:14:40 -0400 Received: from mail-pg1-f177.google.com ([209.85.215.177]:38675) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fcLh5-0003Eo-4O for 32082@debbugs.gnu.org; Sun, 08 Jul 2018 22:14:30 -0400 Received: by mail-pg1-f177.google.com with SMTP id k3-v6so930272pgq.5 for <32082@debbugs.gnu.org>; Sun, 08 Jul 2018 19:14:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=bJr3awCmx0Oulm7cOx+ysCTkEmk/JV725IF4Dtx+reA=; b=et0mVKELy6mczvk30Nl8lMdggYGvzGwJqqn5hrmzBiXVhTJs+D84yEuCZCWL+kIu/z BNoVFEiwEzgeDhK223qiL5WatIDYABosSZMJ8xA6eGGZJNdQKohAgKll6+/YRui0K+wS dJryvgN/IK+ZvbiI9EsOD5ebbVm5gOBCMnCh2CzQIlffCXSCjPiV2NK85fEA4NbGloWN zHGe9huhe8EGrFfSQZ7CO9M2ur75PkbhOI/5LWVuoRnxDzad3oWEHW6ATcmTsCPC9Wk1 KlHWSGbaaJAY49xsqg2ntITjn3azN1QqajV0inXihDgO6Tif6X81hyJFBUg09WPmYZBA wy6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=bJr3awCmx0Oulm7cOx+ysCTkEmk/JV725IF4Dtx+reA=; b=WeFur7e6TmkpStTCrBonh58DGEbSdWl5+nLb32FZKp27PIRmNELiyRuY8NRZX80viq Hni0shMB8SITdIy1HaQ7k7931Nri2AheBCVwIOB9MQECEiuG/rREuoKW2CYM63EvVukg 68fvB7WhCc0yplbaUBDjlwwBqXiZZAuoBn8bnz8CgXiKAZeP4rnpJfd2UZnIvheME+3w XqYHzIkA5oC7PfT6qwcWdJvRJwZfv7qfn8LnQCDmrVJGVG/8vt4HyoLvmsnXzCw8RsFK Wq9A3bp9cfI2blZUod3MWXLnwEgi7fNC9BLeGzpOqGeV4F11ZHSNots/PcmjOSVpxRsz tpYw== X-Gm-Message-State: APt69E2sgURUMxrZ7xGD2b41QRmoUnqqQaRcayNS9cKEH7ydcVn6v7y0 bKOGCSI9KckIGeklJ7uZXpxAyixa X-Google-Smtp-Source: AAOMgpflkPvRp0Gdag2LQsiU0fs05ABiYpfRRtDX10wTtOspqgFXEZ5RG9adYio9DpGM5niSgFVqEQ== X-Received: by 2002:a62:398c:: with SMTP id u12-v6mr19516741pfj.9.1531102460214; Sun, 08 Jul 2018 19:14:20 -0700 (PDT) Received: from tomato.housegordon.com (moose.housegordon.com. [184.68.105.38]) by smtp.googlemail.com with ESMTPSA id h26-v6sm25181684pfj.77.2018.07.08.19.14.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 08 Jul 2018 19:14:18 -0700 (PDT) Subject: Re: bug#32082: heap buffer overflow in sed/execute.c, line 992 To: Jim Meyering References: <20180707110031.GA19713@feusi.co> <916db1d1-f158-fb30-76ef-e9c6f76c40f2@gmail.com> From: Assaf Gordon Message-ID: <92c06adb-8862-673c-addc-7524b6e0bab1@gmail.com> Date: Sun, 8 Jul 2018 20:14:17 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------1F4BBBFFB1C1F318E5AAADE6" Content-Language: en-US X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 32082 Cc: bugs@feusi.co, 32082@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) This is a multi-part message in MIME format. --------------1F4BBBFFB1C1F318E5AAADE6 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 08/07/18 10:36 AM, Jim Meyering wrote: > On Sat, Jul 7, 2018 at 9:28 PM, Assaf Gordon wrote: >> On 07/07/18 05:01 AM, bugs@feusi.co wrote: >>> >>> I am working on a project in which I use the afl fuzzer to fuzz >>> different open-source software. In doing so, I discovered a >>> heap buffer overflow in sed/execute.c, line 992. >> >> Attached is a suggested fix. >> >> comments very welcomed, > > Here are some suggested comment adjustments: Thanks. Attached updated version. I will push it tomorrow if there are no further comments. regards, - assaf --------------1F4BBBFFB1C1F318E5AAADE6 Content-Type: text/x-patch; name="0001-sed-fix-heap-buffer-overflow-from-invalid-references.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0001-sed-fix-heap-buffer-overflow-from-invalid-references.pa"; filename*1="tch" >From c52a676e5e31f4f5c25d78f5dd4c17fab6585d8e Mon Sep 17 00:00:00 2001 From: Assaf Gordon Date: Sat, 7 Jul 2018 22:03:38 -0600 Subject: [PATCH] sed: fix heap buffer overflow from invalid references Under certain conditions sed would access invalid memory based on the requested back-reference (e.g. "s//\9/" would access the 9th element in the regex registers without checking it is at least 9 element in size). The following examples would trigger valgrind errors: seq 2 | valgrind sed -e '/^/s///p ; 2s//\9/' seq 2 | valgrind sed --posix -e '/2/p ; 2s//\9/' Reported by bugs@feusi.co in https://lists.gnu.org/r/bug-sed/2018-07/msg00004.html . * NEWS: Mention the bugfix. * sed/execute.c (append_replacement): Check number of allocated regex replacement registers before accessing the array. * sed/testsuite/bug32082.sh: Test sed for this behaviour under valgrind. * sed/testsuite/local.mk (T): Add new test. --- NEWS | 5 ++++ sed/execute.c | 2 +- testsuite/bug32082.sh | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++ testsuite/local.mk | 1 + 4 files changed, 88 insertions(+), 1 deletion(-) create mode 100755 testsuite/bug32082.sh diff --git a/NEWS b/NEWS index 6c2fbaf..ac166dd 100644 --- a/NEWS +++ b/NEWS @@ -2,6 +2,11 @@ GNU sed NEWS -*- outline -*- * Noteworthy changes in release ?.? (????-??-??) [?] +** Bug fixes + + sed no longer accesses invalid memory (heap overflow) when given invalid + backreferences in 's' command [bug#32082, present at least since sed-4.0.6]. + * Noteworthy changes in release 4.5 (2018-03-31) [stable] diff --git a/sed/execute.c b/sed/execute.c index 2804c5e..7a4850f 100644 --- a/sed/execute.c +++ b/sed/execute.c @@ -987,7 +987,7 @@ static void append_replacement (struct line *buf, struct replacement *p, curr_type &= ~REPL_MODIFIERS; } - if (0 <= i) + if (0 <= i && i < regs->num_regs) { if (regs->end[i] == regs->start[i] && p->repl_type & REPL_MODIFIERS) /* Save this modifier, we shall apply it later. diff --git a/testsuite/bug32082.sh b/testsuite/bug32082.sh new file mode 100755 index 0000000..d5d4e92 --- /dev/null +++ b/testsuite/bug32082.sh @@ -0,0 +1,81 @@ +#!/bin/sh +# sed would access uninitialized memory for certain invalid backreference uses. +# Before sed 4.6 these would result in "Invalid read size of 4" reported +# by valgrind from execute.c:992 + +# Copyright (C) 2018 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +. "${srcdir=.}/testsuite/init.sh"; path_prepend_ ./sed +print_ver_ sed + +require_valgrind_ + +printf '1\n2\n' > in || framework_failure_ +printf '1\n2\n\n' > exp-posix || framework_failure_ +printf '1\n1\n2\n2\n' > exp-no-posix || framework_failure_ + +# +# Test 1: with "--posix" +# +# using "--posix" disables the backref safety check in +# regexp.c:compile_regex_1(), which is reported as: +# "invalid reference \\%d on `s' command's RHS" + +valgrind --quiet --error-exitcode=1 \ + sed --posix -e '/2/p ; 2s//\9/' in > out-posix 2> err-posix || fail=1 + +echo "valgrind report for 'posix' test:" +echo "==================================" +cat err-posix +echo "==================================" + + +# Work around a bug in CentOS 5.10's valgrind +# FIXME: remove in 2018 or when CentOS 5 is no longer officially supported +grep 'valgrind: .*Assertion.*failed' err-posix > /dev/null \ + && skip_ 'you seem to have a buggy version of valgrind' + +compare exp-posix out-posix || fail=1 +compare /dev/null err || fail=1 + + + +# +# Test 2: without "--posix" +# +# When not using "--posix", using a backref to a non-existing group +# would be caught in compile_regex_1. +# As reported in bugs.gnu.org/32082 by bugs@feusi.co, +# using the recent begline/endline optimization with a few "previous regex" +# tricks bypasses this check. + +valgrind --quiet --error-exitcode=1 \ + sed -e '/^/s///p ; 2s//\9/' in > out-no-posix 2> err-no-posix || fail=1 + +echo "valgrind report for 'no-posix' test:" +echo "====================================" +cat err-no-posix +echo "====================================" + +# Work around a bug in CentOS 5.10's valgrind +# FIXME: remove in 2018 or when CentOS 5 is no longer officially supported +grep 'valgrind: .*Assertion.*failed' err-no-posix > /dev/null \ + && skip_ 'you seem to have a buggy version of valgrind' + +compare exp-no-posix out-no-posix || fail=1 +compare /dev/null err || fail=1 + + +Exit $fail diff --git a/testsuite/local.mk b/testsuite/local.mk index b4a4f5a..bf4559f 100644 --- a/testsuite/local.mk +++ b/testsuite/local.mk @@ -43,6 +43,7 @@ LOG_COMPILER = false T = \ testsuite/misc.pl \ + testsuite/bug32082.sh \ testsuite/cmd-l.sh \ testsuite/cmd-R.sh \ testsuite/colon-with-no-label.sh \ -- 2.11.0 --------------1F4BBBFFB1C1F318E5AAADE6-- From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 11 02:52:08 2018 Received: (at control) by debbugs.gnu.org; 11 Jul 2018 06:52:09 +0000 Received: from localhost ([127.0.0.1]:54674 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fd8yu-0000lc-P1 for submit@debbugs.gnu.org; Wed, 11 Jul 2018 02:52:08 -0400 Received: from mail-pf0-f172.google.com ([209.85.192.172]:41570) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fd8yt-0000lE-Cd; Wed, 11 Jul 2018 02:52:07 -0400 Received: by mail-pf0-f172.google.com with SMTP id c21-v6so13156169pfn.8; Tue, 10 Jul 2018 23:52:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=0X7rSfnA/kLDsqpGTU07xI78zVoEjUph9PwxECM8Qmc=; b=J+8x9avbBNWkuJKWFu3QK5b2W/KiRwp6BoTdWPpK26eALxAUl4F2aNBT0t/DVxoh1L iMvCcH4DT+ne3OFy+01ryaN5P3YwAaOPaOlRu0Ouqo3DEbgON94DMOxJV/hvsDoy0+wM P07tbHvZuTinpdwvrtNCfK8ncHGBsmGM8TpHoXpZIQusgnDCf4nWz8lQdlxMK/h3uZjM W5E/XAKn94uO68QFfd0xU9oOntcKTWAyTQkSjNjo3fEqeb3ohJky7W6tj6mGpRi/8SHx 93mQJj1WcklpUBx59YFBiVI7vLDSxXIlR7ojYDYS29atm2vDGV7CJhsvznZ1wDPcR7KG xA0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=0X7rSfnA/kLDsqpGTU07xI78zVoEjUph9PwxECM8Qmc=; b=ePtgdD3Bi1Fq1BUbForABLn0Lb8G6LacLHhFtD/jThNpiLf8k3ps5Z5fOJcMWlxkNf tg4L+7gLsKk+1WQjKOPlkxg02Y0kexQqPjDrrd791ea4/yKXnw0YMbCgXsRWNdoBnGyR lQZofCfFzrJFwSPZBoVuQbD6S0eX6M6zH1SPpRuJ7YhBWcCv1KXh7qXA+j37z1UQZBh2 jbwyRfuKrDEXJSQL0XzRrWQB8CYRQjjT4IBq2gS/Td1BRdm3BAMCgxFXv1obVxw65tTE g7zJUxm1zT0R0yBpcRDHepnTqwWxF0MPh+VGlP3TTwaPr1SPAQnitt5//yap/ivnNfnp EzEA== X-Gm-Message-State: APt69E2EPouQxvLoFfoP2wicU/cuoSTyDb1wIEqmYqDAoHDGhJ1yCJzg VNi0RAWyWa0tYfeR5QpDmAh+5lA6 X-Google-Smtp-Source: AAOMgpf0CaIO3SCk/pzrkYoe6Wsv51v5o+tUzi50i3iqTNxp/yhXQ595tPP5LKyEneEag6qJtgc2Yw== X-Received: by 2002:a62:11c4:: with SMTP id 65-v6mr28843958pfr.54.1531291920829; Tue, 10 Jul 2018 23:52:00 -0700 (PDT) Received: from tomato.housegordon.com (moose.housegordon.com. [184.68.105.38]) by smtp.googlemail.com with ESMTPSA id m5-v6sm30974341pfm.27.2018.07.10.23.51.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Jul 2018 23:51:58 -0700 (PDT) Subject: Re: bug#32082: heap buffer overflow in sed/execute.c, line 992 From: Assaf Gordon To: Jim Meyering References: <20180707110031.GA19713@feusi.co> <916db1d1-f158-fb30-76ef-e9c6f76c40f2@gmail.com> <92c06adb-8862-673c-addc-7524b6e0bab1@gmail.com> Message-ID: Date: Wed, 11 Jul 2018 00:51:57 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <92c06adb-8862-673c-addc-7524b6e0bab1@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: control Cc: bugs@feusi.co, 32082-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) tags 32082 fixed stop >> On Sat, Jul 7, 2018 at 9:28 PM, Assaf Gordon >> wrote:>>> On 07/07/18 05:01 AM, bugs@feusi.co wrote:>>>>>>>> I am working on a project in which I use the afl fuzzer to fuzz>>>> different open-source software. In doing so, I discovered a>>>> heap buffer overflow in sed/execute.c, line 992.>>> Attached is a suggested fix. pushed here: https://git.savannah.gnu.org/cgit/sed.git/commit/?id=c52a676e From unknown Fri Jun 20 18:21:43 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Wed, 08 Aug 2018 11:24:07 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator