From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 17 16:28:38 2020 Received: (at submit) by debbugs.gnu.org; 17 Jan 2020 21:28:38 +0000 Received: from localhost ([127.0.0.1]:40225 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1isZAU-0000ry-Ch for submit@debbugs.gnu.org; Fri, 17 Jan 2020 16:28:38 -0500 Received: from lists.gnu.org ([209.51.188.17]:40266) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1isZAS-0000rr-P1 for submit@debbugs.gnu.org; Fri, 17 Jan 2020 16:28:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52321) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1isZAQ-0000zO-H1 for bug-sed@gnu.org; Fri, 17 Jan 2020 16:28:35 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RCVD_IN_DNSWL_NONE, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1isZAP-0003SY-Dc for bug-sed@gnu.org; Fri, 17 Jan 2020 16:28:34 -0500 Received: from mout.kundenserver.de ([217.72.192.74]:59815) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1isZAP-0003QF-4U for bug-sed@gnu.org; Fri, 17 Jan 2020 16:28:33 -0500 Received: from localhost ([217.87.199.239]) by mrelayeu.kundenserver.de (mreue106 [212.227.15.145]) with ESMTPSA (Nemesis) id 1MMWgb-1jB3M006g3-00Jcm9 for ; Fri, 17 Jan 2020 22:28:31 +0100 Date: Fri, 17 Jan 2020 21:28:28 +0100 From: Tobias Stoeckmann To: bug-sed@gnu.org Subject: [PATCH] sed: handle very long input lines with R Message-ID: <20200117202828.dqwdznwbngr64gyu@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Provags-ID: V03:K1:NxvMjI2/lDIBeoh8SWFy+tE2k/tegfTCgZF2SwYE6Lnp+Rw9brO gXJQqkdZ79WQnTwce+D2XOyt3B4hLQPtIMSB2A/REONC89rPVILdq0muDpST63kmKlOk0ci yZEE70Dgiv+TRyrAxOq7fY8t52z6L+/dAst4HDFUEUuEEPFWoDiPyqupOREPOW0tXHq0slo 2rlOARObSXLmX0g4zmysQ== X-UI-Out-Filterresults: notjunk:1;V03:K0:ZCAtElnFMY0=:GQo0ra27sY7HAt3aRUk+bV fB0bs7S+mxAy3yfBTaUQ/+efiZBxxQNP316MlK45Q4ZCIZoa3EfgT2X2ECRkllq04c74EHqrq IhvfQzaYpQjwDFWPpa2fAbRpXpYEJ9uSam2c+/amQGKD/xd9tCFjanrA05RZqwW2juLUkmYqv UiM/N/EdiyiSaYC0Sv9XBoeoS/XeQccD1UDEhckYY7x5sv7ZUbMuL9Ni2NmC78PJKkR9A8mHt SJxzAiMtTB4wDQkzdn10YLHRvv0kUezf0Edg8yCb8XifglqbChTkoJfK9NArkRUWWYeVEgNrw JWNg4ClGN7qUmwARHViRVpOHotHPAUuyuTbh1zPDTHNEfqXCngb7NY14c2xb49jy/HYB6jjfv 7fpUd4Qic4psZsQe9MiDIUqyFIquzab2WJXYbgI2tNqjO+Jx4blQeGlQrkF1kZijM1DJrzPAY Dk6iQVCP4XaiC4GdFTsm5q3QuNevEWXmcFT36mMNbQKhDzejSJ5HkrvA/uG5Q37I0AWtrUkRB BO09wqp3jdQKsnqyG+8IBNajJwVwOAHOOZL3t9ZtwPJB9t68xsJ6w4G/l19ph3XWPcpAXUy4f txIVS3puy0GmqdOwsJaNrJ8uEKfVHHLSCJiQfcucJrq799L5W/7EIBfdV/vpuNfMKRaox5RUc oNdH2dzIepgp70U30TAd87AriKHLVCpWILbhPw+3C6vVFhLsqXJYPu9Dry4Y/M2Dx9bCz+Z70 C5Ixq39uR9y/3GYQhHhSdyaIwzsjlds1QN7tSjaXgmv4FEiVu2HC31Po4qeDKj2snh6/ckJqS j7qjDx7RNgj4bVhtgeOJh1zctLBf+nurv8y9gmwKTJAoMWARf8kDB/rnnuAKapuiNffUMiRqY rtkFFijqtTqBmpCF+S4tEw+Wn9z7sWqUBw+Y77DvA= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 217.72.192.74 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) It is possible to trigger an out of boundary memory access when using the sed command R with an input file containing very long lines. The problem is that the line length of parsed file is returned as a size_t by ck_getdelim, but temporarily stored in an int and then converted back into a size_t. On systems like amd64, on which this problem can be triggered, size_t and int have different sizes. If the input line is longer than 2 GB (which is parseable on amd64 or other 64 bit systems), this means that the temporarily stored int turns negative. Converting the negative int back into a size_t will lead to an excessively large size_t, as the conversion leads to a lot of leading 1 bits. Eventually ck_fwrite is called with this huge size_t which in turn will lead to an out of boundary access on amd64 systems -- after all the parsed text was just a bit above 2 GB, not near SIZE_MAX. You can trigger this issue with GNU sed on OpenBSD like this: $ dd if=/dev/zero bs=1M count=2049 | tr '\0' 'e' > long.txt $ sed Rlong.txt /etc/fstab Segmentation fault (core dumped) $ _ I was unable to trigger the bug on a Linux system with glibc due to a bug in glibc's fwrite implementation -- it leads to a short write and sed treats that correctly as an error. Signed-off-by: Tobias Stoeckmann --- sed/execute.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sed/execute.c b/sed/execute.c index 8f43f2e..f94b125 100644 --- a/sed/execute.c +++ b/sed/execute.c @@ -1518,7 +1518,7 @@ execute_program (struct vector *vec, struct input *input) struct append_queue *aq; size_t buflen; char *text = NULL; - int result; + size_t result; result = ck_getdelim (&text, &buflen, buffer_delimiter, cur_cmd->x.inf->fp); -- 2.25.0 From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 18 11:14:43 2020 Received: (at 39166-done) by debbugs.gnu.org; 18 Jan 2020 16:14:43 +0000 Received: from localhost ([127.0.0.1]:42062 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1isqkE-000571-Pk for submit@debbugs.gnu.org; Sat, 18 Jan 2020 11:14:43 -0500 Received: from mail-wr1-f45.google.com ([209.85.221.45]:43245) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1isqkC-00056m-UC for 39166-done@debbugs.gnu.org; Sat, 18 Jan 2020 11:14:41 -0500 Received: by mail-wr1-f45.google.com with SMTP id d16so25465626wre.10 for <39166-done@debbugs.gnu.org>; Sat, 18 Jan 2020 08:14:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=JxZZ0ZDdTqJT+93sCvnrAYt3TZbU6b4Pxmd8r55BeAU=; b=ioGNJsKX8QN6Fmr9fhB/RwVgdL8GnWYSmOWX3u3nObDnFqGGV3q/dIE09cX+m+Npae nSQHHyuLMQL6B0hrenEyTDAqnxUdcWum2xLrpJ7b5SMMyKO5jDMD8rZt3gRAH1EIn02e VJOfo8mNsd1EvW6cpLAgnNl2P+ffdoUiXKDkYvNY82irGkU5YH89dLLfv/1xI0ZHUewR jqKZhUmOC5vx9RyAR3j/D+MyOVsXwg52fel4gPa6UJQ0zkLcVUU8gJeYH6ho7h1d7xW9 dV2KHKYI/7kZQm4PnTQeaYo6oIncAaINMyEH3C8BkuFreey/+mD06J4C5132RNxCjZbA Ealg== X-Gm-Message-State: APjAAAU+wFGaYgHsO+duuuBZ+jZ+UPCW8wTrgKFhVGNiIJGYPPHDQ2iI RMnvFpiLHa/deY9TDGD0qUq6SMWdCg1kT+fvdJo= X-Google-Smtp-Source: APXvYqzoqfxEPedODumM4MiILAsHjTrJsSRUO845oDKKwbinVog0HjEkSnRwWTBBscaeU6DRvxOnUtolOfqq6oB6OiQ= X-Received: by 2002:a5d:5452:: with SMTP id w18mr8758102wrv.333.1579364075199; Sat, 18 Jan 2020 08:14:35 -0800 (PST) MIME-Version: 1.0 References: <20200117202828.dqwdznwbngr64gyu@localhost> In-Reply-To: <20200117202828.dqwdznwbngr64gyu@localhost> From: Jim Meyering Date: Sat, 18 Jan 2020 08:14:23 -0800 Message-ID: Subject: Re: bug#39166: [PATCH] sed: handle very long input lines with R To: Tobias Stoeckmann Content-Type: multipart/mixed; boundary="000000000000acb255059c6c5a3c" X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 39166-done Cc: 39166-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) --000000000000acb255059c6c5a3c Content-Type: text/plain; charset="UTF-8" On Fri, Jan 17, 2020 at 1:29 PM Tobias Stoeckmann wrote: > It is possible to trigger an out of boundary memory access when > using the sed command R with an input file containing very long > lines. Thank you for another fine patch. I've adjusted the commit log and will push the attached later today. --000000000000acb255059c6c5a3c Content-Type: application/octet-stream; name="sed-2G-R.diff" Content-Disposition: attachment; filename="sed-2G-R.diff" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_k5jsjs640 RnJvbSBhMWI1YzcxOTczNTM3ZWE3NzY0MjE0NmIyYTA4YWYxY2RiODI5NmVkIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBUb2JpYXMgU3RvZWNrbWFubiA8dG9iaWFzQHN0b2Vja21hbm4u b3JnPgpEYXRlOiBGcmksIDE3IEphbiAyMDIwIDIxOjI4OjI4ICswMTAwClN1YmplY3Q6IFtQQVRD SF0gc2VkOiBoYW5kbGUgdmVyeSBsb25nIGlucHV0IGxpbmVzIHdpdGggUiAodGlueSBjaGFuZ2Up CgpJdCBpcyBwb3NzaWJsZSB0byB0cmlnZ2VyIGFuIG91dCBvZiBib3VuZHMgbWVtb3J5IGFjY2Vz cyB3aGVuCnVzaW5nIHRoZSBzZWQgY29tbWFuZCBSIHdpdGggYW4gaW5wdXQgZmlsZSBjb250YWlu aW5nIHZlcnkgbG9uZwpsaW5lcy4KClRoZSBwcm9ibGVtIGlzIHRoYXQgdGhlIGxpbmUgbGVuZ3Ro IG9mIHBhcnNlZCBmaWxlIGlzIHJldHVybmVkIGFzCmEgc2l6ZV90IGJ5IGNrX2dldGRlbGltLCBi dXQgdGVtcG9yYXJpbHkgc3RvcmVkIGluIGFuIGludCBhbmQKdGhlbiBjb252ZXJ0ZWQgYmFjayBp bnRvIGEgc2l6ZV90LiBPbiBzeXN0ZW1zIGxpa2UgYW1kNjQsIG9uIHdoaWNoCnRoaXMgcHJvYmxl bSBjYW4gYmUgdHJpZ2dlcmVkLCBzaXplX3QgYW5kIGludCBoYXZlIGRpZmZlcmVudCBzaXplcy4K CklmIHRoZSBpbnB1dCBsaW5lIGlzIGxvbmdlciB0aGFuIDIgR0IgKHdoaWNoIGlzIHBhcnNlYWJs ZSBvbiBhbWQ2NApvciBvdGhlciA2NCBiaXQgc3lzdGVtcyksIHRoaXMgbWVhbnMgdGhhdCB0aGUg dGVtcG9yYXJpbHkgc3RvcmVkCmludCB0dXJucyBuZWdhdGl2ZS4gQ29udmVydGluZyB0aGUgbmVn YXRpdmUgaW50IGJhY2sgaW50byBhIHNpemVfdAp3aWxsIGxlYWQgdG8gYW4gZXhjZXNzaXZlbHkg bGFyZ2Ugc2l6ZV90LCBhcyB0aGUgY29udmVyc2lvbiBsZWFkcyB0bwphIGxvdCBvZiBsZWFkaW5n IDEgYml0cy4KCkV2ZW50dWFsbHkgY2tfZndyaXRlIGlzIGNhbGxlZCB3aXRoIHRoaXMgaHVnZSBz aXplX3Qgd2hpY2ggaW4gdHVybgp3aWxsIGxlYWQgdG8gYW4gb3V0IG9mIGJvdW5kcyBhY2Nlc3Mg b24gYW1kNjQgc3lzdGVtcyAtLSBhZnRlciBhbGwKdGhlIHBhcnNlZCB0ZXh0IHdhcyBqdXN0IGEg Yml0IGFib3ZlIDIgR0IsIG5vdCBuZWFyIFNJWkVfTUFYLgoKWW91IGNhbiB0cmlnZ2VyIHRoaXMg aXNzdWUgd2l0aCBHTlUgc2VkIG9uIE9wZW5CU0QgbGlrZSB0aGlzOgoKJCBkZCBpZj0vZGV2L3pl cm8gYnM9MU0gY291bnQ9MjA0OSB8IHRyICdcMCcgZSA+IGxvbmcudHh0CiQgc2VkIFJsb25nLnR4 dCAvZXRjL2ZzdGFiClNlZ21lbnRhdGlvbiBmYXVsdCAoY29yZSBkdW1wZWQpCgpJIHdhcyB1bmFi bGUgdG8gdHJpZ2dlciB0aGUgYnVnIG9uIGEgTGludXggc3lzdGVtIHdpdGggZ2xpYmMgZHVlIHRv CmEgYnVnIGluIGdsaWJjJ3MgZndyaXRlIGltcGxlbWVudGF0aW9uIC0tIGl0IGxlYWRzIHRvIGEg c2hvcnQgd3JpdGUKYW5kIHNlZCB0cmVhdHMgdGhhdCBjb3JyZWN0bHkgYXMgYW4gZXJyb3IuCgoq IHNlZC9leGVjdXRlLmMgKGV4ZWN1dGVfcHJvZ3JhbSkgW2Nhc2UgJ1InXTogRGVjbGFyZSByZXN1 bHQKdG8gYmUgb2YgdHlwZSBzaXplX3QsIG5vdCBpbnQuCiogTkVXUyAoQnVnIGZpeGVzKTogTWVu dGlvbiBpdC4KVGhpcyBhZGRyZXNzZXMgaHR0cHM6Ly9idWdzLmdudS5vcmcvMzkxNjYKLS0tCiBO RVdTICAgICAgICAgIHwgMyArKysKIHNlZC9leGVjdXRlLmMgfCAyICstCiAyIGZpbGVzIGNoYW5n ZWQsIDQgaW5zZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQoKZGlmZiAtLWdpdCBhL05FV1MgYi9O RVdTCmluZGV4IGZlOWNhOTEuLjQ1NTkzYWYgMTAwNjQ0Ci0tLSBhL05FV1MKKysrIGIvTkVXUwpA QCAtNyw2ICs3LDkgQEAgR05VIHNlZCBORVdTICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgLSotIG91dGxpbmUgLSotCiAgIGEgcHJvZ3JhbSB3aXRoIGFuIGV4ZWN1dGlvbiBsaW5l IGxvbmdlciB0aGFuIDJHQiBjYW4gbm8gbG9uZ2VyIHRyaWdnZXIKICAgYW4gb3V0LW9mLWJvdW5k cyBtZW1vcnkgd3JpdGUuCgorICB1c2luZyB0aGUgUiBjb21tYW5kIHRvIHJlYWQgYW4gaW5wdXQg bGluZSBvZiBsZW5ndGggbG9uZ2VyIHRoYW4gMkdCCisgIGNhbiBubyBsb25nZXIgdHJpZ2dlciBh biBvdXQtb2YtYm91bmRzIG1lbW9yeSByZWFkLgorCgogKiBOb3Rld29ydGh5IGNoYW5nZXMgaW4g cmVsZWFzZSA0LjggKDIwMjAtMDEtMTQpIFtzdGFibGVdCgpkaWZmIC0tZ2l0IGEvc2VkL2V4ZWN1 dGUuYyBiL3NlZC9leGVjdXRlLmMKaW5kZXggOGY0M2YyZS4uZjk0YjEyNSAxMDA2NDQKLS0tIGEv c2VkL2V4ZWN1dGUuYworKysgYi9zZWQvZXhlY3V0ZS5jCkBAIC0xNTE4LDcgKzE1MTgsNyBAQCBl eGVjdXRlX3Byb2dyYW0gKHN0cnVjdCB2ZWN0b3IgKnZlYywgc3RydWN0IGlucHV0ICppbnB1dCkK ICAgICAgICAgICAgICAgICAgIHN0cnVjdCBhcHBlbmRfcXVldWUgKmFxOwogICAgICAgICAgICAg ICAgICAgc2l6ZV90IGJ1ZmxlbjsKICAgICAgICAgICAgICAgICAgIGNoYXIgKnRleHQgPSBOVUxM OwotICAgICAgICAgICAgICAgICAgaW50IHJlc3VsdDsKKyAgICAgICAgICAgICAgICAgIHNpemVf dCByZXN1bHQ7CgogICAgICAgICAgICAgICAgICAgcmVzdWx0ID0gY2tfZ2V0ZGVsaW0gKCZ0ZXh0 LCAmYnVmbGVuLCBidWZmZXJfZGVsaW1pdGVyLAogICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIGN1cl9jbWQtPnguaW5mLT5mcCk7Ci0tIAoyLjI1LjAucmMxLjE5LmcwNDJl ZDNlMDQ4Cgo= --000000000000acb255059c6c5a3c-- From unknown Fri Aug 15 21:27:16 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sun, 16 Feb 2020 12:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator