From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 05 21:16:23 2012 Received: (at submit) by debbugs.gnu.org; 6 Dec 2012 02:16:23 +0000 Received: from localhost ([127.0.0.1]:55545 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TgR0a-00047d-Fy for submit@debbugs.gnu.org; Wed, 05 Dec 2012 21:16:23 -0500 Received: from eggs.gnu.org ([208.118.235.92]:41618) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TgQwH-0003yi-Kn for submit@debbugs.gnu.org; Wed, 05 Dec 2012 21:11:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TgQw4-0007Sn-Oe for submit@debbugs.gnu.org; Wed, 05 Dec 2012 21:11:38 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, RP_MATCHES_RCVD autolearn=unavailable version=3.3.2 Received: from lists.gnu.org ([208.118.235.17]:47368) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TgQw4-0007SY-L6 for submit@debbugs.gnu.org; Wed, 05 Dec 2012 21:11:36 -0500 Received: from eggs.gnu.org ([208.118.235.92]:52980) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TgQw2-000527-6G for bug-coreutils@gnu.org; Wed, 05 Dec 2012 21:11:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TgQvz-0007S5-Pv for bug-coreutils@gnu.org; Wed, 05 Dec 2012 21:11:34 -0500 Received: from mailout-eu.gmx.com ([213.165.64.42]:50300) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1TgQvz-0007Ry-FI for bug-coreutils@gnu.org; Wed, 05 Dec 2012 21:11:31 -0500 Received: (qmail invoked by alias); 06 Dec 2012 02:11:29 -0000 Received: from unknown (EHLO smag-R59-R60-R61) [151.65.149.48] by mail.gmx.com (mp-eu004) with SMTP; 06 Dec 2012 03:11:29 +0100 X-Authenticated: #130707387 X-Provags-ID: V01U2FsdGVkX1/xnZAApM3xqUqaLUuEve6exP1iItXxtozkjLflzE m/4ScCXF5WQp0Y Date: Thu, 6 Dec 2012 03:11:48 +0100 From: Cojocaru Alexandru To: bug-coreutils@gnu.org Subject: [PATCH] cut.c: Fix memory leak Message-Id: <20121206031148.03492829e89a0b475caabade@gmx.com> X-Mailer: Sylpheed 3.3.0 (GTK+ 2.24.13; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 208.118.235.17 X-Spam-Score: -4.2 (----) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Wed, 05 Dec 2012 21:16:14 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.0 (-----) >From 82f2b062c0e21d9a0d64f9ceab363d2a79f5a6eb Mon Sep 17 00:00:00 2001 From: Cojocaru Alexandru Date: Thu, 6 Dec 2012 03:03:41 +0100 Subject: [PATCH] cut: fix memory leak * src/cut.c (set_fields): don't allocate memory for `printable_field' if there are no finite ranges. The bug was introduced on 2012-2-7 via commit 2e636af. --- src/cut.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/src/cut.c b/src/cut.c index 4219d24..87abd15 100644 --- a/src/cut.c +++ b/src/cut.c @@ -500,14 +500,13 @@ set_fields (const char *fieldstr) if (rp[i].hi > max_range_endpoint) max_range_endpoint = rp[i].hi; } - if (max_range_endpoint < eol_range_start) - max_range_endpoint = eol_range_start; /* Allocate an array large enough so that it may be indexed by the field numbers corresponding to all finite ranges (i.e. '2-6' or '-4', but not '5-') in FIELDSTR. */ - printable_field = xzalloc (max_range_endpoint / CHAR_BIT + 1); + if (n_rp) + printable_field = xzalloc (max_range_endpoint / CHAR_BIT + 1); qsort (rp, n_rp, sizeof (rp[0]), compare_ranges); @@ -531,8 +530,11 @@ set_fields (const char *fieldstr) if (output_delimiter_specified && !complement - && eol_range_start && !is_printable_field (eol_range_start)) - mark_range_start (eol_range_start); + && eol_range_start + && printable_field && !is_printable_field (eol_range_start)) + { + mark_range_start (eol_range_start); + } free (rp); -- 1.8.0.1 -- Cojocaru Alexandru From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 06 11:42:40 2012 Received: (at 13098-done) by debbugs.gnu.org; 6 Dec 2012 16:42:41 +0000 Received: from localhost ([127.0.0.1]:57006 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TgeX2-0003Bq-GG for submit@debbugs.gnu.org; Thu, 06 Dec 2012 11:42:40 -0500 Received: from mx1.redhat.com ([209.132.183.28]:58768) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TgeWy-0003Bh-Ne for 13098-done@debbugs.gnu.org; Thu, 06 Dec 2012 11:42:38 -0500 Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id qB6GgLVb028579 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 6 Dec 2012 11:42:21 -0500 Received: from [10.36.116.69] (ovpn-116-69.ams2.redhat.com [10.36.116.69]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id qB6GgIoL017302 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 6 Dec 2012 11:42:20 -0500 Message-ID: <50C0CAE5.70704@draigBrady.com> Date: Thu, 06 Dec 2012 16:42:13 +0000 From: =?ISO-8859-1?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20120615 Thunderbird/13.0.1 MIME-Version: 1.0 To: Cojocaru Alexandru Subject: Re: bug#13098: [PATCH] cut.c: Fix memory leak References: <20121206031148.03492829e89a0b475caabade@gmx.com> In-Reply-To: <20121206031148.03492829e89a0b475caabade@gmx.com> Content-Type: multipart/mixed; boundary="------------050907010607080905030709" X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 13098-done Cc: 13098-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.9 (------) This is a multi-part message in MIME format. --------------050907010607080905030709 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id qB6GgLVb028579 On 12/06/2012 02:11 AM, Cojocaru Alexandru wrote: >>>From 82f2b062c0e21d9a0d64f9ceab363d2a79f5a6eb Mon Sep 17 00:00:00 2001 > From: Cojocaru Alexandru > Date: Thu, 6 Dec 2012 03:03:41 +0100 > Subject: [PATCH] cut: fix memory leak > > * src/cut.c (set_fields): don't allocate memory for > `printable_field' if there are no finite ranges. > The bug was introduced on 2012-2-7 via commit 2e636af. > --- > src/cut.c | 12 +++++++----- > 1 file changed, 7 insertions(+), 5 deletions(-) > > diff --git a/src/cut.c b/src/cut.c > index 4219d24..87abd15 100644 > --- a/src/cut.c > +++ b/src/cut.c > @@ -500,14 +500,13 @@ set_fields (const char *fieldstr) > if (rp[i].hi > max_range_endpoint) > max_range_endpoint =3D rp[i].hi; > } > - if (max_range_endpoint < eol_range_start) > - max_range_endpoint =3D eol_range_start; > > /* Allocate an array large enough so that it may be indexed by > the field numbers corresponding to all finite ranges > (i.e. '2-6' or '-4', but not '5-') in FIELDSTR. */ > > - printable_field =3D xzalloc (max_range_endpoint / CHAR_BIT + 1); > + if (n_rp) > + printable_field =3D xzalloc (max_range_endpoint / CHAR_BIT + 1); > > qsort (rp, n_rp, sizeof (rp[0]), compare_ranges); > > @@ -531,8 +530,11 @@ set_fields (const char *fieldstr) > > if (output_delimiter_specified > && !complement > - && eol_range_start && !is_printable_field (eol_range_start)) > - mark_range_start (eol_range_start); > + && eol_range_start > + && printable_field && !is_printable_field (eol_range_start)) > + { > + mark_range_start (eol_range_start); > + } > > free (rp); This looks right. subsequent accesses to the now not alloced bit array, are avoided when max_range_endpoint =3D 0. $ valgrind cut-before --output=3D" " -f1234567- /dev/null 2>&1 | grep rea= chable =3D=3D20949=3D=3D still reachable: 154,323 bytes in 2 blocks $ valgrind cut-after --output=3D" " -f1234567- /dev/null 2>&1 | grep reac= hable =3D=3D20816=3D=3D still reachable: 2 bytes in 1 blocks I wouldn't describe it as a leak though, rather a redundant allocation. Since this is an edge case minor performance issue, I don't think it needs a NEWS entry and will just adjust the commit summary a little to say "advoid a redundant memory allocation". Hmm, it might be a bit more consistent to guard all references to the bit vector array with max_range_endpoint? How about the attached? thanks! P=E1draig. --------------050907010607080905030709 Content-Type: text/x-patch; name="cut-avoid-alloc.diff" Content-Disposition: attachment; filename="cut-avoid-alloc.diff" Content-Transfer-Encoding: 7bit >From c40543afe9fd5a06fac60f47ae92775303f83b49 Mon Sep 17 00:00:00 2001 From: Cojocaru Alexandru Date: Thu, 6 Dec 2012 03:03:41 +0100 Subject: [PATCH] cut: avoid a redundant heap allocation * src/cut.c (set_fields): Don't allocate memory for `printable_field' if there are no finite ranges. The extra allocation was introduced via commit v8.10-3-g2e636af. --- src/cut.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/cut.c b/src/cut.c index dc83072..de9320c 100644 --- a/src/cut.c +++ b/src/cut.c @@ -500,14 +500,13 @@ set_fields (const char *fieldstr) if (rp[i].hi > max_range_endpoint) max_range_endpoint = rp[i].hi; } - if (max_range_endpoint < eol_range_start) - max_range_endpoint = eol_range_start; /* Allocate an array large enough so that it may be indexed by the field numbers corresponding to all finite ranges (i.e. '2-6' or '-4', but not '5-') in FIELDSTR. */ - printable_field = xzalloc (max_range_endpoint / CHAR_BIT + 1); + if (max_range_endpoint) + printable_field = xzalloc (max_range_endpoint / CHAR_BIT + 1); qsort (rp, n_rp, sizeof (rp[0]), compare_ranges); @@ -531,7 +530,8 @@ set_fields (const char *fieldstr) if (output_delimiter_specified && !complement - && eol_range_start && !is_printable_field (eol_range_start)) + && eol_range_start + && max_range_endpoint && !is_printable_field (eol_range_start)) mark_range_start (eol_range_start); free (rp); -- 1.7.6.4 --------------050907010607080905030709-- From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 06 12:37:18 2012 Received: (at 13098-done) by debbugs.gnu.org; 6 Dec 2012 17:37:18 +0000 Received: from localhost ([127.0.0.1]:57045 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TgfNu-0004S9-2l for submit@debbugs.gnu.org; Thu, 06 Dec 2012 12:37:18 -0500 Received: from mailout-eu.gmx.com ([213.165.64.42]:60188) by debbugs.gnu.org with smtp (Exim 4.72) (envelope-from ) id 1TgfNq-0004Rl-P1 for 13098-done@debbugs.gnu.org; Thu, 06 Dec 2012 12:37:15 -0500 Received: (qmail invoked by alias); 06 Dec 2012 16:59:22 -0000 Received: from unknown (EHLO smag-R59-R60-R61) [151.65.149.48] by mail.gmx.com (mp-eu003) with SMTP; 06 Dec 2012 17:59:22 +0100 X-Authenticated: #130707387 X-Provags-ID: V01U2FsdGVkX1+M9/1BUUuMX7dQAZCJRnVvE8v+GHackda2FzwVIn QjvSWqBLhxeBw0 Date: Thu, 6 Dec 2012 17:59:41 +0100 From: Cojocaru Alexandru To: =?ISO-8859-1?Q?P=E1draig?= Brady Subject: Re: bug#13098: [PATCH] cut.c: Fix memory leak Message-Id: <20121206175941.cfa3d9dda45a5d71c6e969f8@gmx.com> In-Reply-To: <50C0CAE5.70704@draigBrady.com> References: <20121206031148.03492829e89a0b475caabade@gmx.com> <50C0CAE5.70704@draigBrady.com> X-Mailer: Sylpheed 3.3.0 (GTK+ 2.24.13; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: > Hmm, it might be a bit more consistent to guard all > references to the bit vector array with max_range_endpoint? > How about the attached? I checked for `n_rp' and `printable_field' becuase it felt more natural. I reasoned as follows: first check if we have any finite ranges (n_rp) if yes, allocate memory for `printable_field'. Then check if `printable_field' has been allocated memory, if yes access it. Anyway both are good solutions. [...] Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (xojoc[at]gmx.com) 1.3 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net [Blocked - see ] -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [213.165.64.42 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% [score: 0.5000] X-Debbugs-Envelope-To: 13098-done Cc: 13098-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: > Hmm, it might be a bit more consistent to guard all > references to the bit vector array with max_range_endpoint? > How about the attached? I checked for `n_rp' and `printable_field' becuase it felt more natural. I reasoned as follows: first check if we have any finite ranges (n_rp) if yes, allocate memory for `printable_field'. Then check if `printable_field' has been allocated memory, if yes access it. Anyway both are good solutions. [...] Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 1.3 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net [Blocked - see ] -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [213.165.64.42 listed in list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (xojoc[at]gmx.com) -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% [score: 0.4961] > Hmm, it might be a bit more consistent to guard all > references to the bit vector array with max_range_endpoint? > How about the attached? I checked for `n_rp' and `printable_field' becuase it felt more natural. I reasoned as follows: first check if we have any finite ranges (n_rp) if yes, allocate memory for `printable_field'. Then check if `printable_field' has been allocated memory, if yes access it. Anyway both are good solutions. Best regards, Cojocaru Alexandru From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 06 23:33:13 2012 Received: (at 13098) by debbugs.gnu.org; 7 Dec 2012 04:33:13 +0000 Received: from localhost ([127.0.0.1]:57601 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Tgpcf-0003uk-8h for submit@debbugs.gnu.org; Thu, 06 Dec 2012 23:33:13 -0500 Received: from mx.meyering.net ([88.168.87.75]:40169) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Tgpcd-0003uc-GQ for 13098@debbugs.gnu.org; Thu, 06 Dec 2012 23:33:12 -0500 Received: from rho.meyering.net (rho.meyering.net [127.0.0.1]) by rho.meyering.net (Acme Bit-Twister) with ESMTP id 87782602DB; Fri, 7 Dec 2012 05:32:53 +0100 (CET) From: Jim Meyering To: 13098@debbugs.gnu.org Subject: Re: bug#13098: [PATCH] cut.c: Fix memory leak In-Reply-To: <50C0CAE5.70704@draigBrady.com> (=?iso-8859-1?Q?=22P=E1draig?= Brady"'s message of "Thu, 06 Dec 2012 16:42:13 +0000") References: <20121206031148.03492829e89a0b475caabade@gmx.com> <50C0CAE5.70704@draigBrady.com> Date: Fri, 07 Dec 2012 05:32:53 +0100 Message-ID: <87ip8ev5vu.fsf@rho.meyering.net> Lines: 26 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.5 (-) X-Debbugs-Envelope-To: 13098 Cc: xojoc@gmx.com, P@draigBrady.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.5 (-) P=E1draig Brady wrote: ... > How about the attached? > From: Cojocaru Alexandru > Date: Thu, 6 Dec 2012 03:03:41 +0100 > Subject: [PATCH] cut: avoid a redundant heap allocation > > * src/cut.c (set_fields): Don't allocate memory for > `printable_field' if there are no finite ranges. > The extra allocation was introduced via commit v8.10-3-g2e636af. ... Thanks to both of you. That's a fine bug fix, actually. Consider that before, this would fail on my 64-bit system: $ : | cut -b999999999999999999- cut: memory exhausted [Exit 1] $ Now, it no longer tries to allocate all that memory, so completes normally: $ : | cut -b999999999999999999- $ From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 07 01:41:35 2012 Received: (at 13098) by debbugs.gnu.org; 7 Dec 2012 06:41:35 +0000 Received: from localhost ([127.0.0.1]:57703 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Tgrct-0006uE-8S for submit@debbugs.gnu.org; Fri, 07 Dec 2012 01:41:35 -0500 Received: from mx.meyering.net ([88.168.87.75]:40349) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Tgrcq-0006u6-AZ for 13098@debbugs.gnu.org; Fri, 07 Dec 2012 01:41:33 -0500 Received: from rho.meyering.net (rho.meyering.net [127.0.0.1]) by rho.meyering.net (Acme Bit-Twister) with ESMTP id 42A3D600BB; Fri, 7 Dec 2012 07:41:14 +0100 (CET) From: Jim Meyering To: 13098@debbugs.gnu.org Subject: Re: bug#13098: [PATCH] cut.c: Fix memory leak In-Reply-To: <87ip8ev5vu.fsf@rho.meyering.net> (Jim Meyering's message of "Fri, 07 Dec 2012 05:32:53 +0100") References: <20121206031148.03492829e89a0b475caabade@gmx.com> <50C0CAE5.70704@draigBrady.com> <87ip8ev5vu.fsf@rho.meyering.net> Date: Fri, 07 Dec 2012 07:41:14 +0100 Message-ID: <87y5hatldh.fsf@rho.meyering.net> Lines: 53 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.5 (-) X-Debbugs-Envelope-To: 13098 Cc: xojoc@gmx.com, P@draigBrady.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.5 (-) Jim Meyering wrote: > P=E1draig Brady wrote: > ... >> How about the attached? > >> From: Cojocaru Alexandru >> Date: Thu, 6 Dec 2012 03:03:41 +0100 >> Subject: [PATCH] cut: avoid a redundant heap allocation >> >> * src/cut.c (set_fields): Don't allocate memory for >> `printable_field' if there are no finite ranges. >> The extra allocation was introduced via commit v8.10-3-g2e636af. > ... > > Thanks to both of you. > That's a fine bug fix, actually. > Consider that before, this would fail on my 64-bit system: > > $ : | cut -b999999999999999999- > cut: memory exhausted > [Exit 1] > $ > > Now, it no longer tries to allocate all that memory, so completes normall= y: > > $ : | cut -b999999999999999999- > $ Note that on a 32-bit system it fails the same way in both cases: $ : | src/cut -b999999999999999999- src/cut: byte offset '999999999999999999' is too large [Exit 1] And even with 2^32-1, it'll probably just allocate the memory, assuming 500MB doesn't cause trouble. To differentiate portably, we probably have to use ulimit: Limit VM to 22000k, but make the old cut require about 10x that: $ (ulimit -v 22000; :| cut -b$(echo 2^31|bc)-) cut: memory exhausted [Exit 1] The new, just-built one passes: $ (ulimit -v 22000; :| ./cut -b$(echo 2^31|bc)-) $ If someone feels like turning the above into a test case, please do (and update NEWS). Otherwise, I'll get to it over the weekend. From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 08 15:27:42 2012 Received: (at 13098) by debbugs.gnu.org; 8 Dec 2012 20:27:42 +0000 Received: from localhost ([127.0.0.1]:33241 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ThQzt-0001XJ-UZ for submit@debbugs.gnu.org; Sat, 08 Dec 2012 15:27:42 -0500 Received: from mx.meyering.net ([88.168.87.75]:42905) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ThQzr-0001XC-U7 for 13098@debbugs.gnu.org; Sat, 08 Dec 2012 15:27:41 -0500 Received: from rho.meyering.net (rho.meyering.net [127.0.0.1]) by rho.meyering.net (Acme Bit-Twister) with ESMTP id CF35F600E3; Sat, 8 Dec 2012 21:27:12 +0100 (CET) From: Jim Meyering To: 13098@debbugs.gnu.org Subject: Re: bug#13098: [PATCH] cut.c: Fix memory leak In-Reply-To: <87y5hatldh.fsf@rho.meyering.net> (Jim Meyering's message of "Fri, 07 Dec 2012 07:41:14 +0100") References: <20121206031148.03492829e89a0b475caabade@gmx.com> <50C0CAE5.70704@draigBrady.com> <87ip8ev5vu.fsf@rho.meyering.net> <87y5hatldh.fsf@rho.meyering.net> Date: Sat, 08 Dec 2012 21:27:12 +0100 Message-ID: <878v98s31b.fsf@rho.meyering.net> Lines: 139 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -4.2 (----) X-Debbugs-Envelope-To: 13098 Cc: xojoc@gmx.com, P@draigBrady.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.2 (----) Jim Meyering wrote: > Jim Meyering wrote: > >> P=E1draig Brady wrote: >> ... >>> How about the attached? >> >>> From: Cojocaru Alexandru >>> Date: Thu, 6 Dec 2012 03:03:41 +0100 >>> Subject: [PATCH] cut: avoid a redundant heap allocation >>> >>> * src/cut.c (set_fields): Don't allocate memory for >>> `printable_field' if there are no finite ranges. >>> The extra allocation was introduced via commit v8.10-3-g2e636af. >> ... >> >> Thanks to both of you. >> That's a fine bug fix, actually. >> Consider that before, this would fail on my 64-bit system: >> >> $ : | cut -b999999999999999999- >> cut: memory exhausted >> [Exit 1] >> $ >> >> Now, it no longer tries to allocate all that memory, so completes normal= ly: >> >> $ : | cut -b999999999999999999- >> $ > > Note that on a 32-bit system it fails the same way in both cases: > > $ : | src/cut -b999999999999999999- > src/cut: byte offset '999999999999999999' is too large > [Exit 1] > > And even with 2^32-1, it'll probably just allocate the memory, > assuming 500MB doesn't cause trouble. > To differentiate portably, we probably have to use ulimit: > > Limit VM to 22000k, but make the old cut require about 10x that: > > $ (ulimit -v 22000; :| cut -b$(echo 2^31|bc)-) > cut: memory exhausted > [Exit 1] > > The new, just-built one passes: > > $ (ulimit -v 22000; :| ./cut -b$(echo 2^31|bc)-) > $ > > If someone feels like turning the above into a test > case, please do (and update NEWS). Otherwise, I'll > get to it over the weekend. Here it is: >From a2833b8399f56bff6fe08eb212c01a5cb1b74606 Mon Sep 17 00:00:00 2001 From: Jim Meyering Date: Sat, 8 Dec 2012 12:04:14 -0800 Subject: [PATCH] tests: add test case and note that last week's cut change = is a bug fix * tests/misc/cut-huge-to-eol-range.sh: New test, showing that the change in v8.20-51-g7d03466 is a bug fix after all. * tests/local.mk (all_tests): Add it. * NEWS (Bug fixes): Mention it. --- NEWS | 4 ++++ tests/local.mk | 1 + tests/misc/cut-huge-to-eol-range.sh | 30 ++++++++++++++++++++++++++++++ 3 files changed, 35 insertions(+) create mode 100755 tests/misc/cut-huge-to-eol-range.sh diff --git a/NEWS b/NEWS index 7c17869..a7a5bc3 100644 --- a/NEWS +++ b/NEWS @@ -10,6 +10,10 @@ GNU coreutils NEWS -*= - outline -*- ** Bug fixes + cut with a range like "N-" no longer allocates N/8 bytes. That buffer + would never be used, and allocation failure could cause cut to fail. + [bug introduced in coreutils-8.10] + cut no longer accepts the invalid range 0-, which made it print empty li= nes. Instead, cut now fails and emits an appropriate diagnostic. [This bug was present in "the beginning".] diff --git a/tests/local.mk b/tests/local.mk index d5bb6f7..5eeddd5 100644 --- a/tests/local.mk +++ b/tests/local.mk @@ -246,6 +246,7 @@ all_tests =3D \ tests/misc/pwd-option.sh \ tests/misc/chcon-fail.sh \ tests/misc/cut.pl \ + tests/misc/cut-huge-to-eol-range.sh \ tests/misc/wc.pl \ tests/misc/wc-files0-from.pl \ tests/misc/wc-files0.sh \ diff --git a/tests/misc/cut-huge-to-eol-range.sh b/tests/misc/cut-huge-to-e= ol-range.sh new file mode 100755 index 0000000..fea8505 --- /dev/null +++ b/tests/misc/cut-huge-to-eol-range.sh @@ -0,0 +1,30 @@ +#!/bin/sh +# Ensure that cut does not allocate mem for a range like -b9999999999999- + +# Copyright (C) 2012 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +. "${srcdir=3D.}/tests/init.sh"; path_prepend_ ./src +print_ver_ cut +require_ulimit_ + +# 2^31-1 +# From coreutils-8.10 through 8.20, this would make cut try to allocate +# a 256MiB bit vector. With a 20MB limit on VM, the following would fail. +(ulimit -v 20000; : | cut -b2147483647- > err 2>&1) || fail=3D1 + +compare /dev/null err || fail=3D1 + +Exit $fail -- 1.8.0.1.352.gfb4c622 From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 09 07:58:36 2012 Received: (at 13098) by debbugs.gnu.org; 9 Dec 2012 12:58:36 +0000 Received: from localhost ([127.0.0.1]:33681 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ThgSp-0000il-HO for submit@debbugs.gnu.org; Sun, 09 Dec 2012 07:58:35 -0500 Received: from mx1.redhat.com ([209.132.183.28]:15134) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ThgSn-0000ie-NG for 13098@debbugs.gnu.org; Sun, 09 Dec 2012 07:58:34 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id qB9Cw3tk030006 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Sun, 9 Dec 2012 07:58:03 -0500 Received: from [10.36.116.39] (ovpn-116-39.ams2.redhat.com [10.36.116.39]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id qB9Cw0vd022397 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 9 Dec 2012 07:58:01 -0500 Message-ID: <50C48AD7.20601@draigBrady.com> Date: Sun, 09 Dec 2012 12:57:59 +0000 From: =?ISO-8859-1?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20120615 Thunderbird/13.0.1 MIME-Version: 1.0 To: Jim Meyering Subject: Re: bug#13098: [PATCH] cut.c: Fix memory leak References: <20121206031148.03492829e89a0b475caabade@gmx.com> <50C0CAE5.70704@draigBrady.com> <87ip8ev5vu.fsf@rho.meyering.net> <87y5hatldh.fsf@rho.meyering.net> <878v98s31b.fsf@rho.meyering.net> In-Reply-To: <878v98s31b.fsf@rho.meyering.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id qB9Cw3tk030006 X-Spam-Score: -4.2 (----) X-Debbugs-Envelope-To: 13098 Cc: xojoc@gmx.com, 13098@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.0 (-----) On 12/08/2012 08:27 PM, Jim Meyering wrote: > +# 2^31-1 > +# From coreutils-8.10 through 8.20, this would make cut try to allocat= e > +# a 256MiB bit vector. With a 20MB limit on VM, the following would f= ail. > +(ulimit -v 20000; : | cut -b2147483647- > err 2>&1) || fail=3D1 > + > +compare /dev/null err || fail=3D1 Perhaps use INT_MAX from getlimits? In either case +1 thanks, P=E1draig. From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 09 18:16:55 2012 Received: (at 13098) by debbugs.gnu.org; 9 Dec 2012 23:16:55 +0000 Received: from localhost ([127.0.0.1]:34419 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Thq7C-0008JL-8i for submit@debbugs.gnu.org; Sun, 09 Dec 2012 18:16:55 -0500 Received: from mx.meyering.net ([88.168.87.75]:44534) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Thq79-0008J9-Gk for 13098@debbugs.gnu.org; Sun, 09 Dec 2012 18:16:52 -0500 Received: from rho.meyering.net (rho.meyering.net [127.0.0.1]) by rho.meyering.net (Acme Bit-Twister) with ESMTP id 747F9601EB; Mon, 10 Dec 2012 00:16:18 +0100 (CET) From: Jim Meyering To: =?iso-8859-1?Q?P=E1draig?= Brady Subject: Re: bug#13098: [PATCH] cut.c: Fix memory leak In-Reply-To: <50C48AD7.20601@draigBrady.com> (=?iso-8859-1?Q?=22P=E1draig?= Brady"'s message of "Sun, 09 Dec 2012 12:57:59 +0000") References: <20121206031148.03492829e89a0b475caabade@gmx.com> <50C0CAE5.70704@draigBrady.com> <87ip8ev5vu.fsf@rho.meyering.net> <87y5hatldh.fsf@rho.meyering.net> <878v98s31b.fsf@rho.meyering.net> <50C48AD7.20601@draigBrady.com> Date: Mon, 10 Dec 2012 00:16:18 +0100 Message-ID: <87y5h6olz1.fsf@rho.meyering.net> Lines: 34 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.5 (-) X-Debbugs-Envelope-To: 13098 Cc: xojoc@gmx.com, 13098@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.3 (--) P=E1draig Brady wrote: > On 12/08/2012 08:27 PM, Jim Meyering wrote: >> +# 2^31-1 > >> +# From coreutils-8.10 through 8.20, this would make cut try to allocate >> +# a 256MiB bit vector. With a 20MB limit on VM, the following would fa= il. >> +(ulimit -v 20000; : | cut -b2147483647- > err 2>&1) || fail=3D1 >> + >> +compare /dev/null err || fail=3D1 > > Perhaps use INT_MAX from getlimits? > In either case +1 Good idea. Thanks for the review. Pushed. Here's the delta. diff --git a/tests/misc/cut-huge-to-eol-range.sh b/tests/misc/cut-huge-to-e= ol-range.sh index fea8505..4e3ee03 100755 --- a/tests/misc/cut-huge-to-eol-range.sh +++ b/tests/misc/cut-huge-to-eol-range.sh @@ -19,11 +19,11 @@ . "${srcdir=3D.}/tests/init.sh"; path_prepend_ ./src print_ver_ cut require_ulimit_ +getlimits_ -# 2^31-1 # From coreutils-8.10 through 8.20, this would make cut try to allocate # a 256MiB bit vector. With a 20MB limit on VM, the following would fail. -(ulimit -v 20000; : | cut -b2147483647- > err 2>&1) || fail=3D1 +(ulimit -v 20000; : | cut -b$INT_MAX- > err 2>&1) || fail=3D1 compare /dev/null err || fail=3D1 From unknown Fri Aug 15 14:46:49 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 07 Jan 2013 12:24:46 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator