From debbugs-submit-bounces@debbugs.gnu.org Mon May 30 13:53:52 2011 Received: (at submit) by debbugs.gnu.org; 30 May 2011 17:53:53 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QR6f2-00063Q-2j for submit@debbugs.gnu.org; Mon, 30 May 2011 13:53:52 -0400 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QR044-0004ah-9V for submit@debbugs.gnu.org; Mon, 30 May 2011 06:51:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QR03y-0000FH-FC for submit@debbugs.gnu.org; Mon, 30 May 2011 06:51:11 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([140.186.70.17]:55154) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QR03y-0000FD-Df for submit@debbugs.gnu.org; Mon, 30 May 2011 06:51:10 -0400 Received: from eggs.gnu.org ([140.186.70.92]:34565) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QR03w-0004S3-OI for bug-coreutils@gnu.org; Mon, 30 May 2011 06:51:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QR03v-0000EV-E8 for bug-coreutils@gnu.org; Mon, 30 May 2011 06:51:08 -0400 Received: from mx2.sogmail.nl ([62.197.130.49]:46055) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QR03v-0000Dv-9k for bug-coreutils@gnu.org; Mon, 30 May 2011 06:51:07 -0400 Received: from localhost (localhost [127.0.0.1]) by mx2.sogmail.nl (Postfix) with ESMTP id 9969EE224 for ; Mon, 30 May 2011 12:51:04 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mx2.sogmail.nl Received: from mx2.sogmail.nl ([127.0.0.1]) by localhost (mx2.sogmail.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IFqGpzXS3OiX for ; Mon, 30 May 2011 12:51:02 +0200 (CEST) Received: from AMS-EX02.Greenpeak.com (unknown [62.197.130.53]) by mx2.sogmail.nl (Postfix) with ESMTPS id 7191DE218 for ; Mon, 30 May 2011 12:51:02 +0200 (CEST) Received: from [192.168.48.122] (192.168.48.122) by AMS-EX02.Greenpeak.com (192.168.45.12) with Microsoft SMTP Server id 14.0.722.0; Mon, 30 May 2011 12:51:01 +0200 Message-ID: <4DE37695.3050601@greenpeak.com> Date: Mon, 30 May 2011 12:51:01 +0200 From: Theo Band User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: Subject: Bug in sha1sum? X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -6.6 (------) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Mon, 30 May 2011 13:53:50 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.6 (------) Hi I'm not sure, but I think I found a bug in sha1sum. It's easy to reproduce with any file that contains a backslash (\) in the name: echo test > test $ sha1sum test 4e1243bd22c66e76c2ba9eddc1f91394e57f9f83 test $ mv test 'test\test' $ sha1sum 'test\test' \4e1243bd22c66e76c2ba9eddc1f91394e57f9f83 test\\test I expect the file sha1sum to be the same after renaming the file (a backslash is prepended to the otherwise correct result). sha1sum --version sha1sum (GNU coreutils) 5.97 coreutils-5.97-23.el5_6.4 Kind regards, Theo Band -- GreenPeak Technologies ------------------------------------------------------------------------ Phone : +31 30 711 5622 Vinkenburgstraat 2a E-mail: Theo.Band@greenpeak.com 3512AB Utrecht Skype : Theo.Band-greenpeak The Netherlands http://www.greenpeak.com .-. CONFIDENTIALITY: this message, including possible attachment(s), /v\ constitutes confidential GreenPeak information, intended for the // \\ use of above named addressee(s) only; any other use or /( )\ disclosure to anyone other than addressee(s), is prohibited. ^^-^^ Chamber of Commerce NL-3210.56.42. ------------------------------------------------------------------------ From debbugs-submit-bounces@debbugs.gnu.org Mon May 30 19:03:29 2011 Received: (at 8766) by debbugs.gnu.org; 30 May 2011 23:03:30 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QRBUf-0004If-E5 for submit@debbugs.gnu.org; Mon, 30 May 2011 19:03:29 -0400 Received: from c-67-162-90-113.hsd1.in.comcast.net ([67.162.90.113] helo=kosh.dhis.org) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1QRBUd-0004IT-9s for 8766@debbugs.gnu.org; Mon, 30 May 2011 19:03:27 -0400 Received: (qmail 450 invoked by uid 1000); 30 May 2011 23:03:21 -0000 Message-ID: <20110530230321.448.qmail@kosh.dhis.org> From: "Alan Curry" Subject: Re: bug#8766: Bug in sha1sum? To: theo.band@greenpeak.com (Theo Band) Date: Mon, 30 May 2011 18:03:21 -0500 (GMT+5) In-Reply-To: <4DE37695.3050601@greenpeak.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 8766 Cc: 8766@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -0.8 (/) Theo Band writes: > > Hi > > I'm not sure, but I think I found a bug in sha1sum. It's easy to > reproduce with any file that contains a backslash (\) in the name: > echo test > test > $ sha1sum test > 4e1243bd22c66e76c2ba9eddc1f91394e57f9f83 test > $ mv test 'test\test' > $ sha1sum 'test\test' > \4e1243bd22c66e76c2ba9eddc1f91394e57f9f83 test\\test > > I expect the file sha1sum to be the same after renaming the file (a > backslash is prepended to the otherwise correct result). This result violated my expectations too, but it turns out to be a documented feature: For each FILE, `md5sum' outputs the MD5 checksum, a flag indicating a binary or text input file, and the file name. If FILE contains a backslash or newline, the line is started with a backslash, and each problematic character in the file name is escaped with a backslash, making the output unambiguous even in the presence of arbitrary file names. If FILE is omitted or specified as `-', standard input is read. (the sha*sum utilities all refer back to md5sum's description) I better go fix all my scripts that rely on /^[0-9a-f]{32} / -- Alan Curry From debbugs-submit-bounces@debbugs.gnu.org Tue May 31 03:39:57 2011 Received: (at 8766) by debbugs.gnu.org; 31 May 2011 07:39:58 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QRJYT-0006uR-3a for submit@debbugs.gnu.org; Tue, 31 May 2011 03:39:57 -0400 Received: from mx2.sogmail.nl ([62.197.130.49]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QRJYQ-0006uE-HJ for 8766@debbugs.gnu.org; Tue, 31 May 2011 03:39:55 -0400 Received: from localhost (localhost [127.0.0.1]) by mx2.sogmail.nl (Postfix) with ESMTP id 6D4D5E2A6; Tue, 31 May 2011 09:39:48 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mx2.sogmail.nl Received: from mx2.sogmail.nl ([127.0.0.1]) by localhost (mx2.sogmail.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1eEHc3YKItAV; Tue, 31 May 2011 09:39:46 +0200 (CEST) Received: from AMS-EX02.Greenpeak.com (unknown [62.197.130.53]) by mx2.sogmail.nl (Postfix) with ESMTPS id 7C657E2A4; Tue, 31 May 2011 09:39:46 +0200 (CEST) Received: from [192.168.48.146] (192.168.48.146) by AMS-EX02.Greenpeak.com (192.168.45.12) with Microsoft SMTP Server id 14.0.722.0; Tue, 31 May 2011 09:39:45 +0200 Message-ID: <4DE49B41.3060201@greenpeak.com> Date: Tue, 31 May 2011 09:39:45 +0200 From: Theo Band User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: Alan Curry Subject: Re: bug#8766: Bug in sha1sum? References: <20110530230321.448.qmail@kosh.dhis.org> In-Reply-To: <20110530230321.448.qmail@kosh.dhis.org> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Spam-Score: -4.6 (----) X-Debbugs-Envelope-To: 8766 Cc: 8766@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.9 (---) On 05/31/2011 01:03 AM, Alan Curry wrote: > Theo Band writes: >> Hi >> >> I'm not sure, but I think I found a bug in sha1sum. It's easy to >> reproduce with any file that contains a backslash (\) in the name: >> echo test > test >> $ sha1sum test >> 4e1243bd22c66e76c2ba9eddc1f91394e57f9f83 test >> $ mv test 'test\test' >> $ sha1sum 'test\test' >> \4e1243bd22c66e76c2ba9eddc1f91394e57f9f83 test\\test >> >> I expect the file sha1sum to be the same after renaming the file (a >> backslash is prepended to the otherwise correct result). > This result violated my expectations too, but it turns out to be a documented > feature: > > For each FILE, `md5sum' outputs the MD5 checksum, a flag indicating > a binary or text input file, and the file name. If FILE contains a > backslash or newline, the line is started with a backslash, and each > problematic character in the file name is escaped with a backslash, > making the output unambiguous even in the presence of arbitrary file > names. If FILE is omitted or specified as `-', standard input is read. > > (the sha*sum utilities all refer back to md5sum's description) > > I better go fix all my scripts that rely on /^[0-9a-f]{32} / > man sha1sum, info sha1sum and sha1sum --help don't show me this info. Instead I read this: > The default mode is to print a line with checksum, a character indicating type (`*' for binary, ` ' for text), and name for each FILE. Would that mean the documentation in the coreutils-5.97-23.el5_6.4 is outdated? If so, is there perhaps an undocumented option that does not output this backslash? I make an index of all my files to find duplicates. The backslash doesn't help. Theo From debbugs-submit-bounces@debbugs.gnu.org Tue May 31 03:56:36 2011 Received: (at 8766) by debbugs.gnu.org; 31 May 2011 07:56:36 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QRJoZ-0007GV-Mv for submit@debbugs.gnu.org; Tue, 31 May 2011 03:56:35 -0400 Received: from mx.meyering.net ([82.230.74.64]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QRJoY-0007GK-6X for 8766@debbugs.gnu.org; Tue, 31 May 2011 03:56:35 -0400 Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id 835CD60060; Tue, 31 May 2011 09:56:28 +0200 (CEST) From: Jim Meyering To: Theo Band Subject: Re: bug#8766: Bug in sha1sum? In-Reply-To: <4DE49B41.3060201@greenpeak.com> (Theo Band's message of "Tue, 31 May 2011 09:39:45 +0200") References: <20110530230321.448.qmail@kosh.dhis.org> <4DE49B41.3060201@greenpeak.com> Date: Tue, 31 May 2011 09:56:28 +0200 Message-ID: <87r57fjob7.fsf@rho.meyering.net> Lines: 56 MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: 8766 Cc: 8766@debbugs.gnu.org, Alan Curry X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.0 (------) Theo Band wrote: > On 05/31/2011 01:03 AM, Alan Curry wrote: >> Theo Band writes: >>> Hi >>> >>> I'm not sure, but I think I found a bug in sha1sum. It's easy to >>> reproduce with any file that contains a backslash (\) in the name: >>> echo test > test >>> $ sha1sum test >>> 4e1243bd22c66e76c2ba9eddc1f91394e57f9f83 test >>> $ mv test 'test\test' >>> $ sha1sum 'test\test' >>> \4e1243bd22c66e76c2ba9eddc1f91394e57f9f83 test\\test >>> >>> I expect the file sha1sum to be the same after renaming the file (a >>> backslash is prepended to the otherwise correct result). >> This result violated my expectations too, but it turns out to be a documented >> feature: >> >> For each FILE, `md5sum' outputs the MD5 checksum, a flag indicating >> a binary or text input file, and the file name. If FILE contains a >> backslash or newline, the line is started with a backslash, and each >> problematic character in the file name is escaped with a backslash, >> making the output unambiguous even in the presence of arbitrary file >> names. If FILE is omitted or specified as `-', standard input is read. >> >> (the sha*sum utilities all refer back to md5sum's description) >> >> I better go fix all my scripts that rely on /^[0-9a-f]{32} / >> > man sha1sum, info sha1sum and sha1sum --help don't show me this info. > Instead I read this: > >> The default mode is to print a line with checksum, a character > indicating type (`*' for binary, ` ' for text), and name for each FILE. > > Would that mean the documentation in the coreutils-5.97-23.el5_6.4 is > outdated? If so, is there perhaps an undocumented option that does not > output this backslash? > I make an index of all my files to find duplicates. The backslash > doesn't help. That feature is required to allow checking the hash of any file name that contains newlines. There is no option to disable it. That omission in the documentation was corrected by COREUTILS-6_8-69-g826ff08. If you're sure you have no newline-afflicted file name, you can safely filter out the backslashes with this: sed 's/^\\//;s/\\\\/\\/g' E.g., $ touch a\\b $ md5sum a\\b | sed 's/^\\//;s/\\\\/\\/g' | md5sum -c - a\b: OK From debbugs-submit-bounces@debbugs.gnu.org Tue May 31 04:01:02 2011 Received: (at control) by debbugs.gnu.org; 31 May 2011 08:01:03 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QRJss-0007Mx-1t for submit@debbugs.gnu.org; Tue, 31 May 2011 04:01:02 -0400 Received: from mx.meyering.net ([82.230.74.64]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QRJsq-0007MX-Lp for control@debbugs.gnu.org; Tue, 31 May 2011 04:01:01 -0400 Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id 07DFC60A54; Tue, 31 May 2011 10:00:54 +0200 (CEST) From: Jim Meyering To: control@debbugs.gnu.org Subject: Date: Tue, 31 May 2011 10:00:48 +0200 Message-ID: <87lixnjo3z.fsf@rho.meyering.net> Lines: 3 MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -3.6 (---) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.6 (---) tags 8766 notabug close 8766 thanks From debbugs-submit-bounces@debbugs.gnu.org Tue May 31 04:52:35 2011 Received: (at 8766-done) by debbugs.gnu.org; 31 May 2011 08:52:36 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QRKgl-0001Oe-Nr for submit@debbugs.gnu.org; Tue, 31 May 2011 04:52:35 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1QRKgi-0001OH-US for 8766-done@debbugs.gnu.org; Tue, 31 May 2011 04:52:33 -0400 Received: (qmail 81935 invoked from network); 31 May 2011 08:52:26 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 31 May 2011 08:52:26 -0000 Message-ID: <4DE4ABB0.9040301@draigBrady.com> Date: Tue, 31 May 2011 09:49:52 +0100 From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: Theo Band Subject: Re: bug#8766: Bug in sha1sum? References: <4DE37695.3050601@greenpeak.com> In-Reply-To: <4DE37695.3050601@greenpeak.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.7 (--) X-Debbugs-Envelope-To: 8766-done Cc: 8766-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) tags 8766 notabug On 30/05/11 11:51, Theo Band wrote: > Hi > > I'm not sure, but I think I found a bug in sha1sum. It's easy to > reproduce with any file that contains a backslash (\) in the name: > echo test > test > $ sha1sum test > 4e1243bd22c66e76c2ba9eddc1f91394e57f9f83 test > $ mv test 'test\test' > $ sha1sum 'test\test' > \4e1243bd22c66e76c2ba9eddc1f91394e57f9f83 test\\test > > I expect the file sha1sum to be the same after renaming the file (a > backslash is prepended to the otherwise correct result). > > sha1sum --version > sha1sum (GNU coreutils) 5.97 > coreutils-5.97-23.el5_6.4 This is expected. Here is a shell function I use in FSlint to clean the output from these utilities when we know we'll not have files with \n chars. cleanup_sum() { sed ' # md5sum and sha1sum et. al. from coreutils at least, # to deal with \n in filenames, convert any \ and \n chars # to \\ and \\n respectively. Currently we ignore files with \n # so just undo this problematic escaping /^\\/{s/^\\//; s/\\\\/\\/g}; # These utils also add a "*" flag character for normal files # on platforms where O_BINARY is significant (like CYGWIN). # We always process in binary mode and so remove that flag here s/^\([^ ]*\) \*/\1 /; ' } So you can just: sha1sum test | cleanup_sum cheers, Pádraig. From unknown Tue Aug 19 01:59:01 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 28 Jun 2011 11:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator