Subject: bug#24795: closed (Re: bug#24795: sha1sum cannot find file to check)

From: [email-masked] (GNU bug Tracking System)
Date: Wed, 26 Oct 2016 14:56:02 +0000

Your bug report

#24795: sha1sum cannot find file to check

which was filed against the coreutils package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
24795: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24795
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#24795: sha1sum cannot find file to check
From: Pádraig Brady <[email-masked]>
To: Francesco Turco <[email-masked]>, [email-masked]
Date: Wed, 26 Oct 2016 15:55:56 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

On 26/10/16 09:30, Francesco Turco wrote:
> On Tue, 2016-10-25 at 19:52 +0100, Pádraig Brady wrote:
>> This seems like the txt file is in DOS format with \r\n line endings.
>> If you remove the --ignore-missing option, are you presented with
>> $'\r' representations in the missing file names?
>> Though when I downloaded the file it was in unix format so I'm
>> confused
> 
> If I remove the --ignore-missing option no \r characters appear in the
> output, so the checksum file uses Unix line endings.
> 
> If I convert the checksum file to DOS line endings with unix2dos I can
> see \r line endings in the output.
> 
> There is another difference by using --ignore-missing: when using DOS
> line endings sha1sum immediately says "no file was verified", without
> checking anything; when using Unix line endings it says the same thing
> but after a few seconds, so it must have checked something behind the
> scenes.

Ouch that is a bug, triggered when --ignore-missing is used
with checksums that start with 00....
The attached should fix it up.

thanks,
Pádraig

From 58ab436353205a56372bf9e899c453b2d8fbb175 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <[email-masked]>
Date: Wed, 26 Oct 2016 15:45:01 +0100
Subject: [PATCH] md5sum,sha*sum: fix --ignore-missing with checksums starting
 with 00

* NEWS: Mention the fix.
* src/md5sum.c (digest_file): Add a new MISSING parameter to
return whether the file was missing, separately from the digest.
* tests/misc/md5sum.pl: Add a test case.
Fixes http://bugs.gnu.org/24795
---
 NEWS                 |  4 ++++
 src/md5sum.c         | 18 +++++++++++-------
 tests/misc/md5sum.pl |  7 +++++++
 3 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/NEWS b/NEWS
index e68de05..9548de8 100644
--- a/NEWS
+++ b/NEWS
@@ -28,6 +28,10 @@ GNU coreutils NEWS                                    -*- outline -*-
   ls --time-style no longer mishandles '%%b' in formats.
   [bug introduced in coreutils-7.2]
 
+  md5sum --check --ignore-missing no longer treats files with checksums
+  starting with "00" as missing.  This also affects sha*sum.
+  [bug introduced with the --ignore-missing feature in coreutils-8.25]
+
   nl now resets numbering for each page section rather than just for each page.
   [This bug was present in "the beginning".]
 
diff --git a/src/md5sum.c b/src/md5sum.c
index 5a39a4f..128cd0b 100644
--- a/src/md5sum.c
+++ b/src/md5sum.c
@@ -466,12 +466,15 @@ print_filename (char const *file, bool escape)
    Return true if successful.  */
 
 static bool
-digest_file (const char *filename, int *binary, unsigned char *bin_result)
+digest_file (const char *filename, int *binary, unsigned char *bin_result,
+             bool *missing)
 {
   FILE *fp;
   int err;
   bool is_stdin = STREQ (filename, "-");
 
+  *missing = false;
+
   if (is_stdin)
     {
       have_read_stdin = true;
@@ -491,7 +494,7 @@ digest_file (const char *filename, int *binary, unsigned char *bin_result)
         {
           if (ignore_missing && errno == ENOENT)
             {
-              *bin_result = '\0';
+              *missing = true;
               return true;
             }
           error (0, errno, "%s", quotef (filename));
@@ -604,6 +607,7 @@ digest_check (const char *checkfile_name)
                                           '8', '9', 'a', 'b',
                                           'c', 'd', 'e', 'f' };
           bool ok;
+          bool missing;
           /* Only escape in the edge case producing multiple lines,
              to ease automatic processing of status output.  */
           bool needs_escape = ! status_only && strchr (filename, '\n');
@@ -611,7 +615,7 @@ digest_check (const char *checkfile_name)
           properly_formatted_lines = true;
 
           *bin_buffer = '\1'; /* flag set to 0 for ignored missing files.  */
-          ok = digest_file (filename, &binary, bin_buffer);
+          ok = digest_file (filename, &binary, bin_buffer, &missing);
 
           if (!ok)
             {
@@ -624,10 +628,9 @@ digest_check (const char *checkfile_name)
                   printf (": %s\n", _("FAILED open or read"));
                 }
             }
-          else if (ignore_missing && ! *bin_buffer)
+          else if (ignore_missing && missing)
             {
-              /* Treat an empty buffer as meaning a missing file,
-                 which is ignored with --ignore-missing.  */
+              /* Just ignore missing files with --ignore-missing.  */
               ;
             }
           else
@@ -878,8 +881,9 @@ main (int argc, char **argv)
       else
         {
           int file_is_binary = binary;
+          bool missing;
 
-          if (! digest_file (file, &file_is_binary, bin_buffer))
+          if (! digest_file (file, &file_is_binary, bin_buffer, &missing))
             ok = false;
           else
             {
diff --git a/tests/misc/md5sum.pl b/tests/misc/md5sum.pl
index 2eb6369..6ea7457 100755
--- a/tests/misc/md5sum.pl
+++ b/tests/misc/md5sum.pl
@@ -149,6 +149,13 @@ my @Tests =
                                 {ERR=>
                                     "md5sum: f.md5: no file was verified\n"},
                                 {EXIT=> 1}],
+     # coreutils-8.25 with --ignore-missing treated checksums starting with 00
+     # as if the file was not present
+     ['check-ignore-missing-6', '--check', '--ignore-missing',
+                                {AUX=> {f=> '9t'}},
+                                {IN=> {'f.md5' =>
+                                    "006999e6df389641adf1fa3a74801d9d  f\n"}},
+                                {OUT=>"f: OK\n"}],
      ['bsd-segv', '--check', {IN=> {'z' => "MD5 ("}}, {EXIT=> 1},
       {ERR=> "$prog: z: no properly formatted MD5 checksum lines found\n"}],
 
-- 
2.5.5

Subject: sha1sum cannot find file to check
From: Francesco Turco <[email-masked]>
To: [email-masked]
Date: Tue, 25 Oct 2016 19:26:24 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

I'm using coreutils-8.25-2 on a Parabola GNU/Linux-libre system.
I think I found a bug with the "sha1sum" command.

Steps to reproduce:
1) wget https://dumps.wikimedia.org/enwikisource/20161020/enwikisource-
20161020-pages-articles.xml.bz2
2) wget https://dumps.wikimedia.org/enwikisource/20161020/enwikisource-
20161020-sha1sums.txt
3) sha1sum -c enwikisource-20161020-sha1sums.txt --ignore-missing

Result:
sha1sum: enwikisource-20161020-sha1sums.txt: no file was verified

Expected result:
enwikisource-20161020-pages-articles.xml.bz2: OK

-- 
https://www.fturco.net/