Subject: Re: bug#23073: wc reports wrong byte counts when using '--from-files0=-'

From: Pádraig Brady <[email-masked]>
Date: Mon, 19 Dec 2016 19:00:31 +0000

On 21/03/16 15:16, Pádraig Brady wrote:
> On 21/03/16 00:59, William R. Fraser wrote:
>> When wc gets its list of files by reading from stdin, using the argument
>> '--from-files0=-', it reuses the same fstatus struct for each file.
>>
>> The problem is that the 'wc' function checks the 'failed' member of this
>> struct and if it is <=0, it skips doing fstat on the file. The main loop
>> doesn't reset this value between files, so only the first file has fstat
>> done on it.
>>
>> This can result in the 'wc' function seeking past the end of
>> subsequent files and then over-reporting their byte counts.
>>
>> See the attached patch, which resets the fstatus struct in between files
>> when reading the file list from stdin.
> 
> Ouch. This seems to be since v7.0-96-gc2e56e0
> It would also mean there would be a lot of redundant reading
> if the initial file was significantly smaller than any other file.
> 
> $ truncate -s1G wc.big
> $ touch wc.small
> $ printf '%s\0' wc.big wc.small | wc -c --files0-from=-
> 1073741824 wc.big
> 1073741760 wc.small
> 2147483584 total

Sorry for the delay.
I didn't go far enough back in my TODO list so missed this.
Proposed patch attached.

thanks,
Pádraig

>From 31bba96cb7dc565719c396b5973085e644939fd5 Mon Sep 17 00:00:00 2001
From: "William R. Fraser" <[email-masked]>
Date: Sun, 20 Mar 2016 17:44:09 -0700
Subject: [PATCH] wc: fix wrong byte counts when using --files-from0

* src/wc.c (main): Reset fstatus[0].failed between files when reusing
the fstatus[0] entry in --files-from0 mode.  This ensures a stat() is
done for each file, avoid incorrect counts and redundant reading.
* NEWS: Mention the bug fix.
* tests/misc/wc-files0.sh: Add a test case.
Fixes http://bugs.gnu.org/23073
---
 NEWS                    |  5 +++++
 src/wc.c                |  3 +++
 tests/misc/wc-files0.sh | 13 ++++++++++++-
 3 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/NEWS b/NEWS
index 179c19b..1ed5bd9 100644
--- a/NEWS
+++ b/NEWS
@@ -8,6 +8,11 @@ GNU coreutils NEWS                                    -*- outline -*-
   158909489063877810457 and 222087527029934481871.
   [bug introduced in coreutils-8.20]
 
+  wc --bytes --files0-from now correctly reports byte counts.
+  Previously it may have returned values that were too large,
+  depending on the size of the first file processed.
+  [bug introduced in coreutils-7.1]
+
 
 * Noteworthy changes in release 8.26 (2016-11-30) [stable]
 
diff --git a/src/wc.c b/src/wc.c
index 412bda0..64df50c 100644
--- a/src/wc.c
+++ b/src/wc.c
@@ -807,6 +807,9 @@ main (int argc, char **argv)
         ok = false;
       else
         ok &= wc_file (file_name, &fstatus[nfiles ? i : 0]);
+
+      if (! nfiles)
+        fstatus[0].failed = 1;
     }
  argv_iter_done:
 
diff --git a/tests/misc/wc-files0.sh b/tests/misc/wc-files0.sh
index 12b7d6a..d92a010 100755
--- a/tests/misc/wc-files0.sh
+++ b/tests/misc/wc-files0.sh
@@ -25,7 +25,7 @@ printf '2b\n2w\n' |tr '\n' '\0' > names || framework_failure_

 wc --files0-from=names > out || fail=1
-cat <<\EOF > exp || fail=1
+cat <<\EOF > exp || framework_failure_
  1  1  2 2b
  1  2  8 2w
  2  3 10 total
@@ -48,4 +48,15 @@ printf '%s\0' "$nlname" | wc --files0-from=- > out || fail=1
 printf '%s\n' "0 0 0 '1'$'\\n''2'" > exp || framework_failure_
 compare exp out || fail=1
 
+# Ensure correct byte counts, which fails between v7.1 and v8.26 inclusive
+truncate -s1G wc.big || framework_failure_
+touch wc.small || framework_failure_
+printf '%s\0' wc.big wc.small | wc -c --files0-from=- >out || fail=1
+cat <<\EOF > exp || framework_failure_
+1073741824 wc.big
+0 wc.small
+1073741824 total
+EOF
+compare exp out || fail=1
+
 Exit $fail
-- 
2.5.5