Subject: bug#18402: closed (Re: [bug-diffutils] bug#18402: Wrong output for single character files without newline)

From: [email-masked] (GNU bug Tracking System)
Date: Wed, 03 Sep 2014 23:07:02 +0000

Your bug report

#18402: Wrong output for single character files without newline

which was filed against the diffutils package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
18402: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18402
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: [bug-diffutils] bug#18402: Wrong output for single character files without newline
From: Paul Eggert <[email-masked]>
To: Eric Blake <[email-masked]>, [email-masked], Matt Johnson <[email-masked]>, [email-masked]
Date: Wed, 03 Sep 2014 16:05:55 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

Thanks for reporting that.  I installed the attached 3 patches; patch #2 
should fix the bug.
From 7bdd6479ce43d6b45803fd0bc4b363370975ceab Mon Sep 17 00:00:00 2001
From: Paul Eggert <[email-masked]>
Date: Wed, 3 Sep 2014 15:25:21 -0700
Subject: [PATCH 1/3] diff: fix performance bug with prefix computation

* src/io.c (find_identical_ends): Fix performance bug:
the test for when the prefix was needed messed up by
the 2002-02-28 integer-overflow fixes, causing performance to be
worse than it needed to be.
---
 src/io.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/src/io.c b/src/io.c
index 05a898c..1a8b936 100644
--- a/src/io.c
+++ b/src/io.c
@@ -538,6 +538,7 @@ find_identical_ends (struct file_data filevec[])
   lin i, lines;
   size_t n0, n1;
   lin alloc_lines0, alloc_lines1;
+  bool prefix_needed;
   lin buffered_prefix, prefix_count, prefix_mask;
   lin middle_guess, suffix_guess;
 
@@ -687,12 +688,13 @@ find_identical_ends (struct file_data filevec[])
   prefix_mask = prefix_count - 1;
   lines = 0;
   linbuf0 = xmalloc (alloc_lines0 * sizeof *linbuf0);
+  prefix_needed = ! (no_diff_means_no_output
+		     && filevec[0].prefix_end == p0
+		     && filevec[1].prefix_end == p1);
   p0 = buffer0;
 
   /* If the prefix is needed, find the prefix lines.  */
-  if (! (no_diff_means_no_output
-	 && filevec[0].prefix_end == p0
-	 && filevec[1].prefix_end == p1))
+  if (prefix_needed)
     {
       end0 = filevec[0].prefix_end;
       while (p0 != end0)
-- 
1.9.3

From d2fd9d4683ef60c259a3b426f71cef1b89ff383d Mon Sep 17 00:00:00 2001
From: Paul Eggert <[email-masked]>
Date: Wed, 3 Sep 2014 15:58:03 -0700
Subject: [PATCH 2/3] diff: fix bug with diff -B and incomplete lines

Reported by Navin Kabra via Eric Blake in:
http://bugs.gnu.org/18402
* src/util.c (analyze_hunk): Don't mishandle incomplete
lines at end of file.
* tests/no-newline-at-eof: Test for the bug.
---
 src/util.c              | 6 ++++--
 tests/no-newline-at-eof | 6 ++++++
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/src/util.c b/src/util.c
index 016057d..44ce61f 100644
--- a/src/util.c
+++ b/src/util.c
@@ -817,7 +817,8 @@ analyze_hunk (struct change *hunk,
       for (i = next->line0; i <= l0 && trivial; i++)
 	{
 	  char const *line = linbuf0[i];
-	  char const *newline = linbuf0[i + 1] - 1;
+	  char const *lastbyte = linbuf0[i + 1] - 1;
+	  char const *newline = lastbyte + (*lastbyte != '\n');
 	  size_t len = newline - line;
 	  char const *p = line;
 	  if (skip_white_space)
@@ -837,7 +838,8 @@ analyze_hunk (struct change *hunk,
       for (i = next->line1; i <= l1 && trivial; i++)
 	{
 	  char const *line = linbuf1[i];
-	  char const *newline = linbuf1[i + 1] - 1;
+	  char const *lastbyte = linbuf1[i + 1] - 1;
+	  char const *newline = lastbyte + (*lastbyte != '\n');
 	  size_t len = newline - line;
 	  char const *p = line;
 	  if (skip_white_space)
diff --git a/tests/no-newline-at-eof b/tests/no-newline-at-eof
index 14d5f49..f503718 100755
--- a/tests/no-newline-at-eof
+++ b/tests/no-newline-at-eof
@@ -50,4 +50,10 @@ compare exp2 out || fail=1
 # expect empty stderr
 compare /dev/null err || fail=1
 
+# Test for Bug#18402.
+printf a > a
+printf b > b
+diff -B a b > out 2>err
+test $? = 1 || fail=1
+
 Exit $fail
-- 
1.9.3

From df3af29627a92495a740da13cb8bb0d4fcc1bf84 Mon Sep 17 00:00:00 2001
From: Paul Eggert <[email-masked]>
Date: Wed, 3 Sep 2014 16:02:35 -0700
Subject: [PATCH 3/3] doc: mention diff -B fix in NEWS

---
 NEWS | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/NEWS b/NEWS
index 58a7cbb..9f388dd 100644
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,9 @@ GNU diffutils NEWS                                    -*- outline -*-
   consider two Asian file names to be the same merely because they
   contain no English characters.
 
+  diff -B no longer generates incorrect output if the two inputs
+  each end with a one-byte incomplete line.
+
 ** Performance changes
 
   diff's default algorithm has been adjusted to output higher-quality
-- 
1.9.3

Subject: Re: Wrong output for single character files without newline
From: Eric Blake <[email-masked]>
To: Navin Kabra <[email-masked]>, [email-masked], [email-masked]
Date: Wed, 03 Sep 2014 15:03:44 -0600
List-Id: <debbugs-submit.debbugs.gnu.org>

[adding bug-diffutils, as requested by diff --help]

On 09/03/2014 04:17 AM, Navin Kabra wrote:
> Consider this:
>     echo -n a > /tmp/a
>     echo -n b > /tmp/b
>     diff -B /tmp/a /tmp/b

'echo -n' is non-portable.  Please get used to using 'printf' instead.

> 
> Clearly, the two files are different, yet, diff seems to think that the
> files are identical. I've managed to reproduce this problem on Ubuntu
> 14.04 with diffutils 3.3, on CloudLinux 5.10 with diffutils 2.8.1, and
> also Ubuntu 10.04 with diffutils 2.8.1.
> 
> If I don't use the -B option, the problem goes away. If the files do end
> with a newline, the problem goes away. If the files contain more than 1
> character, the problem goes away. If combined with *some* of the other
> options (e.g. -e or -y) the problem goes away.

Actually, I couldn't reproduce -y making the problem go away:

$ ./src/diff -By <(printf a) <(printf b)
a								b
$ echo $?
0

Thanks for the extensive analysis; I can confirm that this bug is still
present in the latest diffutils.git sources, although I have not
personally hunted for the culprit line of code yet.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org