Subject: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS)

From: Paul Eggert <[email-masked]>
Date: Tue, 24 May 2016 23:51:33 -0700

Dmitry Gutov wrote:

> - Shouldn't that change be in vc-coding-system-for-diff?
> - It seems to try to fix a separate issue (whether all files use the same coding
> system).

Yes. For emacs-25 that's probably too much, as you suggest. So we can fix the 
problem in vc-coding-system-for-diff. Revised (more-conservative) patch attached.

> - Like Eli pointed out, (coding-system-get coding-system-for-read
> :ascii-compatible-p) should work about as well. Why doesn't it?

It doesn't work for EBCDIC.

> As an aside, how did you manage to create a patch that's using tabs for
> indentation, with indent-tabs-mode bound to nil in .dir-locals.el? That's
> troubling.

I override that setting, as I find it annoying in too many cases. It's just a 
minor annoyance, but there it is.
From 4b608d04b5c71a580a962b014c0399d0c917d9ab Mon Sep 17 00:00:00 2001
From: Paul Eggert <[email-masked]>
Date: Tue, 24 May 2016 22:28:44 -0700
Subject: [PATCH] Fix vc-diff problems with UTF-16

Problem with UTF-16 reported by Uwe Brauer (Bug#23595).
There are similar problems with EBCDIC or with other coding
systems differing greatly from ASCII.
* lisp/vc/vc.el (vc-coding-system-for-diff): Require the file's coding
system to be compatible-enough with ASCII so that messages like "Binary
files differ" are not misdecoded.
---
 lisp/vc/vc.el | 25 +++++++++++++++++++------
 1 file changed, 19 insertions(+), 6 deletions(-)

diff --git a/lisp/vc/vc.el b/lisp/vc/vc.el
index af875e8..0dbdcb0 100644
--- a/lisp/vc/vc.el
+++ b/lisp/vc/vc.el
@@ -1601,18 +1601,31 @@ vc-coding-system-inherit-eol
 (defun vc-coding-system-for-diff (file)
   "Return the coding system for reading diff output for FILE."
   (or coding-system-for-read
-      ;; if we already have this file open,
-      ;; use the buffer's coding system
-      (let ((buf (find-buffer-visiting file)))
-        (when buf (with-current-buffer buf
+      (let ((coding
+	     (or
+	      ;; If we already have this file open,
+	      ;; try the buffer's coding system.
+	      (let ((buf (find-buffer-visiting file)))
+		(when buf
+		  (with-current-buffer buf
 		    (if vc-coding-system-inherit-eol
 			buffer-file-coding-system
 		      ;; Don't inherit the EOL part of the coding-system,
 		      ;; because some Diff tools may choose to use
 		      ;; a different one.  bug#4451.
 		      (coding-system-base buffer-file-coding-system)))))
-      ;; otherwise, try to find one based on the file name
-      (car (find-operation-coding-system 'insert-file-contents file))
+	      ;; Otherwise, try to find one based on the file name.
+	      (car (find-operation-coding-system 'insert-file-contents
+						 file)))))
+	;; Use the files' coding system only if it is compatible
+	;; enough with ASCII.  If the files' coding system is UTF-16,
+	;; diff likely outputs something like "Binary files differ" in
+	;; ASCII, which would be misdecoded by UTF-16.
+	(when (and coding
+		   (let ((samp "Binary files differ"))
+		     (string-equal samp (decode-coding-string
+					 samp coding t))))
+	  last-coding-system-used))
       ;; and a final fallback
       'undecided))
 
-- 
2.5.5