Subject: Re: bug#23595: 25.1.50; file with chinese/japanse chars, vc-diff fails (HG, Git, RCS)

From: Dmitry Gutov <[email-masked]>
Date: Thu, 26 May 2016 02:21:40 +0300

On 05/25/2016 07:22 PM, Eli Zaretskii wrote:

> I think we can make an educated guess, see below.  My conclusion is
> that this does need to be VCS-specific.
>
>> So far, we've seem some pretty similar results with vc-diff using Git, Hg and RCS.
>
> Not quite.  They all fail, but in different ways.  More importantly,
> the solutions are most probably going to be different.

OK, that makes sense.

> Backend-specific, most probably.  Except that currently we only have a
> good idea about the Git backend, for which it is explicitly documented
> that the output will be in UTF-8 when content filters are used.
>
> ...

Taking a step back, I'm not sure it's really necessary to fix this. 
Because Uwe, the only person to complain about diffing UTF-16 files so 
far, can solve it much easier by switching the file to UTF-8.

And properly fixing it, across the board, is either impossible, or at 
least pretty hard.

>> We can always extract a new function when it's needed, though.
>
> True, but I think if we want to support UTF-16 files, the need is
> already here.  vc-diff and its derivatives are just the tip of the
> iceberg, we will need similar stuff for every command that includes
> both text from the versioned file(s) and some text output by the VCS
> program itself.

Here's a patch everybody is welcome to try.
diff --git a/lisp/vc/vc-git.el b/lisp/vc/vc-git.el
index f35c84d..f7bd867 100644
--- a/lisp/vc/vc-git.el
+++ b/lisp/vc/vc-git.el
@@ -1083,12 +1083,19 @@ vc-git-region-history-mode
                     (cdr font-lock-defaults))))

+(defun git--asciify-coding-system ()
+  (unless (let ((samp "Binary files differ"))
+            (string-equal samp (decode-coding-string
+                                samp coding-system-for-read t)))
+    (setq coding-system-for-read 'undecided)))
+
 (autoload 'vc-switches "vc")
 
 (defun vc-git-diff (files &optional rev1 rev2 buffer _async)
   "Get a difference report using Git between two revisions of FILES."
   (let (process-file-side-effects
         (command "diff-tree"))
+    (git--asciify-coding-system)
     (if rev2
         ;; Diffing against the empty tree.
         (unless rev1 (setq rev1 "4b825dc642cb6eb9a060e54bf8d69288fbee4904"))
@@ -1127,6 +1134,7 @@ vc-git-revision-completion-table
     table))
 
 (defun vc-git-annotate-command (file buf &optional rev)
+  (git--asciify-coding-system)
   (let ((name (file-relative-name file)))
     (apply #'vc-git-command buf 'async nil "blame" "--date=short"
 	   (append (vc-switches 'git 'annotate)