Subject: bug#34350: 27.0.50; ediff-revision broken with SVN backend + non ascii chars both in directory and in filename

From: Stefan Monnier <[email-masked]>
Date: Fri, 08 Feb 2019 18:03:49 -0500

>> >> I think intuitively, in terms of encoding for the file's contents the
>> >> backends should always return a byte-sequence (i.e. with no-conversion)
>> >> and the front-end should then decide how to decode it (e.g., obeying
>> >> the -*- coding -*- cookie and such).
>> > Why do we need to leave this to the front end?
>> 
>> So that it's not half-implemented differently in every backend.
>
> But every back-end has its own peculiar ways to do the job.  E.g., Git
> needs to call "git ls-files" first, and we then need to read the file
> names before calling "git cat".  Other VCSes don't necessarily need
> that.  So different implementations cannot be worked around and no
> higher layers can even know what the lower layers do to set up
> encoding correctly for all of them, at least not in most cases.

Then the front/middle end shouldn't set anything up for the backend, but
the backend still needs to return just undecoded bytes.

>> > It's a waste of cycles to do decoding manually in Lisp,
>> 
>> It'd be better to decode "on the fly" rather than first insert the byte
>> stream in a buffer and then decode it.  No doubt.
>> But I can't see how to do that and handle -*-coding-*-,
>> auto-coding-regexp-alist, and friends.
> What do you mean by "how"?  Just do the normal I/O, and all of those
> will be taken care of.  Like when we visit a file.  What am I missing?

The `find-revision` backend operation must put the revision's bytes into
the provided buffer, indeed similarly to when we visit a file.
But the difference is that in 99% of the cases, the bytes don't come
from a file but from a process's stdout, so the backed can't directly
use the "visit a file" trick.

> So you are saying that no other VC function cal call the backend's
> find-revision method?

That's the idea, yes.

> But we already have 2 more functions that do
> it, and I see no problems with doing that.

Then we need an additional middle layer.

>> All other code should be layered on top of that one, so it's done at
>> only one place.  If that doesn't work (because vc-find-revision is
>> not flexible enough) then we should move this decoding code to a
>> middle-layer between vc-find-revision and (vc-call find-revision
>> ...) that all users of `find-revision` can then use.
>
> I don't think this is possible in general, because different callers
> have different needs wrt to encoding/decoding.

Hmm... looking at the code I see 3 places where we call the
`find-revision` backend:

- vc-find-revision-save
- vc-find-revision-no-save
- vc-default-revert

The last one should indeed call the backend directly (as it currently
does) and should be changed not to bind coding-system-for-read/write and
instead to assume that the backend deals with bytes.

The other two are begging to be unified to reduce code redundancy and
are the ones that need the do the file-like decoding (and they indeed do
it).

> You forget VCS operations that return stuff other than the complete
> file's contents, like vc-log or vc-dff or calls that return file names
> etc.

Not really forgetting, no.  Instead I was talking specifically about
things like `find-revision` (i.e. about the content of files).
For filenames, commit messages and other metadata the situation is quite
different, indeed.

        Stefan