GNU bug report logs - #18236
diff-apply-hunk interacts poorly with line endings

Previous Next

To reply to this bug, email your comments to 18236 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: bug-emacs <bug-emacs <at> gnu.org>
Subject: diff-apply-hunk interacts poorly with line endings
Date: Sun, 10 Aug 2014 16:34:23 +0100

[Message part 1 (text/plain, inline)]

I'm using Emacs 24.3. Consider the following two files:

foo:
----cut here----
This is a small test file.
It has DOS line endings.
----cut here----

bar:
----cut here----
This is a small test file.
It too has DOS line endings.
----cut here----

Save them with DOS line endings, and

diff foo bar > foo.patch
cp foo bar

Visit foo.patch. When I do that, each line ends in ^M.

Now, in the patch buffer, M-x diff-apply-hunk. Note that the ^Ms have been
added to the lines. Writing the buffer and examination with "hd" reveals
that each line now ends \r\r\n.

It seems what's happened here is that diff-mode doesn't try to diagnose the
character encoding of the patch file.

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Message #8 received at 18236 <at> debbugs.gnu.org (full text, mbox):

From: Marcin Borkowski <mbork <at> mbork.pl>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: 18236 <at> debbugs.gnu.org
Subject: Re: bug#18236: diff-apply-hunk interacts poorly with line endings
Date: Wed, 17 Feb 2016 16:35:04 +0100

On 2014-08-10, at 17:34, Reuben Thomas <rrt <at> sc3d.org> wrote:

> I'm using Emacs 24.3. Consider the following two files:
>
> foo:
> ----cut here----
> This is a small test file.
> It has DOS line endings.
> ----cut here----
>
> bar:
> ----cut here----
> This is a small test file. 
> It too has DOS line endings. 
> ----cut here----
>
> Save them with DOS line endings, and
>
> diff foo bar > foo.patch
> cp foo bar
>
> Visit foo.patch. When I do that, each line ends in ^M.
>
> Now, in the patch buffer, M-x diff-apply-hunk. Note that the ^Ms have been added to the lines. Writing the buffer and examination with "hd" reveals that each line now ends
> \r\r\n.
>
> It seems what's happened here is that diff-mode doesn't try to diagnose the character encoding of the patch file.

Hi all,

confirmed on GNU Emacs 25.1.50.2 (commit 4ccd268).

Best,

--
Marcin Borkowski

Message #11 received at 18236 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Marcin Borkowski <mbork <at> mbork.pl>
Cc: 18236 <at> debbugs.gnu.org, rrt <at> sc3d.org
Subject: Re: bug#18236: diff-apply-hunk interacts poorly with line endings
Date: Wed, 17 Feb 2016 18:39:38 +0200

> From: Marcin Borkowski <mbork <at> mbork.pl>
> Date: Wed, 17 Feb 2016 16:35:04 +0100
> Cc: 18236 <at> debbugs.gnu.org
> 
> On 2014-08-10, at 17:34, Reuben Thomas <rrt <at> sc3d.org> wrote:
> 
> > I'm using Emacs 24.3. Consider the following two files:
> >
> > foo:
> > ----cut here----
> > This is a small test file.
> > It has DOS line endings.
> > ----cut here----
> >
> > bar:
> > ----cut here----
> > This is a small test file. 
> > It too has DOS line endings. 
> > ----cut here----
> >
> > Save them with DOS line endings, and
> >
> > diff foo bar > foo.patch
> > cp foo bar
> >
> > Visit foo.patch. When I do that, each line ends in ^M.
> >
> > Now, in the patch buffer, M-x diff-apply-hunk. Note that the ^Ms have been added to the lines. Writing the buffer and examination with "hd" reveals that each line now ends
> > \r\r\n.
> >
> > It seems what's happened here is that diff-mode doesn't try to diagnose the character encoding of the patch file.
> 
> Hi all,
> 
> confirmed on GNU Emacs 25.1.50.2 (commit 4ccd268).

When I try doing that, I get an error message.  Is the recipe
complete?  Also, does this happen on a Posix host or on a Windows box?
If the former, I won't expect each line in the patch file to end with
a ^M, only the lines that came from the files being diffed.

I'm confused.

Message #14 received at 18236 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Marcin Borkowski <mbork <at> mbork.pl>, 18236 <at> debbugs.gnu.org
Subject: Re: bug#18236: diff-apply-hunk interacts poorly with line endings
Date: Wed, 17 Feb 2016 19:59:02 +0000

[Message part 1 (text/plain, inline)]

On 17 February 2016 at 16:39, Eli Zaretskii <eliz <at> gnu.org> wrote:

> > From: Marcin Borkowski <mbork <at> mbork.pl>
> > Date: Wed, 17 Feb 2016 16:35:04 +0100
> > Cc: 18236 <at> debbugs.gnu.org
> >
> > On 2014-08-10, at 17:34, Reuben Thomas <rrt <at> sc3d.org> wrote:
> >
> > > I'm using Emacs 24.3. Consider the following two files:
> > >
> > > foo:
> > > ----cut here----
> > > This is a small test file.
> > > It has DOS line endings.
> > > ----cut here----
> > >
> > > bar:
> > > ----cut here----
> > > This is a small test file.
> > > It too has DOS line endings.
> > > ----cut here----
> > >
> > > Save them with DOS line endings, and
> > >
> > > diff foo bar > foo.patch
> > > cp foo bar
> > >
> > > Visit foo.patch. When I do that, each line ends in ^M.
> > >
> > > Now, in the patch buffer, M-x diff-apply-hunk. Note that the ^Ms have
> been added to the lines. Writing the buffer and examination with "hd"
> reveals that each line now ends
> > > \r\r\n.
> > >
> > > It seems what's happened here is that diff-mode doesn't try to
> diagnose the character encoding of the patch file.
> >
> > Hi all,
> >
> > confirmed on GNU Emacs 25.1.50.2 (commit 4ccd268).
>
> When I try doing that, I get an error message.


​I just repeated the recipe, now in Emacs 24.4. It worked as described.​


> Is the recipe
> ​ ​
> complete?


​Seems so.​


> Also, does this happen on a Posix host or on a Windows box?
>

​On a GNU/Linux system.​


> If the former, I won't expect each line in the patch file to end with
> a ^M, only the lines that came from the files being diffed.
>

Sorry, I was imprecise. ​You're quite right, only the lines that come from
the files being diffed end in ^M.​

However, the original problem ​remains, as stated: after applying the patch
hunk, the patched lines of the resultant file "bar" end \r\r\n.

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Message #17 received at 18236 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: mbork <at> mbork.pl, 18236 <at> debbugs.gnu.org
Subject: Re: bug#18236: diff-apply-hunk interacts poorly with line endings
Date: Wed, 17 Feb 2016 22:14:04 +0200

> Date: Wed, 17 Feb 2016 19:59:02 +0000
> From: Reuben Thomas <rrt <at> sc3d.org>
> Cc: Marcin Borkowski <mbork <at> mbork.pl>, 18236 <at> debbugs.gnu.org
> 
>  Is the recipe
>  ​ ​
>  complete?
> 
> ​Seems so.​
> 
> 
>  Also, does this happen on a Posix host or on a Windows box?
> 
> 
> ​On a GNU/Linux system.​
> 
> 
>  If the former, I won't expect each line in the patch file to end with
>  a ^M, only the lines that came from the files being diffed.
> 
> 
> Sorry, I was imprecise. ​You're quite right, only the lines that come from the files being diffed end in ^M.​
> 
> However, the original problem ​remains, as stated: after applying the patch hunk, the patched lines of the
> resultant file "bar" end \r\r\n.

Doesn't happen on Windows.  I will try on GNU/Linux and see what I
find there.

Message #20 received at 18236 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: mbork <at> mbork.pl, 18236 <at> debbugs.gnu.org
Subject: Re: bug#18236: diff-apply-hunk interacts poorly with line endings
Date: Sat, 20 Feb 2016 14:16:06 +0200

> Date: Wed, 17 Feb 2016 22:14:04 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: mbork <at> mbork.pl, 18236 <at> debbugs.gnu.org
> 
> > Date: Wed, 17 Feb 2016 19:59:02 +0000
> > From: Reuben Thomas <rrt <at> sc3d.org>
> > Cc: Marcin Borkowski <mbork <at> mbork.pl>, 18236 <at> debbugs.gnu.org
> > 
> >  Is the recipe
> >  ​ ​
> >  complete?
> > 
> > ​Seems so.​
> > 
> > 
> >  Also, does this happen on a Posix host or on a Windows box?
> > 
> > 
> > ​On a GNU/Linux system.​
> > 
> > 
> >  If the former, I won't expect each line in the patch file to end with
> >  a ^M, only the lines that came from the files being diffed.
> > 
> > 
> > Sorry, I was imprecise. ​You're quite right, only the lines that come from the files being diffed end in ^M.​
> > 
> > However, the original problem ​remains, as stated: after applying the patch hunk, the patched lines of the
> > resultant file "bar" end \r\r\n.
> 
> Doesn't happen on Windows.  I will try on GNU/Linux and see what I
> find there.

I see the problem, but I don't see how this could be solved, in
general.

This problem will always happen when the patch file and the file to be
patched are visited in Emacs using different values of
buffer-file-coding-system.  The problem is not limited to the EOL
format, it can also happen with the text encoding, e.g. if the patch
file is visited using raw-text and the file to be patched uses Latin-1
or some such.

In the case in point, the patch file is visited using the -unix EOL,
whereas the file to be patched is visited using -dos.  If you force
Emacs to visit the patch file using -dos, e.g. by using
"C-x RET c dos RET C-x C-f", then the problem goes away.

I don't see how we can solve this.  We cannot assume that the ^M
characters in the patch file's buffer should be omitted, because they
could really belong to the hunk.  The Patch utility does TRT in this
case because it effectively reads the files as bytes.  The equivalent
in Emacs is to visit the files with no-conversion, but if we do that,
the patched file will be displayed to the user without decoding, which
is not good.

The most we can do is display a warning and ask for confirmation when
the values of buffer-file-coding-system differ between the patch
file's buffer and the buffer of file to be patched.  Will that be
sufficient?

Message #23 received at 18236 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Marcin Borkowski <mbork <at> mbork.pl>, 18236 <at> debbugs.gnu.org
Subject: Re: bug#18236: diff-apply-hunk interacts poorly with line endings
Date: Sat, 20 Feb 2016 12:32:03 +0000

[Message part 1 (text/plain, inline)]

On 20 February 2016 at 12:16, Eli Zaretskii <eliz <at> gnu.org> wrote:

>
> The most we can do is display a warning and ask for confirmation when
> the values of buffer-file-coding-system differ between the patch
> file's buffer and the buffer of file to be patched.  Will that be
> sufficient?
>

Thanks for your analysis. ​This solution sounds fine to me: ​as you say, in
general it's impossible to tell exactly what went wrong, but we can warn
the user so that the unfortunate result I described is not obtained
silently.

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Message #26 received at 18236 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: mbork <at> mbork.pl, 18236 <at> debbugs.gnu.org
Subject: Re: bug#18236: diff-apply-hunk interacts poorly with line endings
Date: Sat, 20 Feb 2016 14:50:47 +0200

> Date: Sat, 20 Feb 2016 12:32:03 +0000
> From: Reuben Thomas <rrt <at> sc3d.org>
> Cc: Marcin Borkowski <mbork <at> mbork.pl>, 18236 <at> debbugs.gnu.org
> 
>  The most we can do is display a warning and ask for confirmation when
>  the values of buffer-file-coding-system differ between the patch
>  file's buffer and the buffer of file to be patched. Will that be
>  sufficient?
> 
> Thanks for your analysis. ​This solution sounds fine to me: ​as you say, in general it's impossible to tell exactly
> what went wrong, but we can warn the user so that the unfortunate result I described is not obtained silently.

Marcin, would you like to work on a change along these lines?

The function to use for comparing 2 coding-systems is
`coding-system-equal'.  The idea is to see if the values of
`buffer-file-coding-system' in the patch-file buffer (where the
command is invoked) and in the buffer that visits the file to be
patched (after `find-file-noselect' call in
`diff-find-source-location' returns) are different, and if so, ask the
user for confirmation.  Ideally, the confirmation should only be
requested once, when applying the 1st hunk from the patch file.

Thanks.

Message #29 received at 18236 <at> debbugs.gnu.org (full text, mbox):

From: Marcin Borkowski <mbork <at> mbork.pl>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 18236 <at> debbugs.gnu.org, Reuben Thomas <rrt <at> sc3d.org>
Subject: Re: bug#18236: diff-apply-hunk interacts poorly with line endings
Date: Sun, 21 Feb 2016 11:09:41 +0100

On 2016-02-20, at 13:50, Eli Zaretskii <eliz <at> gnu.org> wrote:

>> Date: Sat, 20 Feb 2016 12:32:03 +0000
>> From: Reuben Thomas <rrt <at> sc3d.org>
>> Cc: Marcin Borkowski <mbork <at> mbork.pl>, 18236 <at> debbugs.gnu.org
>> 
>>  The most we can do is display a warning and ask for confirmation when
>>  the values of buffer-file-coding-system differ between the patch
>>  file's buffer and the buffer of file to be patched. Will that be
>>  sufficient?
>> 
>> Thanks for your analysis. ​This solution sounds fine to me: ​as you say, in general it's impossible to tell exactly
>> what went wrong, but we can warn the user so that the unfortunate result I described is not obtained silently.
>
> Marcin, would you like to work on a change along these lines?

OK, I'll take a look and report in a few days.

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Message #32 received at 18236 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Marcin Borkowski <mbork <at> mbork.pl>
Cc: 18236 <at> debbugs.gnu.org, rrt <at> sc3d.org
Subject: Re: bug#18236: diff-apply-hunk interacts poorly with line endings
Date: Sun, 21 Feb 2016 17:56:00 +0200

> From: Marcin Borkowski <mbork <at> mbork.pl>
> Cc: Reuben Thomas <rrt <at> sc3d.org>, 18236 <at> debbugs.gnu.org
> Date: Sun, 21 Feb 2016 11:09:41 +0100
> 
> > Marcin, would you like to work on a change along these lines?
> 
> OK, I'll take a look and report in a few days.

Thanks.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.