GNU bug report logs - #1502
CR/LF Unicode Problem

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 1502 in the body.
You can then email your comments to 1502 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>
To: <bug-gnu-emacs <at> gnu.org>
Subject: CR/LF Unicode Problem
Date: Fri, 5 Dec 2008 16:33:53 -0600

[Message part 1 (text/plain, inline)]

Hi,

You guys do a great job. Thanks for all the hard work.

I was blaming the .NET framework on this, but it appears to be an issue
with Emacs.

When I load a Unicode file (UTF-8) where some lines in CR/LF and some
lines end in LF, in hexl-mode the CR/LF EOL's appear as CR/CR/LF and the
LF EOL's appear as CR/LF. See this thread for more information: 

http://social.msdn.microsoft.com/Forums/en-US/netfxbcl/thread/8ef5b69d-1
35d-4584-ae1a-1caeb4afc846/#page:1

If I save the file in hexl mode, it will save it with the extra CRs,
thus modifying the file more than anticipated.

Thanks,

Henry Lafleur               |      ,__o 
Project Lead                |    _-\_<, 
Canrig Enterprise Solutions |   (*)/'(*) 
a division of Canrig Drilling Technology Ltd.
Fax:       281-774-5640
Support: 1-866-433-4345
mailto:Henry.Lafleur <at> canrig.com 
http://www.mywells.com/ 

In GNU Emacs 22.1.1 (i386-mingw-nt5.1.2600)
 of 2007-06-02 on RELEASE
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4) --cflags
-Ic:/gnuwin32/include'

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ENU
  locale-coding-system: cp1252
  default-enable-multibyte-characters: t

Major mode: Hexl

Minor modes in effect:
  ruler-mode: t
  hl-line-mode: t
  encoded-kbd-mode: t
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  unify-8859-on-encoding-mode: t
  utf-translate-cjk-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: identity

Recent input:
S e e SPC t h i s SPC t h r e a d SPC f o r SPC m o 
r e SPC i n f o r m a t i o n : SPC <return> C-v M-v 
<up> <up> <up> <down> <return> C-y <help-echo> <down-mouse-1> 
<mouse-movement> <mouse-movement> <drag-mouse-1> <down-mouse-1> 
<mouse-movement> <mouse-movement> <drag-mouse-1> <down-mouse-1> 
<mouse-1> <down-mouse-1> <mouse-1> <wheel-down> <return> 
<return> <up> <up> <up> <up> <down> <down> Y o u SPC 
g u y s SPC d o SPC a SPC g r e a t SPC j o b SPC w 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<delete> <delete> <down-mouse-3> <mouse-3> <wheel-down> 
<double-wheel-down> <down> <down> <down> <up> <up> 
<up> <down> <down-mouse-1> <mouse-movement> <mouse-movement> 
<help-echo> <mouse-movement> <mouse-movement> <drag-mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <down-mouse-1> 
<mouse-1> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <menu-bar> <file> <open-file> 
<help-echo> <help-echo> M-x h e x l - m o d <tab> <return> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<menu-bar> <help-menu> <report-emacs-bug>

Recent messages:
Loading emacsbug...done
Loading help-mode...done
Auto-saving...done
Mark set
Auto-saving...done
Loading sql...
Loading easymenu...done
Loading sql...done
Loading hexl...done
Loading mule-util...done




-----------------------------------------
CANRIG EMAIL NOTICE - This transmission may be strictly
confidential. If you are not the intended recipient of this
message, you may not disclose, print, copy, or disseminate this
information. If you have received this in error, please reply and
notify the sender (only) and delete the message. Unauthorized
interception of this e-mail is a violation of federal criminal law.
This communication does not reflect an intention by the sender or
the sender's principal to conduct a transaction or make any
agreement by electronic means. Nothing contained in this message or
in any attachment shall satisfy the requirements for a writing, and
nothing contained herein shall constitute a contract or electronic
signature under the Electronic Signatures in Global and National
Commerce Act, any version of the Uniform Electronic Transactions
Act, or any other statute governing electronic transactions.

[Message part 2 (text/html, inline)]

Message #10 received at 1502 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>
To: <1502 <at> debbugs.gnu.org>
Subject: RE: bug#1502: Acknowledgement (CR/LF Unicode Problem)
Date: Fri, 5 Dec 2008 17:02:49 -0600

This problem also appears to happen regardless of if I go into hexl-mode
or not. All I do is open the file (a .SQL file), add a space, and then
save it. These files have the 3 byte order mark at the beginning of the
file that Microsoft puts that Emacs doesn't recognize. Don't know if
that makes a difference.

I can forward the file with the problem if you need it.

I reproduced the problem on NT Emacs 22.3 also.

Thanks,

Henry.

-----Original Message-----
From: Emacs bug Tracking System [mailto:don <at> donarmstrong.com] 
Sent: Friday, December 05, 2008 4:40 PM
To: Lafleur, Henry
Subject: bug#1502: Acknowledgement (CR/LF Unicode Problem)


Thank you for filing a new bug report with Emacs.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 Emacs Bugs <bug-gnu-emacs <at> gnu.org>

If you wish to submit further information on this problem, please send
it to 1502 <at> emacsbugs.donarmstrong.com, as before.

Please do not send mail to don <at> donarmstrong.com unless you wish to
report a problem with the Bug-tracking system.


--
1502: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=1502
Emacs Bug Tracking System
Contact don <at> donarmstrong.com with problems

-----------------------------------------
CANRIG EMAIL NOTICE - This transmission may be strictly
confidential. If you are not the intended recipient of this
message, you may not disclose, print, copy, or disseminate this
information. If you have received this in error, please reply and
notify the sender (only) and delete the message. Unauthorized
interception of this e-mail is a violation of federal criminal law.
This communication does not reflect an intention by the sender or
the sender's principal to conduct a transaction or make any
agreement by electronic means. Nothing contained in this message or
in any attachment shall satisfy the requirements for a writing, and
nothing contained herein shall constitute a contract or electronic
signature under the Electronic Signatures in Global and National
Commerce Act, any version of the Uniform Electronic Transactions
Act, or any other statute governing electronic transactions.

Message #15 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>,
        1502 <at> debbugs.gnu.org
Cc: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1502: CR/LF Unicode Problem
Date: Sat, 06 Dec 2008 10:02:45 +0200

> Date: Fri, 5 Dec 2008 16:33:53 -0600
> From: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>
> Cc: 
> 
> When I load a Unicode file (UTF-8) where some lines in CR/LF and some
> lines end in LF, in hexl-mode the CR/LF EOL's appear as CR/CR/LF and the
> LF EOL's appear as CR/LF. See this thread for more information: 
> 
> http://social.msdn.microsoft.com/Forums/en-US/netfxbcl/thread/8ef5b69d-135d-4584-ae1a-1caeb4afc846/#page:1
> 
> If I save the file in hexl mode, it will save it with the extra CRs,
> thus modifying the file more than anticipated.

Can you post a shortest example of a file that exhibits this behavior?
I didn't see it in the thread you were citing; sorry if I missed
something.

Message #20 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>,
        1502 <at> debbugs.gnu.org
Cc: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1502: Acknowledgement (CR/LF Unicode Problem)
Date: Sat, 06 Dec 2008 10:04:09 +0200

> Date: Fri, 5 Dec 2008 17:02:49 -0600
> From: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>
> Cc: 
> 
> This problem also appears to happen regardless of if I go into hexl-mode
> or not. All I do is open the file (a .SQL file), add a space, and then
> save it. These files have the 3 byte order mark at the beginning of the
> file that Microsoft puts that Emacs doesn't recognize. Don't know if
> that makes a difference.

Does it happen with files that are not .SQL ?

Again, a short example file would help.

Message #35 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>
To: "Eli Zaretskii" <eliz <at> gnu.org>, <1502 <at> debbugs.gnu.org>
Cc: <bug-gnu-emacs <at> gnu.org>
Subject: RE: bug#1502: Acknowledgement (CR/LF Unicode Problem)
Date: Sat, 6 Dec 2008 11:06:12 -0600

[Message part 1 (text/plain, inline)]

Eli,

I'll be back in my office on Monday. I may be able to get the file before then, but probably not. I e-mailed it to myself at home, but my company blocks .SQL files.

I don't know if this is only with .SQL files or not. Haven't tried it yet. I was trying to figure out what was wrong with my program not thinking that Emacs was the problem. I've used it since 1992 and never had a problem (well, except crashing on the Mac, but that was expected back then).

Also, I run Linux at home and I haven't tried the issue on Linux either.

Thanks,

Henry Lafleur               |    ,__o 
Project Lead                |  _-\_<, 
Canrig Enterprise Solutions | (*)/'(*) 
Fax:    281-774-5640
mailto:Henry.Lafleur <at> canrig.com 
http://www.canrig.com/ 



-----Original Message-----
From: Eli Zaretskii [mailto:eliz <at> gnu.org]
Sent: Sat 12/6/2008 2:04 AM
To: Lafleur, Henry; 1502 <at> emacsbugs.donarmstrong.com
Cc: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1502: Acknowledgement (CR/LF Unicode Problem)
 
> Date: Fri, 5 Dec 2008 17:02:49 -0600
> From: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>
> Cc: 
> 
> This problem also appears to happen regardless of if I go into hexl-mode
> or not. All I do is open the file (a .SQL file), add a space, and then
> save it. These files have the 3 byte order mark at the beginning of the
> file that Microsoft puts that Emacs doesn't recognize. Don't know if
> that makes a difference.

Does it happen with files that are not .SQL ?

Again, a short example file would help.




-----------------------------------------
CANRIG EMAIL NOTICE - This transmission may be strictly
confidential. If you are not the intended recipient of this
message, you may not disclose, print, copy, or disseminate this
information. If you have received this in error, please reply and
notify the sender (only) and delete the message. Unauthorized
interception of this e-mail is a violation of federal criminal law.
This communication does not reflect an intention by the sender or
the sender's principal to conduct a transaction or make any
agreement by electronic means. Nothing contained in this message or
in any attachment shall satisfy the requirements for a writing, and
nothing contained herein shall constitute a contract or electronic
signature under the Electronic Signatures in Global and National
Commerce Act, any version of the Uniform Electronic Transactions
Act, or any other statute governing electronic transactions.

[Message part 2 (text/html, inline)]

Message #45 received at 1502 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Henry Lafleur" <lafleurh <at> gmail.com>
To: eliz <at> gnu.org, 1502 <at> debbugs.gnu.org
Subject: Re: FW: bug#1502: CR/LF Unicode Problem
Date: Sat, 6 Dec 2008 11:27:13 -0600

[Message part 1 (text/plain, inline)]

Eli,

This is a file that is something like what I was describing. I'll get you
the error-causing file on Monday. I created this with Emacs/Linux, so I'm
not sure if it is correct--but on Linux it does not seem to exhibit the
problem (i.e. the CR's appended before all the LF's with Unicode files).

Thanks,

Henry.

On Sat, Dec 6, 2008 at 11:16 AM, Lafleur, Henry <Henry.LaFleur <at> canrig.com>wrote:

>
>
> Henry Lafleur               |    ,__o
> Project Lead                |  _-\_<,
> Canrig Enterprise Solutions | (*)/'(*)
> Fax:    281-774-5640
> mailto:Henry.Lafleur <at> canrig.com <Henry.Lafleur <at> canrig.com>
> http://www.canrig.com/
>
>
>
> -----Original Message-----
> From: Eli Zaretskii [mailto:eliz <at> gnu.org <eliz <at> gnu.org>]
> Sent: Sat 12/6/2008 2:02 AM
> To: Lafleur, Henry; 1502 <at> emacsbugs.donarmstrong.com
> Cc: bug-gnu-emacs <at> gnu.org
> Subject: Re: bug#1502: CR/LF Unicode Problem
>
> > Date: Fri, 5 Dec 2008 16:33:53 -0600
> > From: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>
> > Cc:
> >
> > When I load a Unicode file (UTF-8) where some lines in CR/LF and some
> > lines end in LF, in hexl-mode the CR/LF EOL's appear as CR/CR/LF and the
> > LF EOL's appear as CR/LF. See this thread for more information:
> >
> >
> http://social.msdn.microsoft.com/Forums/en-US/netfxbcl/thread/8ef5b69d-135d-4584-ae1a-1caeb4afc846/#page:1
> >
> > If I save the file in hexl mode, it will save it with the extra CRs,
> > thus modifying the file more than anticipated.
>
> Can you post a shortest example of a file that exhibits this behavior?
> I didn't see it in the thread you were citing; sorry if I missed
> something.
>
>
>   ------------------------------
>
> * CANRIG EMAIL NOTICE - This transmission may be strictly confidential. If
> you are not the intended recipient of this message, you may not disclose,
> print, copy, or disseminate this information. If you have received this in
> error, please reply and notify the sender (only) and delete the message.
> Unauthorized interception of this e-mail is a violation of federal criminal
> law. This communication does not reflect an intention by the sender or the
> sender's principal to conduct a transaction or make any agreement by
> electronic means. Nothing contained in this message or in any attachment
> shall satisfy the requirements for a writing, and nothing contained herein
> shall constitute a contract or electronic signature under the Electronic
> Signatures in Global and National Commerce Act, any version of the Uniform
> Electronic Transactions Act, or any other statute governing electronic
> transactions. *
>

[Message part 2 (text/html, inline)]

[sample.sql (text/x-sql, attachment)]

Message #50 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "Lafleur\, Henry" <Henry.LaFleur <at> canrig.com>
Cc: 1502 <at> debbugs.gnu.org, "Eli Zaretskii" <eliz <at> gnu.org>,
        bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1502: Acknowledgement (CR/LF Unicode Problem)
Date: Sat, 06 Dec 2008 17:47:38 -0500

> I'll be back in my office on Monday. I may be able to get the file before then, but probably not. I e-mailed it to myself at home, but my company blocks .SQL files.


I don't know if it's related, but all your messages have spurious extra
^M at the end of each line.  Do you happen to know where that's coming from?


        Stefan

Message #60 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>
To: <monnier <at> iro.umontreal.ca>
Cc: <1502 <at> debbugs.gnu.org>, <eliz <at> gnu.org>, <bug-gnu-emacs <at> gnu.org>
Subject: Re: bug#1502: Acknowledgement (CR/LF Unicode Problem)
Date: Sat, 6 Dec 2008 18:46:37 -0600

[Message part 1 (text/plain, inline)]

Yes,

I did it on purpose. We're in a mixed Windows/Linux environment and some users paste Linux-created files ending in LF only into a Windows app which writes a header & footer ending in CR/LF. When a file is mixed, Emacs is adding an additional CR in front of each of the LF characters, but only for a UTF-8 file with character codes>127. Thus, the file has lines ending with CRCRLF and CRLF.

Thanks,

Henry Lafleur

----- Original Message -----
From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Lafleur, Henry
Cc: 1502 <at> emacsbugs.donarmstrong.com <1502 <at> emacsbugs.donarmstrong.com>; Eli Zaretskii <eliz <at> gnu.org>; bug-gnu-emacs <at> gnu.org <bug-gnu-emacs <at> gnu.org>
Sent: Sat Dec 06 16:47:38 2008
Subject: Re: bug#1502: Acknowledgement (CR/LF Unicode Problem)

> I'll be back in my office on Monday. I may be able to get the file before then, but probably not. I e-mailed it to myself at home, but my company blocks .SQL files.


I don't know if it's related, but all your messages have spurious extra
^M at the end of each line.  Do you happen to know where that's coming from?


        Stefan



-----------------------------------------
CANRIG EMAIL NOTICE - This transmission may be strictly
confidential. If you are not the intended recipient of this
message, you may not disclose, print, copy, or disseminate this
information. If you have received this in error, please reply and
notify the sender (only) and delete the message. Unauthorized
interception of this e-mail is a violation of federal criminal law.
This communication does not reflect an intention by the sender or
the sender's principal to conduct a transaction or make any
agreement by electronic means. Nothing contained in this message or
in any attachment shall satisfy the requirements for a writing, and
nothing contained herein shall constitute a contract or electronic
signature under the Electronic Signatures in Global and National
Commerce Act, any version of the Uniform Electronic Transactions
Act, or any other statute governing electronic transactions.

[Message part 2 (text/html, inline)]

Message #70 received at 1502 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Jason Rumney <jasonr <at> f2s.com>
To: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>,
        1502 <at> debbugs.gnu.org
Subject: Re: bug#1502: CR/LF Unicode Problem
Date: Sun, 07 Dec 2008 23:58:15 +0800

Lafleur, Henry wrote:
>
> When I load a Unicode file (UTF-8) where some lines in CR/LF and some 
> lines end in LF, in hexl-mode the CR/LF EOL's appear as CR/CR/LF and 
> the LF EOL's appear as CR/LF. See this thread for more information:
>
> _http://social.msdn.microsoft.com/Forums/en-US/netfxbcl/thread/8ef5b69d-135d-4584-ae1a-1caeb4afc846/#page:1_ 
>
>
> If I save the file in hexl mode, it will save it with the extra CRs, 
> thus modifying the file more than anticipated.
>

I don't see this with your sample.sql file using Emacs 22.3 here.
Do you still see the bug if you start Emacs from the command line as: 
emacs -Q

Message #75 received at 1502 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Henry Lafleur" <lafleurh <at> gmail.com>
To: 1502 <at> debbugs.gnu.org, eliz <at> gnu.org, jasonrumney <at> gmail.com
Subject: Re: bug#1502: CR/LF Unicode Problem: UTF-8 file with problem
Date: Sun, 7 Dec 2008 23:25:40 -0600

[Message part 1 (text/plain, inline)]

Here's the file that is causing the problem.

It starts with the with the 3-byte order mark and each line ends with with
CR/LF except the two lines with character codes > 128 which end only in LF.
When I open the file in Windows Emacs, each LF is preceeded by an additional
CR so the lines with CR/LF end in CR/CR/LF and lines that end in LF end in
CR/LF.

If I start emacs with the -Q option, I get the same result. Normally I start
Emacs by running the runemacs.exe. I'm running on Windows. (I'll check the
problem on Linux once I get the file there also.)

Thanks,

Henry.

[Message part 2 (text/html, inline)]

[RRTestAppend1.sql (text/plain, attachment)]

Message #80 received at 1502 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Henry Lafleur" <lafleurh <at> gmail.com>
To: 1502 <at> debbugs.gnu.org, eliz <at> gnu.org, jasonrumney <at> gmail.com
Subject: Re: bug#1502: CR/LF Unicode Problem: UTF-8 file with problem
Date: Sun, 7 Dec 2008 23:45:46 -0600

[Message part 1 (text/plain, inline)]

OK, Emacs 22.1 on Linux (OpenSUSE 10.3) works just fine. The problem appears
to be isloated to Windows. I'm running Windows XP service pack 2, Emacs 22.1
or 22.3 and have the problem.

Thanks,

Henry.

On Sun, Dec 7, 2008 at 11:25 PM, Henry Lafleur <lafleurh <at> gmail.com> wrote:

>
> Here's the file that is causing the problem.
>
> It starts with the with the 3-byte order mark and each line ends with with
> CR/LF except the two lines with character codes > 128 which end only in LF.
> When I open the file in Windows Emacs, each LF is preceeded by an additional
> CR so the lines with CR/LF end in CR/CR/LF and lines that end in LF end in
> CR/LF.
>
> If I start emacs with the -Q option, I get the same result. Normally I
> start Emacs by running the runemacs.exe. I'm running on Windows. (I'll check
> the problem on Linux once I get the file there also.)
>
> Thanks,
>
> Henry.
>

[Message part 2 (text/html, inline)]

[RRTestAppend1.sql (text/plain, attachment)]

Message #85 received at 1502 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>
To: "Jason Rumney" <jasonr <at> f2s.com>, <1502 <at> debbugs.gnu.org>
Cc: "Eli Zaretskii" <eliz <at> gnu.org>
Subject: RE: bug#1502: CR/LF Unicode Problem
Date: Mon, 8 Dec 2008 15:26:13 -0600

Jason,

I don't see the problem in the sample.sql on Windows either. I was just
trying to get something to you guys over the weekend. The other file
uploaded, though, exhibits the problem (RRTestAppend1.sql).

Thanks,

Henry. 

-----Original Message-----
From: Jason Rumney [mailto:jasonrumney <at> gmail.com] On Behalf Of Jason
Rumney
Sent: Sunday, December 07, 2008 9:58 AM
To: Lafleur, Henry; 1502 <at> emacsbugs.donarmstrong.com
Subject: Re: bug#1502: CR/LF Unicode Problem

Lafleur, Henry wrote:
>
> When I load a Unicode file (UTF-8) where some lines in CR/LF and some 
> lines end in LF, in hexl-mode the CR/LF EOL's appear as CR/CR/LF and 
> the LF EOL's appear as CR/LF. See this thread for more information:
>
> _http://social.msdn.microsoft.com/Forums/en-US/netfxbcl/thread/8ef5b69
> d-135d-4584-ae1a-1caeb4afc846/#page:1_
>
>
> If I save the file in hexl mode, it will save it with the extra CRs, 
> thus modifying the file more than anticipated.
>

I don't see this with your sample.sql file using Emacs 22.3 here.
Do you still see the bug if you start Emacs from the command line as: 
emacs -Q


-----------------------------------------
CANRIG EMAIL NOTICE - This transmission may be strictly
confidential. If you are not the intended recipient of this
message, you may not disclose, print, copy, or disseminate this
information. If you have received this in error, please reply and
notify the sender (only) and delete the message. Unauthorized
interception of this e-mail is a violation of federal criminal law.
This communication does not reflect an intention by the sender or
the sender's principal to conduct a transaction or make any
agreement by electronic means. Nothing contained in this message or
in any attachment shall satisfy the requirements for a writing, and
nothing contained herein shall constitute a contract or electronic
signature under the Electronic Signatures in Global and National
Commerce Act, any version of the Uniform Electronic Transactions
Act, or any other statute governing electronic transactions.

Message #90 received at 1502 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Lafleur, Henry" <Henry.LaFleur <at> canrig.com>
To: <1502 <at> debbugs.gnu.org>
Subject: RE: bug#1502: Info received (bug#1502: CR/LF Unicode Problem)
Date: Mon, 8 Dec 2008 16:11:53 -0600

One more thing to throw in the mix, I have this in my .emacs file for
UTF-16:

;;
;; Auto-detect UTF-16 files
;;

    ;; Add missing support functions
    (defun utf-16-le-pre-write-conversion (start end) nil)
    (defun utf-16-be-pre-write-conversion (start end) nil)

    ;; Set up auto-load of UTF-16 files using the appropriate coding
system.
    (setq coding-category-utf-16-le 'utf-16-le)
    (push 'coding-category-utf-16-le coding-category-list)

    ;; Detect endianness of UTF-16 containing a Byte Order Mark U+FEFF
    ;; Detect EOL mode by looking for CR/LF on the first line
    (add-to-list 'auto-coding-regexp-alist '("^\xFF\xFE.*\x0D\x00$" .
utf-16-le-dos) t)
    (add-to-list 'auto-coding-regexp-alist '("^\xFE\xFF.*\x0D\x00$" .
utf-16-be-dos) t)
    (add-to-list 'auto-coding-regexp-alist '("^\xFF\xFE" . utf-16-le) t)
    (add-to-list 'auto-coding-regexp-alist '("^\xFE\xFF" . utf-16-be) t)

I have no idea if this would cause the problem. These files start with
xEF xBB xBF, which shouldn't match the byte order marks above.

Thanks,

Henry. 

-----Original Message-----
From: Emacs bug Tracking System [mailto:don <at> donarmstrong.com] 
Sent: Monday, December 08, 2008 3:35 PM
To: Lafleur, Henry
Subject: bug#1502: Info received (bug#1502: CR/LF Unicode Problem)


Thank you for the additional information you have supplied regarding
this bug report.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 Emacs Bugs <bug-gnu-emacs <at> gnu.org>

If you wish to submit further information on this problem, please send
it to 1502 <at> emacsbugs.donarmstrong.com, as before.

Please do not send mail to don <at> donarmstrong.com unless you wish to
report a problem with the Bug-tracking system.


--
1502: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=1502
Emacs Bug Tracking System
Contact don <at> donarmstrong.com with problems

-----------------------------------------
CANRIG EMAIL NOTICE - This transmission may be strictly
confidential. If you are not the intended recipient of this
message, you may not disclose, print, copy, or disseminate this
information. If you have received this in error, please reply and
notify the sender (only) and delete the message. Unauthorized
interception of this e-mail is a violation of federal criminal law.
This communication does not reflect an intention by the sender or
the sender's principal to conduct a transaction or make any
agreement by electronic means. Nothing contained in this message or
in any attachment shall satisfy the requirements for a writing, and
nothing contained herein shall constitute a contract or electronic
signature under the Electronic Signatures in Global and National
Commerce Act, any version of the Uniform Electronic Transactions
Act, or any other statute governing electronic transactions.

Message #95 received at 1502 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: jasonr <at> f2s.com
To: Henry Lafleur <lafleurh <at> gmail.com>, 1502 <at> debbugs.gnu.org
Cc: eliz <at> gnu.org
Subject: Re: bug#1502: CR/LF Unicode Problem: UTF-8 file with problem
Date: Mon,  8 Dec 2008 23:50:48 +0000

found 1502 22.3
fixed 1502 23.0.50
thanks

Quoting Henry Lafleur <lafleurh <at> gmail.com>:

> It starts with the with the 3-byte order mark and each line ends with with
> CR/LF except the two lines with character codes > 128 which end only in LF.
> When I open the file in Windows Emacs, each LF is preceeded by an additional
> CR so the lines with CR/LF end in CR/CR/LF and lines that end in LF end in
> CR/LF.

This seems to be fixed in the CVS trunk, though I can reproduce it in 22.3.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.