GNU bug report logs - #8122
23.2.94; (prefer-coding-system 'utf-8) changes indentation behavior for emacsclient

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 8122 in the body.
You can then email your comments to 8122 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Steve Revilak <steve <at> srevilak.net>
To: bug-gnu-emacs <at> gnu.org
Subject: 23.2.94; (prefer-coding-system 'utf-8) changes indentation
	behavior for emacsclient
Date: Sat, 26 Feb 2011 09:31:16 -0500

[Message part 1 (text/plain, inline)]

This bug report will be sent to the Free Software Foundation,
not to your local site managers!
Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your report will be posted to the bug-gnu-emacs <at> gnu.org mailing list
and the gnu.emacs.bug news group, and at http://debbugs.gnu.org.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug.  If you can, give
a recipe starting from `emacs -Q':

In the 23.2.94 emacs pretest, the use of (prefer-coding-system 'utf-8)
changes text-mode indentation behavior with emacsclient/emacs --daemon.

Steps to reproduce

(1) Start emacs in daemon mode, as follows:
    env TMPDIR=$HOME/tmp emacs  -Q --eval "(prefer-coding-system 'utf-8)" --daemon

(2) Use emacsclient to connect to the server started in step (1),
    editing a new text file.
    rm -f a.txt
    env TMPDIR=$HOME/tmp emacsclient a.txt

(3) In the buffer for a.txt, type the lines that appear below.  Do not
    press return after typing "ccc"

aaa
bbb

  ccc

(4) With point at the end of line "ccc", press RETURN.  If "ccc"
    appeared in line 5 of the buffer, then the cursor will appear at
    line 6, column 2.  

Expected behavior: after pressing RETURN, the cursor should have
appeared line line 6, column 0.

(5) With the cursor at line 6, column 2.  Press ESC-\ (aka
    "delete-horizonal-space").   Now, the cursor appears in line 6,
    column 0

(6) With the cursor in line 6, column 0, press RETURN.  The cursor now
    appears in line 7, column 2.

Expected behavior: after pressing RETURN, the cursor should have
appeared in line 7, column 0.

Regression Notes:

* The behavior I've described appears to be specific to emacsclient.
  For example, 
      emacs  -Q --eval "(prefer-coding-system 'utf-8)"
  does not exhibit the behavior I've described.

* The last emacs pretest that I used on this system was 23.2.91.
  23.2.91 did not exhibit this behavior.
  Later today, I will try building 23.2.92 and 23.2.93, to get see if
  the behavior appeared in those versions.

* I've observed this behavior on Mac OS X 
  Darwin oatmeal 10.4.0 Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010; root:xnu-1504.7.4~1/RELEASE_I386 i386

  and on OpenSUSE 11.3.
  Linux sunny 2.6.34.7-0.7-desktop #1 SMP PREEMPT 2010-12-13 11:13:53 +0100 x86_64 x86_64 x86_64 GNU/Linux


If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
/Applications/Emacs.app/Contents/Resources/etc/DEBUG.


In GNU Emacs 23.2.94.1 (x86_64-apple-darwin10.4.0, NS apple-appkit-1038.32)
 of 2011-02-15 on oatmeal.ma.runwaynine.com
Windowing system distributor `Apple', version 10.3.1038
configured using `configure  '--with-ns''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: C
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  diff-auto-refine-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC 
[ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A 
ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC 
[ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A 
ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC 
[ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A 
ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC 
[ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A 
ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC 
[ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A 
ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC 
[ A ESC [ A ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B 
ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC 
[ B ESC [ B ESC [ B ESC [ B ESC [ B DEL C-x C-s C-@ 
C-e ESC w C-x v u y e s RET C-x C-s ESC x r e p o r 
t - e m a c s TAB RET

Recent messages:
Saving file /Users/srevilak/.emacs...
Wrote /Users/srevilak/.emacs
Saving file /Users/srevilak/.emacs...
Wrote /Users/srevilak/.emacs
Saving file /Users/srevilak/.emacs...
Wrote /Users/srevilak/.emacs
Mark activated
Finding changes in /Users/srevilak/.emacs...
Reverting /Users/srevilak/.emacs...done
(No changes need to be saved)

Load-path shadows:
None found.

Features:
(shadow sort mail-extr message sendmail regexp-opt ecomplete rfc822 mml
mml-sec password-cache mm-decode mm-bodies mm-encode mailcap mail-parse
rfc2231 rfc2047 rfc2045 qp ietf-drums mailabbrev nnheader gnus-util
netrc time-date mm-util mail-prsvr gmm-utils wid-edit mailheader canlock
sha1 hex-util hashcash mail-utils emacsbug log-edit ring pcvs-util
add-log diff-mode easy-mmode vc vc-dispatcher rect pp find-func help-fns
help-mode view debug vc-git tooltip ediff-hook vc-hooks lisp-float-type
mwheel ns-win easymenu tool-bar dnd fontset image fringe lisp-mode
register page menu-bar rfn-eshadow timer select scroll-bar mldrag mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
loaddefs button minibuffer faces cus-face files text-properties overlay
md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process ns multi-tty
emacs)

[Message part 2 (application/pgp-signature, inline)]

Message #8 received at 8122 <at> debbugs.gnu.org (full text, mbox):

From: Steve Revilak <steve <at> srevilak.net>
To: 8122 <at> debbugs.gnu.org
Subject: More regression notes for bug#8122
Date: Sat, 26 Feb 2011 10:05:24 -0500

[Message part 1 (text/plain, inline)]

I've tried to reproduce bug 8122 on pretest versions 23.2.92 and
23.2.93.  Here are my findings:

Bug 8122 occurs in 23.2.94.

Bug 8122 DOES NOT occur with 23.2.91, 23.2.92, or 23.2.93.

So, this appears to be new behavior in 23.2.94.

[Message part 2 (application/pgp-signature, inline)]

Message #11 received at 8122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Steve Revilak <steve <at> srevilak.net>
Cc: 8122 <at> debbugs.gnu.org
Subject: Re: bug#8122: 23.2.94;
	(prefer-coding-system 'utf-8) changes indentation behavior
	for	emacsclient
Date: Sat, 26 Feb 2011 17:37:49 +0200

> Date: Sat, 26 Feb 2011 09:31:16 -0500
> From: Steve Revilak <steve <at> srevilak.net>
> Cc: 
> 
> (3) In the buffer for a.txt, type the lines that appear below.  Do not
>      press return after typing "ccc"
> 
> aaa
> bbb
> 
>    ccc
> 
> (4) With point at the end of line "ccc", press RETURN.  If "ccc"
>      appeared in line 5 of the buffer, then the cursor will appear at
>      line 6, column 2.  
> 
> Expected behavior: after pressing RETURN, the cursor should have
> appeared line line 6, column 0.

This happens because RET is somehow perceived as C-j, which invokes
newline-and-indent, and not as RET that runs newline.  Try "C-h c RET"
and see what it says.

I have no idea how prefer-coding-system is able to modify the way RET
is interpreted by Emacs.

Message #14 received at 8122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Steve Revilak <steve <at> srevilak.net>
Cc: 8122 <at> debbugs.gnu.org
Subject: Re: bug#8122: More regression notes for bug#8122
Date: Sat, 26 Feb 2011 17:49:53 +0200

> Date: Sat, 26 Feb 2011 10:05:24 -0500
> From: Steve Revilak <steve <at> srevilak.net>
> Cc: 
> 
> I've tried to reproduce bug 8122 on pretest versions 23.2.92 and
> 23.2.93.  Here are my findings:
> 
> Bug 8122 occurs in 23.2.94.
> 
> Bug 8122 DOES NOT occur with 23.2.91, 23.2.92, or 23.2.93.
> 
> So, this appears to be new behavior in 23.2.94.

The culprit seems to be this change:

  2011-02-12  Eli Zaretskii  <eliz <at> gnu.org>

	  * terminal.c (create_terminal): Use default-keyboard-coding-system
	  and default-terminal-coding-system to initialize coding systems of
	  the new terminal.  (Bug#7840)

but I'd be damned if I understand how could it affect the character
produced by the RET key.  Hmm...

Message #17 received at 8122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Steve Revilak <steve <at> srevilak.net>, Kenichi Handa <handa <at> m17n.org>
Cc: 8122 <at> debbugs.gnu.org
Subject: Re: bug#8122: 23.2.94;
	(prefer-coding-system 'utf-8) changes indentation behavior
	for	emacsclient
Date: Sat, 26 Feb 2011 18:26:10 +0200

> Date: Sat, 26 Feb 2011 09:31:16 -0500
> From: Steve Revilak <steve <at> srevilak.net>
> Cc: 
> 
> (1) Start emacs in daemon mode, as follows:
>      env TMPDIR=$HOME/tmp emacs  -Q --eval "(prefer-coding-system 'utf-8)" --daemon
> 
> (2) Use emacsclient to connect to the server started in step (1),
>      editing a new text file.
>      rm -f a.txt
>      env TMPDIR=$HOME/tmp emacsclient a.txt
> 
> (3) In the buffer for a.txt, type the lines that appear below.  Do not
>      press return after typing "ccc"
> 
> aaa
> bbb
> 
>    ccc
> 
> (4) With point at the end of line "ccc", press RETURN.  If "ccc"
>      appeared in line 5 of the buffer, then the cursor will appear at
>      line 6, column 2.  
> 
> Expected behavior: after pressing RETURN, the cursor should have
> appeared line line 6, column 0.

I found the reason for this: when keyboard-coding-system is utf-8,
typing RET is decoded as C-j.  If you start the Emacs daemon like
this:

 emacs  -Q --eval "(prefer-coding-system 'utf-8-unix)" --daemon
                                          ^^^^^^^^^^
the problem goes away.

Message #20 received at 8122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Steve Revilak <steve <at> srevilak.net>, Kenichi Handa <handa <at> m17n.org>
Cc: 8122 <at> debbugs.gnu.org
Subject: Re: bug#8122: 23.2.94;
	(prefer-coding-system 'utf-8) changes indentation behavior
	for	emacsclient
Date: Sat, 26 Feb 2011 18:40:32 +0200

> Date: Sat, 26 Feb 2011 09:31:16 -0500
> From: Steve Revilak <steve <at> srevilak.net>
> Cc: 
> 
> (1) Start emacs in daemon mode, as follows:
>      env TMPDIR=$HOME/tmp emacs  -Q --eval "(prefer-coding-system 'utf-8)" --daemon
> 
> (2) Use emacsclient to connect to the server started in step (1),
>      editing a new text file.
>      rm -f a.txt
>      env TMPDIR=$HOME/tmp emacsclient a.txt
> 
> (3) In the buffer for a.txt, type the lines that appear below.  Do not
>      press return after typing "ccc"
> 
> aaa
> bbb
> 
>    ccc
> 
> (4) With point at the end of line "ccc", press RETURN.  If "ccc"
>      appeared in line 5 of the buffer, then the cursor will appear at
>      line 6, column 2.  
> 
> Expected behavior: after pressing RETURN, the cursor should have
> appeared line line 6, column 0.

I found the reason for this: when keyboard-coding-system is utf-8,
typing RET is decoded as C-j.  If you start the Emacs daemon like
this:

 emacs  -Q --eval "(prefer-coding-system 'utf-8-unix)" --daemon
                                          ^^^^^^^^^^
the problem goes away.

This happens because, unlike set-keyboard-coding-system, which adds
"-unix" to its argument, we don't do the same when we set up
default-keyboard-coding-system.

(This problem is quite old; I guess no one used
default-keyboard-coding-system until the change I made in terminal.c
exposed this bug.)

I propose the following change to fix this:

--- lisp/international/mule-cmds.el~0	2011-01-08 12:45:14.000000000 -0500
+++ lisp/international/mule-cmds.el	2011-02-26 11:32:22.483284753 -0500
@@ -368,7 +368,8 @@
 		 (coding-system-get coding-system 'ascii-compatible-p)))
 	(setq default-file-name-coding-system coding-system)))
   (setq default-terminal-coding-system coding-system)
-  (setq default-keyboard-coding-system coding-system)
+  (setq default-keyboard-coding-system
+	(coding-system-change-eol-conversion coding-system 'unix))
   ;; Preserve eol-type from existing default-process-coding-systems.
   ;; On non-unix-like systems in particular, these may have been set
   ;; carefully by the user, or by the startup code, to deal with the

Message #23 received at 8122 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Kenichi Handa <handa <at> m17n.org>, 8122 <at> debbugs.gnu.org,
	Steve Revilak <steve <at> srevilak.net>
Subject: Re: bug#8122: 23.2.94;
	(prefer-coding-system 'utf-8) changes indentation behavior
	for	emacsclient
Date: Sat, 26 Feb 2011 14:56:37 -0500

> I propose the following change to fix this:

Sounds right, thank you,


        Stefan


> --- lisp/international/mule-cmds.el~0	2011-01-08 12:45:14.000000000 -0500
> +++ lisp/international/mule-cmds.el	2011-02-26 11:32:22.483284753 -0500
> @@ -368,7 +368,8 @@
>  		 (coding-system-get coding-system 'ascii-compatible-p)))
>  	(setq default-file-name-coding-system coding-system)))
>    (setq default-terminal-coding-system coding-system)
> -  (setq default-keyboard-coding-system coding-system)
> +  (setq default-keyboard-coding-system
> +	(coding-system-change-eol-conversion coding-system 'unix))
>    ;; Preserve eol-type from existing default-process-coding-systems.
>    ;; On non-unix-like systems in particular, these may have been set
>    ;; carefully by the user, or by the startup code, to deal with the

Message #28 received at 8122-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: handa <at> m17n.org, 8122-done <at> debbugs.gnu.org, steve <at> srevilak.net
Subject: Re: bug#8122: 23.2.94;
	(prefer-coding-system 'utf-8) changes indentation behavior
	for	emacsclient
Date: Sat, 26 Feb 2011 23:32:14 +0200

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: Steve Revilak <steve <at> srevilak.net>,  Kenichi Handa <handa <at> m17n.org>,  8122 <at> debbugs.gnu.org
> Date: Sat, 26 Feb 2011 14:56:37 -0500
> 
> > I propose the following change to fix this:
> 
> Sounds right, thank you,

Installed on the emacs-23 branch.

Message #31 received at 8122 <at> debbugs.gnu.org (full text, mbox):

From: Steve Revilak <steve <at> srevilak.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 8122 <at> debbugs.gnu.org, Kenichi Handa <handa <at> m17n.org>
Subject: Re: bug#8122: 23.2.94;	(prefer-coding-system 'utf-8) changes
	indentation behavior for	emacsclient
Date: Sat, 26 Feb 2011 17:57:38 -0500

[Message part 1 (text/plain, inline)]

>If you start the Emacs daemon like this:
>
> emacs  -Q --eval "(prefer-coding-system 'utf-8-unix)" --daemon
>                                          ^^^^^^^^^^
>the problem goes away.

Eli,

As you suggested, I tried changing (prefer-coding-system 'utf-8) to
(prefer-coding-system 'utf-8-unix).  Indeed, that makes the problem go
away.


>I propose the following change to fix this:
>
>--- lisp/international/mule-cmds.el~0	2011-01-08 12:45:14.000000000 -0500
>+++ lisp/international/mule-cmds.el	2011-02-26 11:32:22.483284753 -0500
>@@ -368,7 +368,8 @@
> 		 (coding-system-get coding-system 'ascii-compatible-p)))
> 	(setq default-file-name-coding-system coding-system)))
>   (setq default-terminal-coding-system coding-system)
>-  (setq default-keyboard-coding-system coding-system)
>+  (setq default-keyboard-coding-system
>+	(coding-system-change-eol-conversion coding-system 'unix))
>   ;; Preserve eol-type from existing default-process-coding-systems.
>   ;; On non-unix-like systems in particular, these may have been set
>   ;; carefully by the user, or by the startup code, to deal with the


I tested this patch on my system (with "(prefer-coding-system
'utf-8)").  It works.

Thanks for the speedy fix!

Steve

[Message part 2 (application/pgp-signature, inline)]

Message #32 received at 8122-done <at> debbugs.gnu.org (full text, mbox):

From: Steve Revilak <steve <at> srevilak.net>
To: 8122-done <at> debbugs.gnu.org
Subject: Re: bug#8122: closed (Re: bug#8122: 23.2.94; (prefer-coding-system
	'utf-8) changes indentation behavior for	emacsclient)
Date: Sat, 26 Feb 2011 17:58:42 -0500

[Message part 1 (text/plain, inline)]

[Message part 2 (application/pgp-signature, inline)]

Message #35 received at 8122 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Steve Revilak <steve <at> srevilak.net>
Cc: 8122 <at> debbugs.gnu.org, handa <at> m17n.org
Subject: Re: bug#8122: 23.2.94;
	(prefer-coding-system 'utf-8) changes indentation behavior
	for	emacsclient
Date: Sun, 27 Feb 2011 05:58:31 +0200

> Date: Sat, 26 Feb 2011 17:57:38 -0500
> From: Steve Revilak <steve <at> srevilak.net>
> Cc: Kenichi Handa <handa <at> m17n.org>, 8122 <at> debbugs.gnu.org
> 
> >If you start the Emacs daemon like this:
> >
> > emacs  -Q --eval "(prefer-coding-system 'utf-8-unix)" --daemon
> >                                          ^^^^^^^^^^
> >the problem goes away.
> 
> Eli,
> 
> As you suggested, I tried changing (prefer-coding-system 'utf-8) to
> (prefer-coding-system 'utf-8-unix).  Indeed, that makes the problem go
> away.
> 
> 
> >I propose the following change to fix this:
> >
> >--- lisp/international/mule-cmds.el~0	2011-01-08 12:45:14.000000000 -0500
> >+++ lisp/international/mule-cmds.el	2011-02-26 11:32:22.483284753 -0500
> >@@ -368,7 +368,8 @@
> > 		 (coding-system-get coding-system 'ascii-compatible-p)))
> > 	(setq default-file-name-coding-system coding-system)))
> >   (setq default-terminal-coding-system coding-system)
> >-  (setq default-keyboard-coding-system coding-system)
> >+  (setq default-keyboard-coding-system
> >+	(coding-system-change-eol-conversion coding-system 'unix))
> >   ;; Preserve eol-type from existing default-process-coding-systems.
> >   ;; On non-unix-like systems in particular, these may have been set
> >   ;; carefully by the user, or by the startup code, to deal with the
> 
> 
> I tested this patch on my system (with "(prefer-coding-system
> 'utf-8)").  It works.

Thanks for testing.  The fix is now on the release branch.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.