GNU bug report logs - #1003
23.0.60; set-keyboard-coding-system incompatible with TeX input method

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 1003 in the body.
You can then email your comments to 1003 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: lconrad <at> clavichord (Laura Conrad)
To: emacs-pretest-bug <at> gnu.org
Subject: 23.0.60; set-keyboard-coding-system incompatible with TeX input method
Date: Fri, 19 Sep 2008 12:11:17 -0400 (EDT)

in TeX input mode, I expect to be able to type ént by typing  "\'ent".
When I  set:

     (set-keyboard-coding-system 'utf-8)

typing those four characters gives me some Chinese character.

This is with the ubuntu Linux 8.04 emacs-snapshot package.

Message #10 received at 1003 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: lconrad <at> laymusic.org
Cc: 1003 <at> debbugs.gnu.org
Subject: Re: bug#1003: 23.0.60; set-keyboard-coding-system incompatible with TeX input method
Date: Sat, 20 Sep 2008 16:45:41 -0400

> in TeX input mode, I expect to be able to type ént by typing  "\'ent".
> When I  set:

>      (set-keyboard-coding-system 'utf-8)

> typing those four characters gives me some Chinese character.

IIUC the problem is that the input method processing is done before the
keyboard-coding-system processing.  So the sequence "\ ' e n t" is
turned by the input method into "é n t" (i.e. 233 110 116) which is then
interpreted as a utf-8 sequence which happens to give you
a chinese character.

In Emacs-22, bytes such as 233 were different from any other char, so
such confusion wasn't possible, which worked around the underlying
ordering problem.

A real fix would probably be to perform keyboard-coding-system decoding
earlier, but it's a non trivial change which may introduce its own set
of problems.

Maybe another way to fix the problem is to let quail emit not just é but
the sequence [im-char é] and then have input-decode-map strip the
`im-char' pseudo event; this would protect those im-generated chars from
being processed by the keyboard-coding-system processing.

Handa?


        Stefan

Message #15 received at 1003 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Kenichi Handa <handa <at> m17n.org>
To: monnier <at> iro.umontreal.ca
Cc: lconrad <at> laymusic.org, 1003 <at> debbugs.gnu.org
Subject: bug#1003: 23.0.60; set-keyboard-coding-system incompatible with TeX input method
Date: Mon, 25 May 2009 13:44:12 +0900

I've just noticed that this bug is not yet closed.

Laura Conrad <lconrad <at> laymusic.org> wrote:
> in TeX input mode, I expect to be able to type ént by typing  "\'ent".
> When I  set:
> 
>      (set-keyboard-coding-system 'utf-8)
> 
> typing those four characters gives me some Chinese character.
> 
> This is with the ubuntu Linux 8.04 emacs-snapshot package.

It seems that this bug is now fixed by Stefan's workaround
patch accompanied with this comment:

        ;; Invalid utf-8 sequence.  Might be because Quail got involved
        ;; in-between and the bytes we thought we were reading were actually
        ;; latin-1 chars.  Let's presume that `event' is the second "byte",
        ;; i.e. there weren't any "apprently correct" between `lead' and
        ;; `event': it's easy to recover in this case, and the more general
        ;; case seems pretty unlikely.
        ;; FIXME: We should really do encoded-kbd decoding before processing
        ;; input-methods.

The strange thing is that the date of the orignal bug report
is 2008-09-19 but the above patch was installed on
2008-06-15.  Laura, do you still see the problem?

Stefan Monnier <monnier <at> iro.umontreal.ca> wrote:
> A real fix would probably be to perform keyboard-coding-system decoding
> earlier, but it's a non trivial change which may introduce its own set
> of problems.

I vaguely remember that I sent a patch to do that long ago,
but it seems that the discussion fizzled out without
reaching any conclusion.

> Maybe another way to fix the problem is to let quail emit not just é but
> the sequence [im-char é] and then have input-decode-map strip the
> `im-char' pseudo event; this would protect those im-generated chars from
> being processed by the keyboard-coding-system processing.

That's one idea, but seems tricky.  I prefer the first
method.

---
Kenichi Handa
handa <at> m17n.org

Message #20 received at 1003 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Laura Conrad <lconrad <at> laymusic.org>
To: Kenichi Handa <handa <at> m17n.org>
Cc: monnier <at> iro.umontreal.ca, 1003 <at> debbugs.gnu.org
Subject: Re: bug#1003: 23.0.60; set-keyboard-coding-system incompatible with TeX input method
Date: Mon, 25 May 2009 07:01:31 -0400

>>>>> "Kenichi" == Kenichi Handa <handa <at> m17n.org> writes:

    Kenichi> The strange thing is that the date of the orignal bug report
    Kenichi> is 2008-09-19 but the above patch was installed on
    Kenichi> 2008-06-15.  Laura, do you still see the problem?

No.  Not on ubuntu 9.04 with GNU Emacs 23.0.91.1.

The report was made when I upgraded to Ubuntu 8.04, with whatever
version their emacs-snapshot was.

Thanks for the fix.

-- 
Laura   (mailto:lconrad <at> laymusic.org http://www.laymusic.org/ )
(617) 661-8097	233 Broadway, Cambridge, MA 02139   

Jesus, I'm Thirsty.  (Danku)

'Who's getting the beer?'
Judas stepped up to the bar.
He'd had a windfall!

Danny Reynolds

Message #25 received at 1003 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Kenichi Handa <handa <at> m17n.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: lconrad <at> laymusic.org, 1003 <at> debbugs.gnu.org
Subject: Re: bug#1003: 23.0.60; set-keyboard-coding-system incompatible with
 TeX input method
Date: Wed, 01 Jul 2009 20:49:22 +0900

In article <jwv1vqd47n7.fsf-monnier+emacsbugreports <at> gnu.org>, Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>>> A real fix would probably be to perform keyboard-coding-system decoding
>>> earlier, but it's a non trivial change which may introduce its own set
>>> of problems.

> > I vaguely remember that I sent a patch to do that long ago,
> > but it seems that the discussion fizzled out without
> > reaching any conclusion.

> I also had a vague feeling of seeing such a thing, but couldn't find it
> in the archives, so I figured I was confused.
> If not, please try and remember to install it for Emacs-23.2.

I've just installed the change.  As encoded-kb.el is now not
necessary, I deleted that file.

---
Kenichi Handa
handa <at> m17n.org

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.