GNU bug report logs - #76669
read_key_sequence discards events without attempting remapping

Previous Next

To reply to this bug, email your comments to 76669 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: bug-gnu-emacs <at> gnu.org
Subject: read_key_sequence discards events without attempting remapping
Date: Sat, 01 Mar 2025 18:35:31 -0500

Suppose we want to use local-function-key-map to translate
s-<down-mouse-3> to <down-mouse-3>.  If we don't have a concrete binding
for s-<down-mouse-3>, we never attempt to translate it: instead,
read_key_sequence discards the down event and skips right to
s-<mouse-3>, which we *can* translate.

Message #8 received at 76669 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Daniel Colascione <dancol <at> dancol.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 76669 <at> debbugs.gnu.org
Subject: Re: bug#76669: read_key_sequence discards events without attempting
 remapping
Date: Sun, 02 Mar 2025 07:51:35 +0200

> From: Daniel Colascione <dancol <at> dancol.org>
> Date: Sat, 01 Mar 2025 18:35:31 -0500
> 
> Suppose we want to use local-function-key-map to translate
> s-<down-mouse-3> to <down-mouse-3>.  If we don't have a concrete binding
> for s-<down-mouse-3>, we never attempt to translate it: instead,
> read_key_sequence discards the down event and skips right to
> s-<mouse-3>, which we *can* translate.

I think it's ambiguous which one should come first: the discarding of
'down' modifier or translation via that map.  The ELisp manual says:

     The function ‘read-key-sequence’ ignores any button-down events that
  don't have command bindings; therefore, the Emacs command loop ignores
  them too.  This means that you need not worry about defining button-down
  events unless you want them to do something.  The usual reason to define
  a button-down event is so that you can track mouse motion (by reading
  motion events) until the button is released.  *Note Motion Events::.

And also:

     If an input character is upper-case (or has the shift modifier) and
  has no key binding, but its lower-case equivalent has one, then
  ‘read-key-sequence’ converts the character to lower case.  (This
  behavior can be disabled by setting the
  ‘translate-upper-case-key-bindings’ user option to ‘nil’.)  Note that
  ‘lookup-key’ does not perform case conversion in this way.

     When reading input results in such a “shift-translation”, Emacs sets
  the variable ‘this-command-keys-shift-translated’ to a non-‘nil’ value.
  Lisp programs can examine this variable if they need to modify their
  behavior when invoked by shift-translated keys.  For example, the
  function ‘handle-shift-selection’ examines the value of this variable to
  determine how to activate or deactivate the region (*note
  handle-shift-selection: The Mark.).

     The function ‘read-key-sequence’ also transforms some mouse events.
  It converts unbound drag events into click events, and discards unbound
  button-down events entirely.  It also reshuffles focus events and
  miscellaneous window events so that they never appear in a key sequence
  with any other events.

Adding Stefan to the discussion.

Message #11 received at 76669 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 76669 <at> debbugs.gnu.org
Subject: Re: bug#76669: read_key_sequence discards events without attempting
 remapping
Date: Sun, 02 Mar 2025 02:02:50 -0500

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Daniel Colascione <dancol <at> dancol.org>
>> Date: Sat, 01 Mar 2025 18:35:31 -0500
>> 
>> Suppose we want to use local-function-key-map to translate
>> s-<down-mouse-3> to <down-mouse-3>.  If we don't have a concrete binding
>> for s-<down-mouse-3>, we never attempt to translate it: instead,
>> read_key_sequence discards the down event and skips right to
>> s-<mouse-3>, which we *can* translate.
>
> I think it's ambiguous which one should come first: the discarding of
> 'down' modifier or translation via that map.  The ELisp manual says:
>
>      The function ‘read-key-sequence’ ignores any button-down events that
>   don't have command bindings; therefore, the Emacs command loop ignores
>   them too.  This means that you need not worry about defining button-down
>   events unless you want them to do something.  The usual reason to define
>   a button-down event is so that you can track mouse motion (by reading
>   motion events) until the button is released.  *Note Motion Events::.
>
> And also:
>
>      If an input character is upper-case (or has the shift modifier) and
>   has no key binding, but its lower-case equivalent has one, then
>   ‘read-key-sequence’ converts the character to lower case.  (This
>   behavior can be disabled by setting the
>   ‘translate-upper-case-key-bindings’ user option to ‘nil’.)  Note that
>   ‘lookup-key’ does not perform case conversion in this way.
>
>      When reading input results in such a “shift-translation”, Emacs sets
>   the variable ‘this-command-keys-shift-translated’ to a non-‘nil’ value.
>   Lisp programs can examine this variable if they need to modify their
>   behavior when invoked by shift-translated keys.  For example, the
>   function ‘handle-shift-selection’ examines the value of this variable to
>   determine how to activate or deactivate the region (*note
>   handle-shift-selection: The Mark.).
>
>      The function ‘read-key-sequence’ also transforms some mouse events.
>   It converts unbound drag events into click events, and discards unbound
>   button-down events entirely.  It also reshuffles focus events and
>   miscellaneous window events so that they never appear in a key sequence
>   with any other events.
>
> Adding Stefan to the discussion.

Ambiguity aside, it's useful to translate down-mouse and other
discardable events like other events.  Consider trying to make a
local-function-key-map that translates super-modified mouse events to
unmodified mouse events without overriding explicit super-modified
bindings.  This translation can't translate s-down-mouse-1 to
down-mouse-1 unless there's a binding for down-mouse-1 (which defeats
the purpose of a translation).

Wouldn't it be simpler and more powerful to get the modifier-munging
stuff out of the main read_key_sequence loop and implement the down
event dropping in access_keymap_keyremap?

Message #14 received at 76669 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 76669 <at> debbugs.gnu.org
Subject: Re: bug#76669: read_key_sequence discards events without attempting
 remapping
Date: Sun, 02 Mar 2025 09:23:39 -0500

>>> Suppose we want to use local-function-key-map to translate
>>> s-<down-mouse-3> to <down-mouse-3>.  If we don't have a concrete binding
>>> for s-<down-mouse-3>, we never attempt to translate it: instead,
>>> read_key_sequence discards the down event and skips right to
>>> s-<mouse-3>, which we *can* translate.

Hmm... to be honest, I'm surprised that this is what happens, because my
recollection of the code was that dropping down/double/drag (and shift)
is done "as late as possible".

Maybe this is just a bug?

>> I think it's ambiguous which one should come first: the discarding of
>> 'down' modifier or translation via that map.  The ELisp manual says:

Given that translation is "customizable" it's desirable to go through
translation before deciding to drop `down`, but it's also desirable to
go through translation after dropping `down`.

FWIW, I have several times wished the `function-key-map`-style "remap if
there's no binding" was applied repeatedly.

> Wouldn't it be simpler and more powerful to get the modifier-munging
> stuff out of the main read_key_sequence loop and implement the down
> event dropping in access_keymap_keyremap?

That would prevent applying remapping after the `down` thingy
was dropped.

In my wildest dreams, `read_key_sequence` has an `input-fallback-map`
which is applied repeatedly, and keymaps can be functions so that they
can do things like remap anything that looks like `FOO-mouse-1` to
`FOO-mouse-2` (without having to list the hundreds of possible FOO) or
remap anything that looks like `down-FOO` to `FOO`.
Then `read_key_sequence` doesn't treat shift and `down` specially,
instead that can be moved out to `input-fallback-map`.

But, yeah, that still leaves open the question of whether we should drop
`shift` before we drop `down` or the reverse, and whether the
`FOO-mouse-1` to `FOO-mouse-2` remapping should happen before or after
dropping `down`.  The only "principled" way to solve this problem,
AFAICT, would be to consider those remapping and returning all the
possibilities (as a set of possible keysequences) and then when we do
the corresponding key lookup, we'd have to decide what to do if there's
more than one binding for those keysequences.


        Stefan

Message #17 received at 76669 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 76669 <at> debbugs.gnu.org
Subject: Re: bug#76669: read_key_sequence discards events without attempting
 remapping
Date: Sun, 02 Mar 2025 12:15:58 -0500

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>>>> Suppose we want to use local-function-key-map to translate
>>>> s-<down-mouse-3> to <down-mouse-3>.  If we don't have a concrete binding
>>>> for s-<down-mouse-3>, we never attempt to translate it: instead,
>>>> read_key_sequence discards the down event and skips right to
>>>> s-<mouse-3>, which we *can* translate.
>
> Hmm... to be honest, I'm surprised that this is what happens, because my
> recollection of the code was that dropping down/double/drag (and shift)
> is done "as late as possible".
>
> Maybe this is just a bug?
>
>>> I think it's ambiguous which one should come first: the discarding of
>>> 'down' modifier or translation via that map.  The ELisp manual says:
>
> Given that translation is "customizable" it's desirable to go through
> translation before deciding to drop `down`, but it's also desirable to
> go through translation after dropping `down`.
>
> FWIW, I have several times wished the `function-key-map`-style "remap if
> there's no binding" was applied repeatedly.
>
>> Wouldn't it be simpler and more powerful to get the modifier-munging
>> stuff out of the main read_key_sequence loop and implement the down
>> event dropping in access_keymap_keyremap?
>
> That would prevent applying remapping after the `down` thingy
> was dropped.

No --- I meant we'd have the remapping code *do* the dropping.
Apply the remapping in a loop until we reach a fixed point.

Remapping phases:

1. [s-down-mouse-1 s-mouse-1]
2. [down-mouse-1 mouse-1]
3. [mouse-1]

> In my wildest dreams, `read_key_sequence` has an `input-fallback-map`
> which is applied repeatedly, and keymaps can be functions so that they
> can do things like remap anything that looks like `FOO-mouse-1` to
> `FOO-mouse-2` (without having to list the hundreds of possible FOO) or
> remap anything that looks like `down-FOO` to `FOO`.
> Then `read_key_sequence` doesn't treat shift and `down` specially,
> instead that can be moved out to `input-fallback-map`.

Keymap entries can already be functions, and I think the function entry
would work with the fallback finding t too.  There's no good way to
compose such functions, though --- not without just clobbering them with
add-function and such.  Child keymap fallback entries will shadow those
of the parents.

> But, yeah, that still leaves open the question of whether we should drop
> `shift` before we drop `down` or the reverse, and whether the
> `FOO-mouse-1` to `FOO-mouse-2` remapping should happen before or after
> dropping `down`.  The only "principled" way to solve this problem,
> AFAICT, would be to consider those remapping and returning all the
> possibilities (as a set of possible keysequences) and then when we do
> the corresponding key lookup, we'd have to decide what to do if there's
> more than one binding for those keysequences.

If we're going to fix this, we might as well move read-key-sequence to
Lisp, where it's easier to express the right algorithm.

We can model the raw input event stream as a lazy infinite sequence with
lookahead.  The access-keymap operation would do another physical event
read if it wanted lookahead and we didn't have the physical event yet.

We can model key remapping with recursion, yes?  A remapping keymap
would just recursively try the whole key mapping sequence with its
new binding.

Here's a sketch.  (Real code would have much better style.)

(defvar all-keymaps ...)  ; global-map local-map function-key-map etc.

(def-defstruct candidate-keys
   ;; (- (length keys) nr-committed) is the
  ;; amount of lookahead we have
  (keys :type vector)
  (nr-committed :type integer))

(defun try-match-key-sequence (candidate-keys)
  ;; Go through all our keymaps.  If any of them returns
  ;; a concrete binding, use it immediately.  Otherwise,
  ;; return :want-lookahead if any keymap wants more lookahead.
  (cl-loop
    with want-lookahead = nil
    for keymap in all-keymaps
    for binding = (keymap-try-get-binding keymap candidate-keys)
    if (eq binding :want-lookahead)
    do (setf want-lookahead :want-lookahead)
    else if binding return binding
    finally return want-lookahead))

(defun keymap-try-get-binding (keymap candidate-keys)
  ;; See whether keymap has a binding for candidate-keys
  ;; If definitely no binding, return nil.  If we're not
  ;; sure and need to see more input (e.g. because
  ;; the committed part of candidate-keys names a non-empty prefix map)
  ;; returns :want-lookahead.
  (pcase (access-keymap keymap candidate-keys)
    (':want-lookahead  ; need more input
      :want-lookahead)
    (`(:remap ,translated-candidate-keys)
      ;; just retry the whole matching step with the
      ;; new candidate keys
      (try-match-key-sequence translated-candidate-keys))
    (r r)))

(defun read-key-sequence (...)
  ;; Match the shortest bound sequence.
  ;; Leave any remaining lookahead in
  ;; raw-event-queue to get picked up next time.
  (cl-loop
    with raw-event-queue = ()
    for nr-keys upfrom 0
    for binding = (cl-loop
      for maybe-binding = (try-match-key-sequence
        (make-candidate-keys
          :keys raw-event-queue
          :nr nr-keys))
      while (eq maybe-binding :want-lookahead)
      ;; Yes, I know appendf doesn't exist
      do (appendf raw-event-queue (read-char))
      finally return maybe-binding)
    until binding
    finally return (progn
      ;; Put any unconsumed readahead in the queue for the next
      ;; event.  BINDING here is (translated-keys . command) or
      ;; something like that
      (setf unread-command-events (seq-subseq raw-event-queue nr-keys))
      (setf this-command-keys (car binding))  ; translated
      (cdr binding)))
 

Message #20 received at 76669 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 76669 <at> debbugs.gnu.org
Subject: Re: bug#76669: read_key_sequence discards events without attempting
 remapping
Date: Sun, 02 Mar 2025 13:14:38 -0500

[...]
>> FWIW, I have several times wished the `function-key-map`-style "remap if
>> there's no binding" was applied repeatedly.
[...]
> No --- I meant we'd have the remapping code *do* the dropping.
> Apply the remapping in a loop until we reach a fixed point.

That's what I meant by "applied repeatedly", yes.
`function-key-map` is *defined* as not being applied again after itself,
so we'd need to introduce a new keymap (or equivalent).

> Remapping phases:
>
> 1. [s-down-mouse-1 s-mouse-1]
> 2. [down-mouse-1 mouse-1]
> 3. [mouse-1]

[ Of course, there are other remapping steps and orders possible.  🙁 ]

>> In my wildest dreams, `read_key_sequence` has an `input-fallback-map`
>> which is applied repeatedly, and keymaps can be functions so that they
>> can do things like remap anything that looks like `FOO-mouse-1` to
>> `FOO-mouse-2` (without having to list the hundreds of possible FOO) or
>> remap anything that looks like `down-FOO` to `FOO`.
>> Then `read_key_sequence` doesn't treat shift and `down` specially,
>> instead that can be moved out to `input-fallback-map`.
> Keymap entries can already be functions, and I think the function entry
> would work with the fallback finding t too.

Indeed, we have things which cover various parts (there are also those
`:filter` functions in `menu-item`s), but not well enough to be usable
in practice.

> There's no good way to compose such functions, though --- not without
> just clobbering them with add-function and such.  Child keymap
> fallback entries will shadow those of the parents.

Maybe we only need this flexibility for key remapping and not for normal
keymaps, so we could have a "sequence of remapping thingies" where each
thingy can be a keymap or a function so we don't add any special
functionality to keymaps?

>> But, yeah, that still leaves open the question of whether we should drop
>> `shift` before we drop `down` or the reverse, and whether the
>> `FOO-mouse-1` to `FOO-mouse-2` remapping should happen before or after
>> dropping `down`.  The only "principled" way to solve this problem,
>> AFAICT, would be to consider those remapping and returning all the
>> possibilities (as a set of possible keysequences) and then when we do
>> the corresponding key lookup, we'd have to decide what to do if there's
>> more than one binding for those keysequences.
>
> If we're going to fix this, we might as well move read-key-sequence to
> Lisp, where it's easier to express the right algorithm.

That could be nice, yes.  Tho some of the complexity is not just in the
fact that it's written in C but in the details of the semantics
(especially the various remapping keymaps and their interactions, as
well as their interaction with the decision of when exactly to stop
waiting for more input).

>   (cl-loop
>     with want-lookahead = nil
>     for keymap in all-keymaps
>     for binding = (keymap-try-get-binding keymap candidate-keys)
>     if (eq binding :want-lookahead)
>     do (setf want-lookahead :want-lookahead)
>     else if binding return binding
>     finally return want-lookahead))

I think this can be replaced with just

    (lookup-key all-keymaps candidate-keys)

where `integerp` tells us if we "want-lookahead".


        Stefan

Message #23 received at 76669 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 76669 <at> debbugs.gnu.org
Subject: Re: bug#76669: read_key_sequence discards events without attempting remapping
Date: Sun, 02 Mar 2025 20:24:12 -0500

On March 2, 2025 1:14:38 PM EST, Stefan Monnier <monnier <at> iro.umontreal.ca> wrote:
>[...]
>>> FWIW, I have several times wished the `function-key-map`-style "remap if
>>> there's no binding" was applied repeatedly.
>[...]
>> No --- I meant we'd have the remapping code *do* the dropping.
>> Apply the remapping in a loop until we reach a fixed point.
>
>That's what I meant by "applied repeatedly", yes.
>`function-key-map` is *defined* as not being applied again after itself,
>so we'd need to introduce a new keymap (or equivalent).
>
>> Remapping phases:
>>
>> 1. [s-down-mouse-1 s-mouse-1]
>> 2. [down-mouse-1 mouse-1]
>> 3. [mouse-1]
>
>[ Of course, there are other remapping steps and orders possible.  🙁 ]
>
>>> In my wildest dreams, `read_key_sequence` has an `input-fallback-map`
>>> which is applied repeatedly, and keymaps can be functions so that they
>>> can do things like remap anything that looks like `FOO-mouse-1` to
>>> `FOO-mouse-2` (without having to list the hundreds of possible FOO) or
>>> remap anything that looks like `down-FOO` to `FOO`.
>>> Then `read_key_sequence` doesn't treat shift and `down` specially,
>>> instead that can be moved out to `input-fallback-map`.
>> Keymap entries can already be functions, and I think the function entry
>> would work with the fallback finding t too.
>
>Indeed, we have things which cover various parts (there are also those
>`:filter` functions in `menu-item`s), but not well enough to be usable
>in practice.
>
>> There's no good way to compose such functions, though --- not without
>> just clobbering them with add-function and such.  Child keymap
>> fallback entries will shadow those of the parents.
>
>Maybe we only need this flexibility for key remapping and not for normal
>keymaps, so we could have a "sequence of remapping thingies" where each
>thingy can be a keymap or a function so we don't add any special
>functionality to keymaps?
>
>>> But, yeah, that still leaves open the question of whether we should drop
>>> `shift` before we drop `down` or the reverse, and whether the
>>> `FOO-mouse-1` to `FOO-mouse-2` remapping should happen before or after
>>> dropping `down`.  The only "principled" way to solve this problem,
>>> AFAICT, would be to consider those remapping and returning all the
>>> possibilities (as a set of possible keysequences) and then when we do
>>> the corresponding key lookup, we'd have to decide what to do if there's
>>> more than one binding for those keysequences.
>>
>> If we're going to fix this, we might as well move read-key-sequence to
>> Lisp, where it's easier to express the right algorithm.
>
>That could be nice, yes.  Tho some of the complexity is not just in the
>fact that it's written in C but in the details of the semantics
>(especially the various remapping keymaps and their interactions, as
>well as their interaction with the decision of when exactly to stop
>waiting for more input).
>
>>   (cl-loop
>>     with want-lookahead = nil
>>     for keymap in all-keymaps
>>     for binding = (keymap-try-get-binding keymap candidate-keys)
>>     if (eq binding :want-lookahead)
>>     do (setf want-lookahead :want-lookahead)
>>     else if binding return binding
>>     finally return want-lookahead))
>
>I think this can be replaced with just
>
>    (lookup-key all-keymaps candidate-keys)
>
>where `integerp` tells us if we "want-lookahead".

lookup-key wouldn't know to recurse after remapping though.

We'd also have to do something about terminal/keyboard switching and input methods.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.