Subject: bug#77754: 31.0.50; completion-table-with-quoting and completion-lazy-hilit don't work together

From: Spencer Baugh <[email-masked]>
Date: Mon, 14 Apr 2025 13:41:30 -0400

Spencer Baugh <[email-masked]> writes:
> 1. touch 'foo bxx' 'foo byy'
> 2. emacs -Q
> 3. M-x shell
> 4. Type "echo foo" and hit TAB; it will complete to "echo foo\ b"
> 5. Hit ? to bring up *Completions*
> 6. Note that the highlighting is incorrect; the next character should
>    be highlighted in pink as the next difference, but it's not.
>
> If you modify minibuffer-completion-help to not bind
> completion-lazy-hilit to t (which I recently added), the highlighting
> will be correct.
>
> This is because completion-lazy-hilit=t makes the highlighting function
> run on the *quoted* string (with the \), whereas
> completion-lazy-hilit=nil runs highlighting on the *unquoted* string,
> which is then highlighted correctly.
>
> IMO, we should fix this by changing the API exposed for
> completion-table-with-quoting.  As mentioned in a comment in
> completion--twq-all:
>
>    ;; The better solution is to not quote the *Completions* display,
>    ;; which nicely circumvents the problem.
>
> If we just stopped quoting *Completions*, we would avoid this problem.
>
> This would necessitate adding some new way to quote the completion
> before inserting it.  We could add some new piece of completion metadata
> which provides a function which gets called when a completion candidate
> is about to be inserted, and returns a string that should be inserted
> instead - it can perform the quoting.  (Or it could even be a new
> completion table action?)

Fortunately it actually turns out to be much, much simpler than this.
We continue to return the quoted completions from
completion-all-completions, and just use completion--unquoted for
display instead of the quoted completion.

Completion UIs should start calling the new completion-for-display
function, but even if they don't, nothing new breaks.

>From a42a44b996096a8c80bc972089b0e6e1f0bbd1f8 Mon Sep 17 00:00:00 2001
From: Spencer Baugh <[email-masked]>
Date: Mon, 14 Apr 2025 13:37:11 -0400
Subject: [PATCH] Display unquoted candidates in *Completions*

To fix bug#77754 and generally improve *Completions* behavior,
start displaying the unquoted versions of completion candidates
in *Completions* and in icomplete.

External completion UIs can call completion-for-display to
obtain the same behavior (and fix similar bugs in their UI).

* lisp/minibuffer.el (completion-for-display): Add.
(completion--twq-all): Add FIXME to simplify this function.
(completion--insert): Call completion-for-display. (bug#77754)
* lisp/icomplete.el (icomplete--render-vertical)
(icomplete-completions): Call completion-for-display.
---
 lisp/icomplete.el  |  4 ++--
 lisp/minibuffer.el | 10 +++++++++-
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/lisp/icomplete.el b/lisp/icomplete.el
index 35842b53e6b..bc6a8f82ab3 100644
--- a/lisp/icomplete.el
+++ b/lisp/icomplete.el
@@ -1024,7 +1024,7 @@ icomplete--render-vertical
                                 'icomplete-selected-match 'append comp)
      collect (concat prefix
                      (make-string (max 0 (- max-prefix-len (length prefix))) ? )
-                     (completion-lazy-hilit comp)
+                     (completion-for-display comp)
                      (make-string (max 0 (- max-comp-len (length comp))) ? )
                      suffix)
      into lines-aux
@@ -1188,7 +1188,7 @@ icomplete-completions
                   (setq determ (concat open-bracket "" close-bracket)))
                 (while (and comps (not limit))
                   (setq comp
-                        (let ((cur (completion-lazy-hilit (car comps))))
+                        (let ((cur (completion-for-display (car comps))))
                           (if prefix-len (substring cur prefix-len) cur))
                         comps (cdr comps))
                   (setq prospects-len
diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 09a91534322..2bb5b551430 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -696,6 +696,8 @@ completion--twq-all
       ;; would be to return unquoted elements together with a function to
       ;; requote them, so that *Completions* can show nicer unquoted values
       ;; which only get quoted when needed by choose-completion.
+      ;; FIXME: *Completions* now shows unquoted values by using
+      ;; completion--unquoted, so this function can be greatly simplified.
       (nconc
        (mapcar (lambda (completion)
                  (cl-assert (string-prefix-p prefix completion 'ignore-case) t)
@@ -2432,7 +2434,7 @@ completion--insert
   (if (not (consp str))
       (add-text-properties
        (point)
-       (let ((str (completion-lazy-hilit str)))
+       (let ((str (completion-for-display str)))
          (insert
           (if group-fun
               (funcall group-fun str 'transform)
@@ -4322,6 +4324,12 @@ completion-lazy-hilit
       (funcall completion-lazy-hilit-fn (copy-sequence str))
     str))
 
+(defun completion-for-display (str)
+  "Return the string that should be displayed for completion candidate STR.
+
+This will be `face'-propertized as appropriate."
+  (completion-lazy-hilit (or (get-text-property 0 'completion--unquoted str) str)))
+
 (defun completion--hilit-from-re (string regexp &optional point-idx)
   "Fontify STRING using REGEXP POINT-IDX.
 Uses `completions-common-part' and `completions-first-difference'
-- 
2.39.3