From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 11 17:47:25 2025 Received: (at submit) by debbugs.gnu.org; 11 Apr 2025 21:47:26 +0000 Received: from localhost ([127.0.0.1]:52053 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u3MDd-00037c-B1 for submit@debbugs.gnu.org; Fri, 11 Apr 2025 17:47:25 -0400 Received: from lists.gnu.org ([2001:470:142::17]:56394) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u3MDZ-00037G-QD for submit@debbugs.gnu.org; Fri, 11 Apr 2025 17:47:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u3MDT-0004PY-Ad for bug-gnu-emacs@gnu.org; Fri, 11 Apr 2025 17:47:15 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u3MDQ-0000xI-6N for bug-gnu-emacs@gnu.org; Fri, 11 Apr 2025 17:47:15 -0400 From: Spencer Baugh To: bug-gnu-emacs@gnu.org Subject: 31.0.50; completion-table-with-quoting and completion-lazy-hilit don't work together X-Debbugs-Cc: Stefan Monnier , Dmitry Gutov Date: Fri, 11 Apr 2025 17:47:09 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=waixah; t=1744408029; bh=r1wxffBCzvM9cLtREdvsEL0lhtIH4DXd6YLPAsDgLas=; h=From:To:Subject:Date; b=hCehWUi3fyCrImLaHAbiLvBWzKdtXgL84q0vR7n3nt0Y11i/EpMd4Bd6e984I0v+8 3glwX+4ZKFnU6k3WrTk21wJzGnrW/EKQIHFT6SSCU8PGMdAJnf89O8d79eySPt7XRk ok60UVA6BINImsniISY/ysI8cynDCL0KyTMw9Ml7KFusXO4n9c1mUZZm/+oNIiauyd Tcw11pGCQggbIjsc4FeK5e889VGbwA3AvI3v1weSQEEyT/XZ8T3gMukNESrxpWTANk uSXwkGVlPwTtjTJDWJH/NVraiU/vN4qKwxtk6wDlugB211s1cdddfHrcF3Fheiwd92 InS0P4vWXF4eA== Received-SPF: pass client-ip=64.215.233.18; envelope-from=sbaugh@janestreet.com; helo=mxout5.mail.janestreet.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) 1. touch 'foo bxx' 'foo byy' 2. emacs -Q 3. M-x shell 4. Type "echo foo" and hit TAB; it will complete to "echo foo\ b" 5. Hit ? to bring up *Completions* 6. Note that the highlighting is incorrect; the next character should be highlighted in pink as the next difference, but it's not. If you modify minibuffer-completion-help to not bind completion-lazy-hilit to t (which I recently added), the highlighting will be correct. This is because completion-lazy-hilit=t makes the highlighting function run on the *quoted* string (with the \), whereas completion-lazy-hilit=nil runs highlighting on the *unquoted* string, which is then highlighted correctly. IMO, we should fix this by changing the API exposed for completion-table-with-quoting. As mentioned in a comment in completion--twq-all: ;; The better solution is to not quote the *Completions* display, ;; which nicely circumvents the problem. If we just stopped quoting *Completions*, we would avoid this problem. This would necessitate adding some new way to quote the completion before inserting it. We could add some new piece of completion metadata which provides a function which gets called when a completion candidate is about to be inserted, and returns a string that should be inserted instead - it can perform the quoting. (Or it could even be a new completion table action?) I'm happy to make that change if that sounds reasonable to others. (While adding this, I'd also like to make this new function return a cons whose cdr is the new location of point after inserting the candidate - that would be useful for various other fixes, including making the emacs22 style behave better) In GNU Emacs 31.0.50 (build 9, x86_64-pc-linux-gnu, GTK+ Version 3.22.30, cairo version 1.15.12) of 2025-04-08 built on igm-qws-u22796a Repository revision: fb4ae239053b5e03ade2af28573fe6608458a8b7 Repository branch: HEAD Windowing system distributor 'The X.Org Foundation', version 11.0.12011000 System Description: Rocky Linux 8.10 (Green Obsidian) Configured using: 'configure -C --with-gif=no' Configured features: CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XINERAMA XINPUT2 XPM XRANDR GTK3 ZLIB Important settings: value of $LANG: en_US.UTF-8 locale-coding-system: utf-8-unix Major mode: Eshell Minor modes in effect: eshell-prompt-mode: t eshell-pred-mode: t eshell-hist-mode: t eshell-cmpl-mode: t eshell-proc-mode: t eshell-arg-mode: t tooltip-mode: t global-eldoc-mode: t eldoc-mode: t show-paren-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t minibuffer-regexp-mode: t line-number-mode: t indent-tabs-mode: t transient-mark-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug lisp-mnt message mailcap yank-media puny dired dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068 epg-config gnus-util time-date mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils pcmpl-unix cl-extra help-mode em-unix em-term term disp-table ehelp em-script em-prompt em-pred em-ls em-hist em-glob em-extpipe em-cmpl em-dirs em-basic em-banner em-alias esh-mode esh-var eshell esh-cmd generator esh-ext esh-proc esh-opt esh-io esh-arg esh-module esh-module-loaddefs esh-util pp cl-seq project byte-opt gv sh-script smie treesit executable compile text-property-search files-x shell pcomplete comint subr-x ansi-osc ansi-color ring comp-run bytecomp byte-compile comp-common rx warnings icons cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic indonesian philippine cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp files window text-properties overlay sha1 md5 base64 format env code-pages mule custom widget keymap hashtable-print-readable backquote threads dbusbind inotify dynamic-setting system-font-setting font-render-setting cairo gtk x-toolkit xinput2 x multi-tty move-toolbar make-network-process tty-child-frames native-compile emacs) Memory information: ((conses 16 136537 11841) (symbols 48 9968 0) (strings 32 57881 2879) (string-bytes 1 2842600) (vectors 16 25782) (vector-slots 8 321965 10866) (floats 8 62 66) (intervals 56 493 7) (buffers 984 16)) From debbugs-submit-bounces@debbugs.gnu.org Mon Apr 14 13:42:25 2025 Received: (at 77754) by debbugs.gnu.org; 14 Apr 2025 17:42:25 +0000 Received: from localhost ([127.0.0.1]:48840 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u4NpA-0000Oy-SG for submit@debbugs.gnu.org; Mon, 14 Apr 2025 13:42:25 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:48663) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u4NoN-0000N9-Qk for 77754@debbugs.gnu.org; Mon, 14 Apr 2025 13:41:36 -0400 From: Spencer Baugh To: 77754@debbugs.gnu.org Subject: Re: bug#77754: 31.0.50; completion-table-with-quoting and completion-lazy-hilit don't work together In-Reply-To: (Spencer Baugh's message of "Fri, 11 Apr 2025 17:47:09 -0400") References: Date: Mon, 14 Apr 2025 13:41:30 -0400 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=waixah; t=1744652490; bh=abTLpvgRM4re21QuTM1eYuieyKEYZPt3j0Kpv/5FDTg=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=VHWhf8fN8zijVVSjQY1d4m+1YTIJ7mwfDxwGxkxMAkLM7qkMalqytxSnBDa6PcBXK 852KXo66467YW1FqKF3pUHL40H0rw3hPxkQKLYTZIRxVmkX90nyu7xPnhJhzDgqjZN Rdm4+f9b4tTh4g8+72I/iYlO04Q/mCVhSsRodaKvwTi6BqCZ0+g5cmafIfn4WcT3Vl tJJT/CeErdSqGjE0k+j3NvTflXVgTRhmm2b+cuEriCIWd524+NwMfFxCHFpmqDYg7o cepjGoZBG1wTKK5Kk1lA1hKO698GYWksOXpptX20BP406+ZVZ6MJChyZzExcKnuiT5 zccyaTlm7/xcQ== X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 77754 Cc: Dmitry Gutov , Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain Spencer Baugh writes: > 1. touch 'foo bxx' 'foo byy' > 2. emacs -Q > 3. M-x shell > 4. Type "echo foo" and hit TAB; it will complete to "echo foo\ b" > 5. Hit ? to bring up *Completions* > 6. Note that the highlighting is incorrect; the next character should > be highlighted in pink as the next difference, but it's not. > > If you modify minibuffer-completion-help to not bind > completion-lazy-hilit to t (which I recently added), the highlighting > will be correct. > > This is because completion-lazy-hilit=t makes the highlighting function > run on the *quoted* string (with the \), whereas > completion-lazy-hilit=nil runs highlighting on the *unquoted* string, > which is then highlighted correctly. > > IMO, we should fix this by changing the API exposed for > completion-table-with-quoting. As mentioned in a comment in > completion--twq-all: > > ;; The better solution is to not quote the *Completions* display, > ;; which nicely circumvents the problem. > > If we just stopped quoting *Completions*, we would avoid this problem. > > This would necessitate adding some new way to quote the completion > before inserting it. We could add some new piece of completion metadata > which provides a function which gets called when a completion candidate > is about to be inserted, and returns a string that should be inserted > instead - it can perform the quoting. (Or it could even be a new > completion table action?) Fortunately it actually turns out to be much, much simpler than this. We continue to return the quoted completions from completion-all-completions, and just use completion--unquoted for display instead of the quoted completion. Completion UIs should start calling the new completion-for-display function, but even if they don't, nothing new breaks. --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-Display-unquoted-candidates-in-Completions.patch >From a42a44b996096a8c80bc972089b0e6e1f0bbd1f8 Mon Sep 17 00:00:00 2001 From: Spencer Baugh Date: Mon, 14 Apr 2025 13:37:11 -0400 Subject: [PATCH] Display unquoted candidates in *Completions* To fix bug#77754 and generally improve *Completions* behavior, start displaying the unquoted versions of completion candidates in *Completions* and in icomplete. External completion UIs can call completion-for-display to obtain the same behavior (and fix similar bugs in their UI). * lisp/minibuffer.el (completion-for-display): Add. (completion--twq-all): Add FIXME to simplify this function. (completion--insert): Call completion-for-display. (bug#77754) * lisp/icomplete.el (icomplete--render-vertical) (icomplete-completions): Call completion-for-display. --- lisp/icomplete.el | 4 ++-- lisp/minibuffer.el | 10 +++++++++- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/lisp/icomplete.el b/lisp/icomplete.el index 35842b53e6b..bc6a8f82ab3 100644 --- a/lisp/icomplete.el +++ b/lisp/icomplete.el @@ -1024,7 +1024,7 @@ icomplete--render-vertical 'icomplete-selected-match 'append comp) collect (concat prefix (make-string (max 0 (- max-prefix-len (length prefix))) ? ) - (completion-lazy-hilit comp) + (completion-for-display comp) (make-string (max 0 (- max-comp-len (length comp))) ? ) suffix) into lines-aux @@ -1188,7 +1188,7 @@ icomplete-completions (setq determ (concat open-bracket "" close-bracket))) (while (and comps (not limit)) (setq comp - (let ((cur (completion-lazy-hilit (car comps)))) + (let ((cur (completion-for-display (car comps)))) (if prefix-len (substring cur prefix-len) cur)) comps (cdr comps)) (setq prospects-len diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el index 09a91534322..2bb5b551430 100644 --- a/lisp/minibuffer.el +++ b/lisp/minibuffer.el @@ -696,6 +696,8 @@ completion--twq-all ;; would be to return unquoted elements together with a function to ;; requote them, so that *Completions* can show nicer unquoted values ;; which only get quoted when needed by choose-completion. + ;; FIXME: *Completions* now shows unquoted values by using + ;; completion--unquoted, so this function can be greatly simplified. (nconc (mapcar (lambda (completion) (cl-assert (string-prefix-p prefix completion 'ignore-case) t) @@ -2432,7 +2434,7 @@ completion--insert (if (not (consp str)) (add-text-properties (point) - (let ((str (completion-lazy-hilit str))) + (let ((str (completion-for-display str))) (insert (if group-fun (funcall group-fun str 'transform) @@ -4322,6 +4324,12 @@ completion-lazy-hilit (funcall completion-lazy-hilit-fn (copy-sequence str)) str)) +(defun completion-for-display (str) + "Return the string that should be displayed for completion candidate STR. + +This will be `face'-propertized as appropriate." + (completion-lazy-hilit (or (get-text-property 0 'completion--unquoted str) str))) + (defun completion--hilit-from-re (string regexp &optional point-idx) "Fontify STRING using REGEXP POINT-IDX. Uses `completions-common-part' and `completions-first-difference' -- 2.39.3 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Apr 28 13:53:05 2025 Received: (at 77754) by debbugs.gnu.org; 28 Apr 2025 17:53:06 +0000 Received: from localhost ([127.0.0.1]:34369 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u9Sf9-0004rG-U9 for submit@debbugs.gnu.org; Mon, 28 Apr 2025 13:53:05 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:59247) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u9Sf6-0004q6-9S for 77754@debbugs.gnu.org; Mon, 28 Apr 2025 13:53:01 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id D36FF80911; Mon, 28 Apr 2025 13:52:54 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1745862774; bh=KPhUfCLtzR4KJc9pOiCqzr/6ouUg+vuuyRDXO6+d2Bs=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=RhhDGhl6g2rPd3ApvGdFqjMDLnUgFJ4CEOinRg+hiKND7GFDLBCZw25AiuOIB27xF xw6cWodgPxBlUREifhHem7kGJ1tMVfXpCTfsR7rN5ft6ZyDpzu9bkFUqcTqBp+wfYw d6HjwItVfYChVwFuPHeOhMpDuCkAHq/TKMwj5Mj5Ht8RkhGum7gEomUU1cjkyykaId Z0PAV+dZvcWZdjCY5mJYoc8hpGWFG9qdd+xALbHnIdA7BvU7is/1pYUMdoGPBuZWlj ZMn0zmM8W7AB4FldWwFvTb9IA+ec700LpNB+ddtE2zi6X1fx4lZuWfOQ6DLB/3kukY LqNLoDkaGDnwA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 0BB9B806BF; Mon, 28 Apr 2025 13:52:54 -0400 (EDT) Received: from pastel (104-195-232-56.cpe.teksavvy.com [104.195.232.56]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id D01E51202BB; Mon, 28 Apr 2025 13:52:53 -0400 (EDT) From: Stefan Monnier To: Spencer Baugh Subject: Re: bug#77754: 31.0.50; completion-table-with-quoting and completion-lazy-hilit don't work together In-Reply-To: Message-ID: References: Date: Mon, 28 Apr 2025 13:52:53 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.063 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 77754 Cc: 77754@debbugs.gnu.org, Dmitry Gutov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Fortunately it actually turns out to be much, much simpler than this. > We continue to return the quoted completions from > completion-all-completions, and just use completion--unquoted for > display instead of the quoted completion. Hmm... my first reaction is that it's an ugly hack, but it's growing on me. How does it interaction `choose-completion`? I expected a need for a corresponding change there but can't see anything in the patch. Stefan