Subject: bug#61726: closed (Re: bug#61726: [PATCH] Eglot: Support positionEncoding capability)

From: [email-masked] (GNU bug Tracking System)
Date: Mon, 27 Feb 2023 11:14:03 +0000

Your bug report

#61726: [PATCH] Eglot: Support positionEncoding capability

which was filed against the emacs package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
61726: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=61726
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#61726: [PATCH] Eglot: Support positionEncoding capability
From: João Távora <[email-masked]>
To: Eli Zaretskii <[email-masked]>, [email-masked]
Cc: [email-masked]
Date: Mon, 27 Feb 2023 11:15:00 +0000
List-Id: <debbugs-submit.debbugs.gnu.org>

On Sun, Feb 26, 2023 at 3:37 PM Eli Zaretskii <[email-masked]> wrote:
>
> > From: João Távora <[email-masked]>
> > Cc: [email-masked],  [email-masked]
> > Date: Sun, 26 Feb 2023 15:15:57 +0000
> >
> > Eli Zaretskii <[email-masked]> writes:
> >
> >
> > >> But as I said this is just a nit.
> > >
> > > I made one more fix.
> >
> > FTR, I think it's way worse now. The function variable has no clear
> > protocol: it's very odd to state its return type as number of code units
> > _or_ bytes _or_ code points.  A good docstring for such a variable notes
> > that, for any buffer position, a plugged-in function must return the
> > same _type_ but may return a different _value_ to match the
> > unit-counting strategy being used by the LSP server.
>
> It cannot return the same value,. it must return a value in the same
> units as its opposite.

I wrote "return the same _type_ but may return a different _value_".

The doc string says:
>
>   This is the inverse of `eglot-move-to-linepos-function' (which see).
>   It is a function of no arguments returning the number of code units
>   or bytes or codepoints corresponding to the current position of point,
>   relative to line beginning, as expected by the function that is the
>   value of `eglot-move-to-linepos-function'.")
>
> Note the last sentence.

Yes, I understand the intention.  That's a good detail, to state that
when that other function is fed the return value of this one, position
should remain unaltered.  I just did that.

> It's your package, so feel free to make any changes you like.

Done, and closing. We can deal with the `line-beginning-position` thing
afterwards.  It's only tangentially related to this encoding-supporting feature.

Subject: [PATCH] Eglot: Support positionEncoding capability
From: Augusto Stoffel <[email-masked]>
To: [email-masked]
Date: Thu, 23 Feb 2023 09:05:35 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

Tags: patch

There is a new LSP capability allowing the server and client to agree on
a way to count character offsets.  What do you think fo the attached
patch?

It expresses Eglot's preferences as counting character offsets, then
byte offsets, then the UTF-16 nonsense, in that order.

I would also suggest preparing the stage to eventually make
`eglot-current-column-function' and `eglot-move-to-column-function'
obsolete.  For that, I suggest renaming

- eglot-current-column -> eglot--current-column-utf-32
- eglot-lsp-abiding-column -> eglot--current-columns-utf-16
- eglot-move-to-column -> eglot--move-to-columns-utf-32
- eglot-move-to-lsp-abiding-column -> eglot--move-to-columns-utf-16

and then making the old names obsolete aliases of the new names.

(I have tested the utf-32 case superficially, and not at all the utf-8.
So this is a RFC.)

>From a86601f4e80dfbf21a84230433c431375e3012aa Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <[email-masked]>
Date: Thu, 23 Feb 2023 08:55:58 +0100
Subject: [PATCH] * lisp/progmodes/eglot.el: Support positionEncoding
 capability

---
 lisp/progmodes/eglot.el | 65 +++++++++++++++++++++++++++++------------
 1 file changed, 46 insertions(+), 19 deletions(-)

diff --git a/lisp/progmodes/eglot.el b/lisp/progmodes/eglot.el
index b569c03e8c2..0268fbf63a5 100644
--- a/lisp/progmodes/eglot.el
+++ b/lisp/progmodes/eglot.el
@@ -816,6 +816,9 @@ eglot-client-capabilities
                                        `(:valueSet
                                          [,@(mapcar
                                              #'car eglot--tag-faces)])))
+            :general
+            (list
+             :positionEncodings ["utf-32" "utf-8" "utf-16"])
             :experimental eglot--{})))
 
 (cl-defgeneric eglot-workspace-folders (server)
@@ -1439,20 +1442,26 @@ eglot--warn
   (let ((warning-minimum-level :error))
     (display-warning 'eglot (apply #'format format args) :warning)))
 
-(defun eglot-current-column () (- (point) (line-beginning-position)))
+(defun eglot-current-column ()
+  "Calculate current column, counting Unicode codepoints."
+  (- (point) (line-beginning-position)))
+
+(defun eglot--current-column-utf-8 ()
+  "Calculate current column, counting bytes."
+  (- (position-bytes (point)) (position-bytes (line-beginning-position))))
 
-(defvar eglot-current-column-function #'eglot-lsp-abiding-column
+(defvar eglot-current-column-function nil
   "Function to calculate the current column.
 
 This is the inverse operation of
 `eglot-move-to-column-function' (which see).  It is a function of
 no arguments returning a column number.  For buffers managed by
-fully LSP-compliant servers, this should be set to
-`eglot-lsp-abiding-column' (the default), and
-`eglot-current-column' for all others.")
+fully LSP-compliant servers, this should be nil.  For others, it
+can be set to `eglot-current-colum' or
+`eglot--current-column-utf-8'.")
 
 (defun eglot-lsp-abiding-column (&optional lbp)
-  "Calculate current COLUMN as defined by the LSP spec.
+  "Calculate current column, counting UTF-16 code units as in the original LSP spec.
 LBP defaults to `line-beginning-position'."
   (/ (- (length (encode-coding-region (or lbp (line-beginning-position))
                                       ;; Fix github#860
@@ -1462,13 +1471,19 @@ eglot-lsp-abiding-column
 
 (defun eglot--pos-to-lsp-position (&optional pos)
   "Convert point POS to LSP position."
-  (eglot--widening
-   ;; LSP line is zero-origin; emacs is one-origin.
-   (list :line (1- (line-number-at-pos pos t))
-         :character (progn (when pos (goto-char pos))
-                           (funcall eglot-current-column-function)))))
-
-(defvar eglot-move-to-column-function #'eglot-move-to-lsp-abiding-column
+  (let ((columnfn (or eglot-current-column-function
+                      (pcase (plist-get (eglot--capabilities (eglot-current-server))
+                                        :positionEncoding)
+                        ("utf-32" #'eglot-current-column)
+                        ("utf-8" #'eglot--current-column-utf-8)
+                        (_ #'eglot-lsp-abiding-column)))))
+    (eglot--widening
+     ;; LSP line is zero-origin; emacs is one-origin.
+     (list :line (1- (line-number-at-pos pos t))
+           :character (progn (when pos (goto-char pos))
+                             (funcall columnfn))))))
+
+(defvar eglot-move-to-column-function nil
   "Function to move to a column reported by the LSP server.
 
 According to the standard, LSP column/character offsets are based
@@ -1478,11 +1493,11 @@ eglot-move-to-column-function
 `c'. However, many servers don't follow the spec this closely.
 
 For buffers managed by fully LSP-compliant servers, this should
-be set to `eglot-move-to-lsp-abiding-column' (the default), and
-`eglot-move-to-column' for all others.")
+be letft nil.  For others, it can be set to
+`eglot-move-to-column' or `eglot--move-to-column-utf-8'.")
 
 (defun eglot-move-to-column (column)
-  "Move to COLUMN without closely following the LSP spec."
+  "Move to COLUMN, counting Unicode codepoints."
   ;; We cannot use `move-to-column' here, because it moves to *visual*
   ;; columns, which can be different from LSP columns in case of
   ;; `whitespace-mode', `prettify-symbols-mode', etc.  (github#296,
@@ -1490,8 +1505,14 @@ eglot-move-to-column
   (goto-char (min (+ (line-beginning-position) column)
                   (line-end-position))))
 
+(defun eglot--move-to-column-utf-8 (column)
+  "Move to COLUMN, regarded as a byte offset."
+  (goto-char (min (byte-to-position
+                   (+ (position-bytes (line-beginning-position)) column))
+                  (line-end-position))))
+
 (defun eglot-move-to-lsp-abiding-column (column)
-  "Move to COLUMN abiding by the LSP spec."
+  "Move to COLUMN, counting UTF-16 code units as in the original LSP spec."
   (save-restriction
     (cl-loop
      with lbp = (line-beginning-position)
@@ -1515,14 +1536,20 @@ eglot--lsp-position-to-point
       (forward-line (min most-positive-fixnum
                          (plist-get pos-plist :line)))
       (unless (eobp) ;; if line was excessive leave point at eob
-        (let ((tab-width 1)
+        (let ((movefn (or eglot-move-to-column-function
+                          (pcase (plist-get (eglot--capabilities (eglot-current-server))
+                                            :positionEncoding)
+                            ("utf-32" #'eglot-move-to-column)
+                            ("utf-8" #'eglot--move-to-column-utf-8)
+                            (_ #'eglot-move-to-lsp-abiding-column))))
+              (tab-width 1)
               (col (plist-get pos-plist :character)))
           (unless (wholenump col)
             (eglot--warn
              "Caution: LSP server sent invalid character position %s. Using 0 instead."
              col)
             (setq col 0))
-          (funcall eglot-move-to-column-function col)))
+          (funcall movefn col)))
       (if marker (copy-marker (point-marker)) (point)))))
 
 (defconst eglot--uri-path-allowed-chars
-- 
2.39.2

PS: João, since you may have had trouble receiving some emails, did you
notice bug#58141?