GNU bug report logs - #68217
30.0.50; Fix for eglot--format-markup to handle bad markdown

Previous Next

To reply to this bug, email your comments to 68217 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Gary Oberbrunner <garyo <at> oberbrunner.com>
To: bug-gnu-emacs <at> gnu.org
Cc: jblevins <at> xbeta.org, João Távora <joaotavora <at> gmail.com>
Subject: 30.0.50; Fix for eglot--format-markup to handle bad markdown
Date: Tue, 2 Jan 2024 13:28:16 -0500

[Message part 1 (text/plain, inline)]

With most recent eglot and emacs (as of a couple of days ago) I'm
getting an error in my python-mode buffers. While eglot is trying to
show hover info, it gets a treesit error.

I'm using eldoc and treesit a bunch of other stuff so I'm not sure where
the real bug is. But basically it looks like treesit is trying to
fontify a temp python-mode buffer that just contains

  (class) openfx

and getting an error. Markdown mode is involved too (see the stack
trace). I think the simplest solution is to ignore errors while
fontifying the temp buffer, and I believe joaotavora agrees.

The markup arg passed to eglot--format-markup is:

  (:kind "markdown" :value "```python\n(class) openfx\n```")

I'm using pyright-langserver as the LSP server; I expect it is
returning malformed markup or something that eglot doesn't expect.

I note that the following patch appears to fix it for me:

modified   eglot.el
@@ -1813,7 +1813,9 @@ Doubles as an indicator of snippet support."
             (message-log-max nil)
             match)
         (ignore-errors (delay-mode-hooks (funcall mode)))
-        (font-lock-ensure)
+        (condition-case-unless-debug oops
+            (font-lock-ensure)
+          (error (eglot--warn (error-message-string oops))))
         (goto-char (point-min))
         (let ((inhibit-read-only t))
           (when (fboundp 'text-property-search-forward) ;; FIXME: use
compat

See https://github.com/joaotavora/eglot/discussions/1345 for more details.

The stack trace looks like this:

Debugger entered--Lisp error: (treesit-query-error "Node type error at" 271
"[\"as\" \"assert\" \"async\" \"await\" \"break\" \"case\" \"class\"
\"continue\" \"def\" \"del\" \"elif\" \"else\" \"except\" \"exec\"
\"finally\" \"for\" \"from\" \"global\" \"if\" \"import\" \"lambda\"
\"match\" \"nonlocal\" \"pass\" \"print\" \"raise\" \"return\" \"try\"
\"while\" \"with\" \"yield\" \"and\" \"in\" \"is\" \"not\" \"or\" \"not
in\"] @font-lock-keyword-face ((identifier) @font-lock-keyword-face (#match
\"\\\\`self\\\\'\" @font-lock-keyword-face))" "Debug the query with
`treesit-query-validate'")
  treesit--font-lock-fontify-region-1(#<treesit-node module in 1-17>
#<treesit-compiled-query> 1 17 nil nil)
  treesit-font-lock-fontify-region(1 17 nil)
  font-lock-fontify-syntactically-region(1 17 nil)
  font-lock-default-fontify-region(1 17 nil)
  font-lock-fontify-region(1 17)
  #f(compiled-function (beg end) #<bytecode -0x147d5368d5be984b>)(1 17)
  font-lock-ensure()
  (save-current-buffer (set-buffer (get-buffer-create (concat "
markdown-code-fontification:" (symbol-name lang-mode)))) (let
((inhibit-modification-hooks nil)) (delete-region (point-min) (point-max))
(insert string " ")) (if (eq major-mode lang-mode) nil (funcall lang-mode))
(font-lock-ensure) (setq pos (point-min)) (while (setq next
(next-single-property-change pos 'face)) (let ((val (get-text-property pos
'face))) (if val (progn (put-text-property (+ start (1- pos)) (1- (+ start
next)) 'face val markdown-buffer)))) (setq pos next)))
  (let ((string (buffer-substring-no-properties start end)) (modified
(buffer-modified-p)) (markdown-buffer (current-buffer)) pos next)
(remove-text-properties start end '(face nil)) (save-current-buffer
(set-buffer (get-buffer-create (concat " markdown-code-fontification:"
(symbol-name lang-mode)))) (let ((inhibit-modification-hooks nil))
(delete-region (point-min) (point-max)) (insert string " ")) (if (eq
major-mode lang-mode) nil (funcall lang-mode)) (font-lock-ensure) (setq pos
(point-min)) (while (setq next (next-single-property-change pos 'face))
(let ((val (get-text-property pos 'face))) (if val (progn
(put-text-property (+ start ...) (1- ...) 'face val markdown-buffer))))
(setq pos next))) (add-text-properties start end '(font-lock-fontified t
fontified t font-lock-multiline t)) (set-buffer-modified-p modified))
  (progn (let ((string (buffer-substring-no-properties start end))
(modified (buffer-modified-p)) (markdown-buffer (current-buffer)) pos next)
(remove-text-properties start end '(face nil)) (save-current-buffer
(set-buffer (get-buffer-create (concat " markdown-code-fontification:"
(symbol-name lang-mode)))) (let ((inhibit-modification-hooks nil))
(delete-region (point-min) (point-max)) (insert string " ")) (if (eq
major-mode lang-mode) nil (funcall lang-mode)) (font-lock-ensure) (setq pos
(point-min)) (while (setq next (next-single-property-change pos 'face))
(let ((val (get-text-property pos ...))) (if val (progn (put-text-property
... ... ... val markdown-buffer)))) (setq pos next))) (add-text-properties
start end '(font-lock-fontified t fontified t font-lock-multiline t))
(set-buffer-modified-p modified)))
  (if (fboundp lang-mode) (progn (let ((string
(buffer-substring-no-properties start end)) (modified (buffer-modified-p))
(markdown-buffer (current-buffer)) pos next) (remove-text-properties start
end '(face nil)) (save-current-buffer (set-buffer (get-buffer-create
(concat " markdown-code-fontification:" (symbol-name lang-mode)))) (let
((inhibit-modification-hooks nil)) (delete-region (point-min) (point-max))
(insert string " ")) (if (eq major-mode lang-mode) nil (funcall lang-mode))
(font-lock-ensure) (setq pos (point-min)) (while (setq next
(next-single-property-change pos 'face)) (let ((val ...)) (if val (progn
...))) (setq pos next))) (add-text-properties start end
'(font-lock-fontified t fontified t font-lock-multiline t))
(set-buffer-modified-p modified))))
  (let ((lang-mode (if lang (markdown-get-lang-mode lang)
markdown-fontify-code-block-default-mode))) (if (fboundp lang-mode) (progn
(let ((string (buffer-substring-no-properties start end)) (modified
(buffer-modified-p)) (markdown-buffer (current-buffer)) pos next)
(remove-text-properties start end '(face nil)) (save-current-buffer
(set-buffer (get-buffer-create (concat " markdown-code-fontification:"
...))) (let ((inhibit-modification-hooks nil)) (delete-region (point-min)
(point-max)) (insert string " ")) (if (eq major-mode lang-mode) nil
(funcall lang-mode)) (font-lock-ensure) (setq pos (point-min)) (while (setq
next (next-single-property-change pos ...)) (let (...) (if val ...)) (setq
pos next))) (add-text-properties start end '(font-lock-fontified t
fontified t font-lock-multiline t)) (set-buffer-modified-p modified)))))
  markdown-fontify-code-block-natively("python" 11 26)
  (if (and markdown-fontify-code-blocks-natively (or (setq lang
(markdown-code-block-lang)) markdown-fontify-code-block-default-mode))
(markdown-fontify-code-block-natively lang start end) (add-text-properties
start end '(face markdown-pre-face)))
  (let* ((start (match-beginning 0)) (end (match-end 0)) (bol-prev (progn
(goto-char start) (if (bolp) (line-beginning-position 0)
(line-beginning-position)))) (eol-next (progn (goto-char end) (if (bolp)
(line-beginning-position 2) (line-beginning-position 3)))) lang) (if (and
markdown-fontify-code-blocks-natively (or (setq lang
(markdown-code-block-lang)) markdown-fontify-code-block-default-mode))
(markdown-fontify-code-block-natively lang start end) (add-text-properties
start end '(face markdown-pre-face))) (font-lock-append-text-property
bol-prev eol-next 'face 'markdown-code-face) (add-text-properties bol-prev
start '(invisible markdown-markup)) (add-text-properties end eol-next
'(invisible markdown-markup)))
  (progn (let* ((start (match-beginning 0)) (end (match-end 0)) (bol-prev
(progn (goto-char start) (if (bolp) (line-beginning-position 0)
(line-beginning-position)))) (eol-next (progn (goto-char end) (if (bolp)
(line-beginning-position 2) (line-beginning-position 3)))) lang) (if (and
markdown-fontify-code-blocks-natively (or (setq lang
(markdown-code-block-lang)) markdown-fontify-code-block-default-mode))
(markdown-fontify-code-block-natively lang start end) (add-text-properties
start end '(face markdown-pre-face))) (font-lock-append-text-property
bol-prev eol-next 'face 'markdown-code-face) (add-text-properties bol-prev
start '(invisible markdown-markup)) (add-text-properties end eol-next
'(invisible markdown-markup))))
  (unwind-protect (progn (let* ((start (match-beginning 0)) (end (match-end
0)) (bol-prev (progn (goto-char start) (if (bolp) (line-beginning-position
0) (line-beginning-position)))) (eol-next (progn (goto-char end) (if (bolp)
(line-beginning-position 2) (line-beginning-position 3)))) lang) (if (and
markdown-fontify-code-blocks-natively (or (setq lang
(markdown-code-block-lang)) markdown-fontify-code-block-default-mode))
(markdown-fontify-code-block-natively lang start end) (add-text-properties
start end '(face markdown-pre-face))) (font-lock-append-text-property
bol-prev eol-next 'face 'markdown-code-face) (add-text-properties bol-prev
start '(invisible markdown-markup)) (add-text-properties end eol-next
'(invisible markdown-markup)))) (set-match-data saved-match-data t))
  (let ((saved-match-data (match-data))) (unwind-protect (progn (let*
((start (match-beginning 0)) (end (match-end 0)) (bol-prev (progn
(goto-char start) (if ... ... ...))) (eol-next (progn (goto-char end) (if
... ... ...))) lang) (if (and markdown-fontify-code-blocks-natively (or
(setq lang ...) markdown-fontify-code-block-default-mode))
(markdown-fontify-code-block-natively lang start end) (add-text-properties
start end '(face markdown-pre-face))) (font-lock-append-text-property
bol-prev eol-next 'face 'markdown-code-face) (add-text-properties bol-prev
start '(invisible markdown-markup)) (add-text-properties end eol-next
'(invisible markdown-markup)))) (set-match-data saved-match-data t)))
  (save-excursion (let ((saved-match-data (match-data))) (unwind-protect
(progn (let* ((start (match-beginning 0)) (end (match-end 0)) (bol-prev
(progn ... ...)) (eol-next (progn ... ...)) lang) (if (and
markdown-fontify-code-blocks-natively (or ...
markdown-fontify-code-block-default-mode))
(markdown-fontify-code-block-natively lang start end) (add-text-properties
start end '...)) (font-lock-append-text-property bol-prev eol-next 'face
'markdown-code-face) (add-text-properties bol-prev start '(invisible
markdown-markup)) (add-text-properties end eol-next '(invisible
markdown-markup)))) (set-match-data saved-match-data t))))
  (progn (save-excursion (let ((saved-match-data (match-data)))
(unwind-protect (progn (let* ((start ...) (end ...) (bol-prev ...)
(eol-next ...) lang) (if (and markdown-fontify-code-blocks-natively ...)
(markdown-fontify-code-block-natively lang start end) (add-text-properties
start end ...)) (font-lock-append-text-property bol-prev eol-next 'face
'markdown-code-face) (add-text-properties bol-prev start '...)
(add-text-properties end eol-next '...))) (set-match-data saved-match-data
t)))) t)
  (if (funcall matcher last) (progn (save-excursion (let ((saved-match-data
(match-data))) (unwind-protect (progn (let* (... ... ... ... lang) (if ...
... ...) (font-lock-append-text-property bol-prev eol-next ... ...)
(add-text-properties bol-prev start ...) (add-text-properties end eol-next
...))) (set-match-data saved-match-data t)))) t))
  markdown-fontify-code-blocks-generic(markdown-match-gfm-code-blocks 29)
  markdown-fontify-gfm-code-blocks(29)
  font-lock-fontify-keywords-region(1 29 nil)
  font-lock-default-fontify-region(1 29 nil)
  font-lock-fontify-region(1 29)
  #f(compiled-function (beg end) #<bytecode -0x147d5368d5be984b>)(1 29)
  font-lock-ensure()
  (let ((inhibit-message t) (message-log-max nil) match) (condition-case
nil (progn (progn (make-local-variable 'delay-mode-hooks) (let
((delay-mode-hooks t)) (funcall mode)))) (error nil)) (font-lock-ensure)
(goto-char (point-min)) (let ((inhibit-read-only t)) (if (fboundp
'text-property-search-forward) (progn (while (setq match
(text-property-search-forward 'invisible)) (delete-region (let* (...)
(progn ... ...)) (let* (...) (progn ... ...))))))) (string-trim
(buffer-string)))
  (progn (set (make-local-variable 'markdown-fontify-code-blocks-natively)
t) (insert string) (let ((inhibit-message t) (message-log-max nil) match)
(condition-case nil (progn (progn (make-local-variable 'delay-mode-hooks)
(let ((delay-mode-hooks t)) (funcall mode)))) (error nil))
(font-lock-ensure) (goto-char (point-min)) (let ((inhibit-read-only t)) (if
(fboundp 'text-property-search-forward) (progn (while (setq match
(text-property-search-forward ...)) (delete-region (let* ... ...) (let* ...
...)))))) (string-trim (buffer-string))))
  (unwind-protect (progn (set (make-local-variable
'markdown-fontify-code-blocks-natively) t) (insert string) (let
((inhibit-message t) (message-log-max nil) match) (condition-case nil
(progn (progn (make-local-variable 'delay-mode-hooks) (let (...) (funcall
mode)))) (error nil)) (font-lock-ensure) (goto-char (point-min)) (let
((inhibit-read-only t)) (if (fboundp 'text-property-search-forward) (progn
(while (setq match ...) (delete-region ... ...))))) (string-trim
(buffer-string)))) (and (buffer-name temp-buffer) (kill-buffer
temp-buffer)))
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (set
(make-local-variable 'markdown-fontify-code-blocks-natively) t) (insert
string) (let ((inhibit-message t) (message-log-max nil) match)
(condition-case nil (progn (progn (make-local-variable ...) (let ... ...)))
(error nil)) (font-lock-ensure) (goto-char (point-min)) (let
((inhibit-read-only t)) (if (fboundp 'text-property-search-forward) (progn
(while ... ...)))) (string-trim (buffer-string)))) (and (buffer-name
temp-buffer) (kill-buffer temp-buffer))))
  (let ((temp-buffer (generate-new-buffer " *temp*" t)))
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (set
(make-local-variable 'markdown-fontify-code-blocks-natively) t) (insert
string) (let ((inhibit-message t) (message-log-max nil) match)
(condition-case nil (progn (progn ... ...)) (error nil)) (font-lock-ensure)
(goto-char (point-min)) (let ((inhibit-read-only t)) (if (fboundp ...)
(progn ...))) (string-trim (buffer-string)))) (and (buffer-name
temp-buffer) (kill-buffer temp-buffer)))))
  (let ((string x614) (mode x616)) (let ((temp-buffer (generate-new-buffer
" *temp*" t))) (save-current-buffer (set-buffer temp-buffer)
(unwind-protect (progn (set (make-local-variable
'markdown-fontify-code-blocks-natively) t) (insert string) (let
((inhibit-message t) (message-log-max nil) match) (condition-case nil
(progn ...) (error nil)) (font-lock-ensure) (goto-char (point-min)) (let
(...) (if ... ...)) (string-trim (buffer-string)))) (and (buffer-name
temp-buffer) (kill-buffer temp-buffer))))))
  (progn (ignore (null x617)) (let ((string x614) (mode x616)) (let
((temp-buffer (generate-new-buffer " *temp*" t))) (save-current-buffer
(set-buffer temp-buffer) (unwind-protect (progn (set (make-local-variable
...) t) (insert string) (let (... ... match) (condition-case nil ... ...)
(font-lock-ensure) (goto-char ...) (let ... ...) (string-trim ...))) (and
(buffer-name temp-buffer) (kill-buffer temp-buffer)))))))
  (let* ((x616 (car-safe x615)) (x617 (cdr-safe x615))) (progn (ignore
(null x617)) (let ((string x614) (mode x616)) (let ((temp-buffer
(generate-new-buffer " *temp*" t))) (save-current-buffer (set-buffer
temp-buffer) (unwind-protect (progn (set ... t) (insert string) (let ...
... ... ... ... ...)) (and (buffer-name temp-buffer) (kill-buffer
temp-buffer))))))))
  (progn (ignore (consp x615)) (let* ((x616 (car-safe x615)) (x617
(cdr-safe x615))) (progn (ignore (null x617)) (let ((string x614) (mode
x616)) (let ((temp-buffer (generate-new-buffer " *temp*" t)))
(save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn ...
... ...) (and ... ...))))))))
  (let* ((x614 (car-safe val)) (x615 (cdr-safe val))) (progn (ignore (consp
x615)) (let* ((x616 (car-safe x615)) (x617 (cdr-safe x615))) (progn (ignore
(null x617)) (let ((string x614) (mode x616)) (let ((temp-buffer ...))
(save-current-buffer (set-buffer temp-buffer) (unwind-protect ...
...))))))))
  (progn (ignore (consp val)) (let* ((x614 (car-safe val)) (x615 (cdr-safe
val))) (progn (ignore (consp x615)) (let* ((x616 (car-safe x615)) (x617
(cdr-safe x615))) (progn (ignore (null x617)) (let ((string x614) (mode
x616)) (let (...) (save-current-buffer ... ...))))))))
  (let* ((val (if (stringp markup) (list markup 'gfm-view-mode) (list
(plist-get markup :value) (let* ((val ...)) (cond (... ...) (... ...) (t
...))))))) (progn (ignore (consp val)) (let* ((x614 (car-safe val)) (x615
(cdr-safe val))) (progn (ignore (consp x615)) (let* ((x616 (car-safe x615))
(x617 (cdr-safe x615))) (progn (ignore (null x617)) (let (... ...) (let ...
...))))))))
  eglot--format-markup((:kind "markdown" :value "```python\n(class)
openfx\n```"))
  mapconcat(eglot--format-markup ((:kind "markdown" :value
"```python\n(class) openfx\n```")) "\n")
  eglot--hover-info((:kind "markdown" :value "```python\n(class)
openfx\n```") (:start (:line 7 :character 6) :end (:line 7 :character


In GNU Emacs 30.0.50 (build 1, x86_64-w64-mingw32) of 2023-12-23 built
on fv-az1258-535Repository revision:
5c3ff1494b69bf45b99125f2423174222badfa43
Repository branch: master
Windowing system distributor 'Microsoft Corp.', version 10.0.22635
System Description: Microsoft Windows 10 Pro (v10.0.2009.22635.2915)

Configured using:
 'configure --prefix=/d/a/emacs-build/emacs-build/pkg/master-x86_64
'CFLAGS=-Ofast -fno-finite-math-only -fomit-frame-pointer -funroll-loops
-floop-parallelize-all -ftree-parallelize-loops=4 -march=skylake
-mtune=znver1' --with-native-compilation --with-zlib --with-xpm --with-xml2
--with-tree-sitter --with-tiff --with-rsvg --with-png --with-lcms2
--with-json --with-jpeg --with-harfbuzz --with-gnutls --with-gif
--with-cairo --without-dbus --with-compress-install'


-- 
Gary

[Message part 2 (text/html, inline)]

Message #8 received at 68217 <at> debbugs.gnu.org (full text, mbox):

From: João Távora <joaotavora <at> gmail.com>
To: Gary Oberbrunner <garyo <at> oberbrunner.com>, Yuan Fu <casouri <at> gmail.com>
Cc: jblevins <at> xbeta.org, 68217 <at> debbugs.gnu.org
Subject: Re: bug#68217: 30.0.50;
 Fix for eglot--format-markup to handle bad markdown
Date: Tue, 2 Jan 2024 22:11:29 +0000

On Tue, Jan 2, 2024 at 6:29 PM Gary Oberbrunner <garyo <at> oberbrunner.com> wrote:
> The markup arg passed to eglot--format-markup is:
>
>   (:kind "markdown" :value "```python\n(class) openfx\n```")
>
> I'm using pyright-langserver as the LSP server; I expect it is
> returning malformed markup or something that eglot doesn't expect.

Just for a bit of context to readers, markdown-mode at least in
certain situations, uses Emacs major modes to achieve syntactic
fontification of markdown code blocks.

Here, I don't think it's malformed markdown per se, it's that between
the markdown code block that the server says is Python there seems
to be some odd Python snippet:

(class) fx

So yes, the server is returning something fishy, but the way we
react right now isn't reasonable.

For me, it seems likely that python-mode when invoked by
markdown-mode didn't care about this and fontified away (when
triggered with font-lock-ensure), but python-ts-mode doesn't
and signals this error.

It could be caused by markdown-mode defaulting to the (new?) TS
mode when fontifying such markup.  Why is it defaulting like that?
Maybe it's just doing what Emacs would do if the user visited a
Python file?

I think markdown-mode choosing the same mode  that Emacs would
choose for interactive operation is correct.  But I'd first like
to confirm it isn't doing this arbitrarily: in the recent
past, I've seen markdown-mode load up cc-mode when I'm not using
the TS mode by default.

Jason, can you confirm what the intended behaviour is?

Also, I think python-ts-mode is also correct to signal the error
when finding this odd form, at least when the fontification routines
are invoked non-interactively.

Yuan, can you confirm what the intended behaviour is?

> I note that the following patch appears to fix it for me:
>
> modified   eglot.el
> @@ -1813,7 +1813,9 @@ Doubles as an indicator of snippet support."
>              (message-log-max nil)
>              match)
>          (ignore-errors (delay-mode-hooks (funcall mode)))
> -        (font-lock-ensure)
> +        (condition-case-unless-debug oops
> +            (font-lock-ensure)
> +          (error (eglot--warn (error-message-string oops))))
>          (goto-char (point-min))
>          (let ((inhibit-read-only t))
>            (when (fboundp 'text-property-search-forward) ;; FIXME: use compat

Finally there is this muffling of the error unless debug-on-error is true.
I think this is the right thing to do, at least right now.  In
an alternate universe with a restart system, it would probably be more
elegantly solved, but that's what we have now.

João

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.