Subject: [PATCH] Problem with python--treesit-syntax-propertize

From: kobarity <[email-masked]>
Date: Sun, 14 Jan 2024 18:15:07 +0900

Hi,

I found a problem with python--treesit-syntax-propertize recently
introduced by the Bug#67977 patch.

1. emacs -Q
2. Open a file in python-ts-mode with the following contents:

#+begin_src python
"""Docstring.

test.
"""
S = """string."""
#+end_src

3. Locate the point on the third line.
4. M-q
5. An empty line will be inserted.
6. M-q
7. The string literal on the last line will be split as follows:

S = ""

"string."""

This problem does not occur in python-mode.

The direct cause of this problem is that the string-delimiter property
set in the docstring is removed.  python--treesit-syntax-propertize is
called to set the property, but it fails to set it properly.  Here is
the trace of python--treesit-syntax-propertize from step 4 above.

======================================================================
1 -> (python--treesit-syntax-propertize 1 45)
1 <- python--treesit-syntax-propertize: nil
======================================================================
1 -> (python--treesit-syntax-propertize 16 45)
1 <- python--treesit-syntax-propertize: nil

python--treesit-syntax-propertize is called with argument START 16.
This is the position inside the docstring.

It seems to me that python--treesit-syntax-propertize assumes that the
START argument is outside the triple-quoted string.  So one solution
might be to change START to the start of the string if it is within a
string, as in the attached patch.  However, I'm not sure this is the
right approach.  Should we use
syntax-propertize-extend-region-functions?

--
In GNU Emacs 30.0.50 (build 5, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.16.0, Xaw scroll bars) of 2024-01-13 built on ubuntu
Repository revision: 106cd9aafe8248ef91d7e89161adc5f912ea54eb
Repository branch: master
System Description: Ubuntu 22.04.3 LTS
>From 809c506504913b9e6af9b7de13dd5892cf91fec3 Mon Sep 17 00:00:00 2001
From: kobarity <[email-masked]>
Date: Sun, 14 Jan 2024 17:50:16 +0900
Subject: [PATCH] Fix python--treesit-syntax-propertize

* lisp/progmodes/python.el (python--treesit-syntax-propertize): If
START is in a string, move start to the beginning of the string.
---
 lisp/progmodes/python.el | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index e2f614f52c2..1430fd84141 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -1358,6 +1358,9 @@ python--treesit-syntax-propertize
   "Propertize triple-quote strings between START and END."
   (save-excursion
     (goto-char start)
+    (let ((node (treesit-node-at (point))))
+      (when (equal (treesit-node-type node) "string_content")
+        (goto-char (treesit-node-start (treesit-node-parent node)))))
     (while (re-search-forward (rx (or "\"\"\"" "'''")) end t)
       (let ((node (treesit-node-at (point))))
         ;; The triple quotes surround a non-empty string.
-- 
2.34.1