Subject: 29.3; comint--intersect-regions fontifies some output using input function
From: JD Smith <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Date: Fri, 21 Jun 2024 13:11:58 -0400
List-Id: <debbugs-submit.debbugs.gnu.org>

This relates to bug#51940: 29.0.50; [PATCH] Fontification and indentation in M-x shell.

I found a tiny bug in the introduced command `comint--intersect-regions'.  It pertains to the "field-expansion" in the following section, which is responsible for narrowing to alternating regions of input and output:

(let ((beg2 beg1)
      (end2 end1))
  (when (= beg2 beg)
    (setq beg2 (field-beginning beg2))) ; <--- bug here
  (when (= end2 end)
    (setq end2 (field-end end2)))
  ;; Narrow to the whole field surrounding the region
  (narrow-to-region beg2 end2))

The nature of the bug is that `field-beginning' is left-associative, whereas `(get-text-property (point) 'field)` is right associative.  You can demonstrate this to yourself as follows:

(progn
  (insert (concat "\n" (propertize "AAA" 'field 'a) (propertize "B" 'field 'b) (propertize "CCC" 'field 'c)))
  (goto-char (- (point) 4))             ; cursor on 'B'
  (list (get-text-property (point) 'field) (field-beginning) (field-end)
        (buffer-substring-no-properties (field-beginning) (field-end))))
;; gives: (b 356 359 "AAA")

So on the line:

              (setq beg2 (field-beginning beg2))) ; <--- bug here

if beg is at the first character of its field, `field-beginning` will actually return the start of the prior field.  This leads to `comint--intersect-regions' unwittingly including `output' field text in its input fontification narrowing.  I noticed this in a new python interactive mode I'm working on: the prompt itself (i.e. output) was being partially fontified by python-mode.

I believe the correct fix is:

              (setq beg2 (field-beginning (1+ beg2))))

This takes care of the problem here, even for single-character fields (like `boundary').  We know (< beg1 end) from the loop condition, so it's safe to increment beg2.

--- /var/folders/by/19mt78cj63v5_tslqh826g7w0000gn/T/jka-comrWKsIJ	2024-06-21 13:08:59
+++ /var/folders/by/19mt78cj63v5_tslqh826g7w0000gn/T/buffer-content-Q8dmID	2024-06-21 13:08:59
@@ -4122,7 +4122,7 @@
           (let ((beg2 beg1)
                 (end2 end1))
             (when (= beg2 beg)
-              (setq beg2 (field-beginning beg2)))
+              (setq beg2 (field-beginning (1+ beg2))))
             (when (= end2 end)
               (setq end2 (field-end end2)))
             ;; Narrow to the whole field surrounding the region