Subject: Re: bug#65912: 2023-08-31; Fix xref errors when matching in unvisited files

From: Ikumi Keita <[email-masked]>
Date: Fri, 15 Sep 2023 14:45:17 +0900

Hi David, thanks for raising these issues.

>>>>> David Fussner via bug-auctex via Bug reporting list for AUCTeX <[email-masked]> writes:

> I enclose a patch (0001), which puts the required call, with a comment, at
> the end of font-latex-set-syntactic-keywords, which seems to work for
> all the AUCTeX modes, but there may be a better place for it.

1. This breaks doctex fontification. Doctex mode calls
   `font-latex-setup' twice; once in `VirTeX-common-initialization',
   once in `docTeX-mode'. Since the first one calls
   `font-lock-set-defaults' which sets the variable
   `font-lock-set-defaults' to t (and `major-mode' isn't `doctex-mode'
   yet), the second call to `font-lock-set-defaults' does nothing and
   doctex specific fontification isn't taken into account.
   Adding
(setq font-lock-set-defaults nil)
   before (funcall TeX-install-font-lock) in `docTeX-mode' addresses
   this problem.

2. I think better place for `(font-lock-set-defaults)' is
   `font-latex-setup' rather than `font-latex-set-syntactic-keywords';
   the latter is sometimes called afterwards by AUCTeX style files.

I modified your proposal to make the attached patch. This fixes your
first bug keeping the doctex fontification.

> If you apply patch 0001, and make sure font-latex.el is reloaded, you
> can then see the next bug: in specific cases, xref-find-references
> reports a hit only in visited files, despite there being identical text
> in the unvisited one.

I confirmed the reported result.

> The attached patch (0002) parses and applies styles in the temp
> buffer, and only in such non-file-visiting buffers. It does so both in
> LaTeX-common-initialization and in plain-TeX-common-initialization,
> which I think covers all the modes.

It seems reasonable basically. However, this makes two regression tests
fail:
2 unexpected results:
   FAILED  LaTeX-filling
   FAILED  LaTeX-style-hook-with-class-option

I thought the following advice in latex-test.el is interfering badly:
----------------------------------------------------------------------
;; We need to ensure that font-lock has put the syntax properties
;; already which won't happen in batch mode.  So trigger font-lock
;; immediately.
(define-advice LaTeX-common-initialization (:after ())
  (font-lock-ensure))
----------------------------------------------------------------------
However, the failures remain even after I deleted this advice. I haven't
figured out what's going on yet. (Maybe the failures only appear in
batch mode and aren't real issues.) Does anyone out there have insights?

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine

diff --git a/font-latex.el b/font-latex.el
index 8fc567e2..92eb7046 100644
--- a/font-latex.el
+++ b/font-latex.el
@@ -1353,7 +1353,13 @@ triggers Font Lock to recognize the change."
 
   ;; Make sure fontification will be refreshed if a user sets variables
   ;; influencing fontification in her file-local variables section.
-  (add-hook 'hack-local-variables-hook #'font-latex-after-hacking-local-variables t t))
+  (add-hook 'hack-local-variables-hook #'font-latex-after-hacking-local-variables t t)
+
+  ;; We may be using the mode programmatically to extract data, and we
+  ;; then need this to be set up first so that a command like
+  ;; `xref-find-references' doesn't bug out when matching hits in
+  ;; files that Emacs isn't visiting.
+  (font-lock-set-defaults))
 
 (defun font-latex-update-font-lock (&optional _syntactic-kws)
   "Tell font-lock about updates of fontification rules.
diff --git a/latex.el b/latex.el
index 3abb00f2..a7f236e8 100644
--- a/latex.el
+++ b/latex.el
@@ -8068,6 +8068,7 @@ runs the hooks in `docTeX-mode-hook'."
         TeX-comment-start-regexp "\\(?:%\\(?:<[^>]+>\\)?\\)")
   (setq TeX-base-mode-name "docTeX")
   (TeX-set-mode-name)
+  (setq font-lock-set-defaults nil)
   (funcall TeX-install-font-lock))
 
 ;; Enable LaTeX abbrevs in docTeX mode buffer.