Subject: Re: bug#65912: 2023-08-31; Fix xref errors when matching in unvisited files

From: Ikumi Keita <[email-masked]>
Date: Sun, 17 Sep 2023 22:02:40 +0900

Hi David,

> On Fri, 15 Sept 2023, 06:45 Ikumi Keita, <[email-masked]> wrote:
>> It seems reasonable basically. However, this makes two regression tests
>> fail:
>> 2 unexpected results:
>> FAILED  LaTeX-filling
>> FAILED  LaTeX-style-hook-with-class-option
>> 
>> I thought the following advice in latex-test.el is interfering badly:
>> ----------------------------------------------------------------------
>> ;; We need to ensure that font-lock has put the syntax properties
>> ;; already which won't happen in batch mode.  So trigger font-lock
>> ;; immediately.
>> (define-advice LaTeX-common-initialization (:after ())
>> (font-lock-ensure))
>> ----------------------------------------------------------------------
>> However, the failures remain even after I deleted this advice. I haven't
>> figured out what's going on yet. (Maybe the failures only appear in
>> batch mode and aren't real issues.) Does anyone out there have insights?

I think I now understand the reason. Some of the style hooks presume
that the file local variables are already established. Therefore,
I have to arrange regression tests so that `TeX-update-style' in
`latex-mode' runs inside `let' form where relevant variables are bound.

Attached is my current proposal.

>> > The attached patch (0002) parses and applies styles in the temp
>> > buffer, and only in such non-file-visiting buffers. It does so both in
>> > LaTeX-common-initialization and in plain-TeX-common-initialization,

LaTeX-common-initialization isn't a suitable place because of the
following reasons:
1. LaTeX mode makes addtion to `TeX-update-style-hook' after it calls
   LaTeX-common-initialization.
2. It is legitimate for style hooks to assume that they are called after
   mode hooks (e.g. LaTeX-mode-hook) run.
I moved the call to TeX-update-style at the last of latex-mode,
plain-tex-mode and ams-tex-mode. This doesn't cover context mode, but I
think it doesn't harm.
It still doesn't wait mode hooks in doctex mode since doctex mode is
defined by `define-derived-mode', but I'd postpone that issue for now.
I'll do I can do in feature/fix-mode-names-overlap branch later.

>> > which I think covers all the modes.

Actually it (and my patch as well) doesn't cover Texinfo mode, but I
suppose texinfo mode is out of the scope for the first place. Is this
right?

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine

diff --git a/font-latex.el b/font-latex.el
index 8fc567e2..4c49f99e 100644
--- a/font-latex.el
+++ b/font-latex.el
@@ -1353,7 +1353,13 @@ triggers Font Lock to recognize the change."
 
   ;; Make sure fontification will be refreshed if a user sets variables
   ;; influencing fontification in her file-local variables section.
-  (add-hook 'hack-local-variables-hook #'font-latex-after-hacking-local-variables t t))
+  (add-hook 'hack-local-variables-hook #'font-latex-after-hacking-local-variables t t)
+
+  ;; We may be using the mode programmatically to extract data, and we
+  ;; then need this to be set up first so that a command like
+  ;; `xref-find-references' doesn't bug out when matching hits in
+  ;; files that Emacs isn't visiting. (bug#65912)
+  (font-lock-set-defaults))
 
 (defun font-latex-update-font-lock (&optional _syntactic-kws)
   "Tell font-lock about updates of fontification rules.
diff --git a/latex.el b/latex.el
index 3abb00f2..ef9f5f0d 100644
--- a/latex.el
+++ b/latex.el
@@ -8045,7 +8045,14 @@ of `LaTeX-mode-hook'."
            filladapt-mode)
       (turn-off-filladapt-mode))
   ;; Set up flymake backend, see latex-flymake.el
-  (add-hook 'flymake-diagnostic-functions #'LaTeX-flymake nil t))
+  (add-hook 'flymake-diagnostic-functions #'LaTeX-flymake nil t)
+
+  ;; Complete style initialization in buffers which don't visit files
+  ;; and which are therefore missed by the setting of `find-file-hook'
+  ;; in `VirTeX-common-initialization'.  This is necessary for
+  ;; `xref-find-references', for example. (bug#65912)
+  (unless buffer-file-truename
+    (TeX-update-style)))
 
 (TeX-abbrev-mode-setup doctex-mode)
 
@@ -8068,6 +8075,8 @@ runs the hooks in `docTeX-mode-hook'."
         TeX-comment-start-regexp "\\(?:%\\(?:<[^>]+>\\)?\\)")
   (setq TeX-base-mode-name "docTeX")
   (TeX-set-mode-name)
+  ;; Force doctex specific fontification rules.
+  (setq font-lock-set-defaults nil)
   (funcall TeX-install-font-lock))
 
 ;; Enable LaTeX abbrevs in docTeX mode buffer.
diff --git a/plain-tex.el b/plain-tex.el
index 0800b2f3..b4c78fe4 100644
--- a/plain-tex.el
+++ b/plain-tex.el
@@ -135,7 +135,13 @@ of `plain-TeX-mode-hook'."
   (add-hook 'tool-bar-mode-hook #'plain-TeX-maybe-install-toolbar nil t)
   (plain-TeX-maybe-install-toolbar)
   (run-mode-hooks 'text-mode-hook 'TeX-mode-hook 'plain-TeX-mode-hook)
-  (TeX-set-mode-name))
+  (TeX-set-mode-name)
+  ;; Complete style initialization in buffers which don't visit files
+  ;; and which are therefore missed by the setting of `find-file-hook'
+  ;; in `VirTeX-common-initialization'.  This is necessary for
+  ;; `xref-find-references', for example. (bug#65912)
+  (unless buffer-file-truename
+    (TeX-update-style)))
 
 (defun plain-TeX-common-initialization ()
   "Common initialization for plain TeX like modes."
@@ -308,7 +314,13 @@ of `AmS-TeX-mode-hook'."
   (setq TeX-base-mode-name "AmS-TeX")
   (setq TeX-command-default "AmSTeX")
   (run-mode-hooks 'text-mode-hook 'TeX-mode-hook 'AmS-TeX-mode-hook)
-  (TeX-set-mode-name))
+  (TeX-set-mode-name)
+  ;; Complete style initialization in buffers which don't visit files
+  ;; and which are therefore missed by the setting of `find-file-hook'
+  ;; in `VirTeX-common-initialization'.  This is necessary for
+  ;; `xref-find-references', for example. (bug#65912)
+  (unless buffer-file-truename
+    (TeX-update-style)))
 
 (defcustom AmSTeX-clean-intermediate-suffixes
   TeX-clean-default-intermediate-suffixes
diff --git a/tests/context/context-test.el b/tests/context/context-test.el
index cc7c8d54..4eeab4b6 100644
--- a/tests/context/context-test.el
+++ b/tests/context/context-test.el
@@ -24,12 +24,6 @@
 (require 'ert)
 (require 'context)
 
-;; We need to ensure that font-lock has put the syntax properties
-;; already which won't happen in batch mode.  So trigger font-lock
-;; immediately.
-(define-advice ConTeXt-mode-common-initialization (:after ())
-  (font-lock-ensure))
-
 (AUCTeX-set-ert-path
  'ConTeXt-indent-test/in
  "context-indentation-in.tex"
diff --git a/tests/latex/font-latex-test.el b/tests/latex/font-latex-test.el
index 40f9633a..94150ac0 100644
--- a/tests/latex/font-latex-test.el
+++ b/tests/latex/font-latex-test.el
@@ -27,12 +27,6 @@
 (defvar font-lock-beg)
 (defvar font-lock-end)
 
-;; We need to ensure that font-lock has put the syntax properties
-;; already which won't happen in batch mode.  So trigger font-lock
-;; immediately.
-(define-advice LaTeX-common-initialization (:after ())
-  (font-lock-ensure))
-
 (ert-deftest font-latex-three-dollars ()
   "Test three consecutive dollar is ignored."
   ;; When the function `font-latex-match-dollar-math' encounters three
diff --git a/tests/latex/latex-test.el b/tests/latex/latex-test.el
index fbd64e09..13cbf482 100644
--- a/tests/latex/latex-test.el
+++ b/tests/latex/latex-test.el
@@ -24,12 +24,6 @@
 (require 'ert)
 (require 'latex)
 
-;; We need to ensure that font-lock has put the syntax properties
-;; already which won't happen in batch mode.  So trigger font-lock
-;; immediately.
-(define-advice LaTeX-common-initialization (:after ())
-  (font-lock-ensure))
-
 (AUCTeX-set-ert-path
  'LaTeX-indent-tabular-test/in
  "tabular-in.tex"
@@ -174,11 +168,10 @@
   (should (string=
            (with-temp-buffer
              (insert-file-contents LaTeX-filling/in)
-             (LaTeX-mode)
              (let ((fill-column 70)
                    (LaTeX-shortvrb-chars '(?\"))
                    (TeX-parse-self t))
-               (TeX-update-style t)
+               (LaTeX-mode)
                (search-forward "Lorem")
                (fill-paragraph)
 
@@ -416,9 +409,8 @@ backend=biber % here is a comment
 
       ;; dvipdfmx option should not trigger `TeX-PDF-from-DVI' for
       ;; XeLaTeX document
-      (latex-mode)
       (let ((TeX-engine 'xetex))
-        (TeX-update-style))
+        (LaTeX-mode))
       (should TeX-PDF-mode)
       (should (not (TeX-PDF-from-DVI)))
       (should (not (member "dvipdfmx" TeX-active-styles)))
diff --git a/tests/latex/texmathp-test.el b/tests/latex/texmathp-test.el
index 121af22c..a18df63a 100644
--- a/tests/latex/texmathp-test.el
+++ b/tests/latex/texmathp-test.el
@@ -55,7 +55,6 @@
     (should-not (with-temp-buffer
                   (insert "a $b$ \\verb|$| c ")
                   (LaTeX-mode)
-                  (font-lock-ensure)
                   (texmathp)))
 
     (should-not (with-temp-buffer
@@ -67,7 +66,6 @@ $
 \\end{verbatim}
 c")
                   (LaTeX-mode)
-                  (font-lock-ensure)
                   (texmathp)))))
 
 ;;; texmathp-test.el ends here