GNU bug report logs - #73302
TeX-find-closing-brace and verbatim macros

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 73302 in the body.
You can then email your comments to 73302 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: bug-auctex <at> gnu.org
Subject: TeX-find-closing-brace and verbatim macros
Date: Mon, 16 Sep 2024 18:12:10 +0200

Hello,

Steps to reproduce:
1. Create a TeX file with a verbatim macro, e.g., \email{foo <at> bar.com},
or anything in (LaTeX-verbatim-macros-with-braces).
2. Put the cursor on the first character inside the braces.
3. M-: (TeX-find-closing-brace)

This should return position after closing brace, but instead returns nil.

This bug affects the folding of verbatim macros, through a chain of
function calls from TeX-fold-hide-item to TeX-find-closing-brace.

The issue is similar to one addressed by up-list in lisp.el - the key
point is to recover gracefully from the failed call to (scan-lists
(point) 1 1), rather than to give up altogether.  A similar fix might
be possible, but I thought I'd raise the issue here first.

Thanks, best,

Paul

Message #8 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Paul Nelson <ultrono <at> gmail.com>
Cc: 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Mon, 16 Sep 2024 21:26:43 +0200

Hi Paul,

Paul Nelson <ultrono <at> gmail.com> writes:

> Steps to reproduce:
> 1. Create a TeX file with a verbatim macro, e.g., \email{foo <at> bar.com},
> or anything in (LaTeX-verbatim-macros-with-braces).

I tried it with this .tex file:

--8<---------------cut here---------------start------------->8---
\documentclass{article}

\usepackage{fvextra}
\begin{document}

\Verb{foo <at> bar}

\end{document}
--8<---------------cut here---------------end--------------->8---

> 2. Put the cursor on the first character inside the braces.
> 3. M-: (TeX-find-closing-brace)
>
> This should return position after closing brace, but instead returns
> nil.

I'm not sure if this is possibly wrong expectation.  If you put the
cursor on { after \Verb and hit 'C-u C-x =', you'll see among other
things:

  syntax: |       which means: string fence

whereas on { after \begin you get:

  syntax: (}      which means: open, matches }

so `scan-lists' is expected to fail after \Verb.

> This bug affects the folding of verbatim macros, through a chain of
> function calls from TeX-fold-hide-item to TeX-find-closing-brace.
>
> The issue is similar to one addressed by up-list in lisp.el - the key
> point is to recover gracefully from the failed call to (scan-lists
> (point) 1 1), rather than to give up altogether.  A similar fix might
> be possible, but I thought I'd raise the issue here first.

I haven't looked at lisp.el, but we have some functions in latex.el for
verbatim macros, namely, `LaTeX-verbatim-p',
`LaTeX-verbatim-macro-boundaries' and `LaTeX-current-verbatim-macro'.
Maybe we can use them to fix the issue.

So maybe we should look at the issue you're facing with folding and see
how we can fix it with the functions above.

Best, Arash

Message #11 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Arash Esbati <arash <at> gnu.org>
Cc: 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Mon, 16 Sep 2024 22:13:07 +0200

[Message part 1 (text/plain, inline)]

Hi Arash,

Thanks, that clarifies things.

To give an example, suppose we wanted to fold your \Verb macro.  To do
so, we might try adding

  ("{1}" ("lVerb"))

to TeX-fold-macro-spec-list.  Folding then fails because, with point
at the start of "\Verb",

  (TeX-fold-macro-nth-arg 1 (point))

returns nil, but should return a string with contents "foo <at> bar".

The function LaTeX-verbatim-macro-boundaries that you mentioned leads
to a workaround: if we tweak TeX-fold-macro-nth-arg as in the attached
(incomplete) patch and add

  (my-display ("Verb"))

to TeX-fold-macro-spec-list, where

  (defun my-display (text &rest args) text)

then folding of \Verb{...} works as intended.  We would need to tweak
TeX-fold-macro-nth-arg further (and probably make it look a bit hacky)
to get it to work correctly with ("{1}" ("lVerb")), however.  Any
suggestions would be welcome.

Thanks, best,

Paul

[0001-Fix-folding-of-verbatim-macros.patch (application/octet-stream, attachment)]

Message #14 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Arash Esbati <arash <at> gnu.org>
Cc: 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Tue, 17 Sep 2024 12:02:43 +0200

[Message part 1 (text/plain, inline)]

I took another look just now, and the fix to allow string specs was
simpler than I had thought.  The attached patch suffices for my
practical purposes, but I'll confess that I don't know all the ways
verbatim arguments can appear in macros.

Paul

[0001-Fix-folding-of-verbatim-macros.patch (application/octet-stream, attachment)]

Message #17 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: Paul Nelson <ultrono <at> gmail.com>
Cc: Arash Esbati <arash <at> gnu.org>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Thu, 19 Sep 2024 23:47:10 +0900

Hi Paul,

>>>>> Paul Nelson <ultrono <at> gmail.com> writes:
> I took another look just now, and the fix to allow string specs was
> simpler than I had thought.  The attached patch suffices for my
> practical purposes, but I'll confess that I don't know all the ways
> verbatim arguments can appear in macros.

TeX-fold-mode is supposed to run in none-LaTeX-mode buffers, so
`LaTeX-verbatim-p' should be wrapped with some conditional using
`derived-mode-p'. Can you work on it?

(Actually, AUCTeX has generic function `TeX-verbatim-p', but only LaTeX
mode has meaningful implementation for it.)

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine
#Gaza #StopMassiveKilling #CeasefireNOW

Message #20 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Ikumi Keita <ikumi <at> ikumi.que.jp>
Cc: Arash Esbati <arash <at> gnu.org>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Thu, 19 Sep 2024 17:35:32 +0200

Hi Ikumi,

Sure, I'd be happy to work on this.

> TeX-fold-mode is supposed to run in none-LaTeX-mode buffers, so
> `LaTeX-verbatim-p' should be wrapped with some conditional using
> `derived-mode-p'. Can you work on it?
>
> (Actually, AUCTeX has generic function `TeX-verbatim-p', but only LaTeX
> mode has meaningful implementation for it.)

According to the documentation, TeX-verbatim-p always returns nil
outside LaTeX-mode, so it seems to me that to fix the issue you raise,
it would suffice to use that function in place of LaTeX-verbatim-p
(rather than doing derive-mode-p checks).  Does that sound right to
you?  (I never use the other modes, so my knowledge there is a bit
spotty.)

Thanks, best,

Paul

Message #23 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: Paul Nelson <ultrono <at> gmail.com>
Cc: Arash Esbati <arash <at> gnu.org>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Fri, 20 Sep 2024 01:14:45 +0900

Hi Paul,

>>>>> Paul Nelson <ultrono <at> gmail.com> writes:
>> (Actually, AUCTeX has generic function `TeX-verbatim-p', but only LaTeX
>> mode has meaningful implementation for it.)

> According to the documentation, TeX-verbatim-p always returns nil
> outside LaTeX-mode, so it seems to me that to fix the issue you raise,
> it would suffice to use that function in place of LaTeX-verbatim-p
> (rather than doing derive-mode-p checks).  Does that sound right to
> you?  (I never use the other modes, so my knowledge there is a bit
> spotty.)

Hmm, then the code takes the form
  (if (TeX-verbatim-p)
       (cdr (LaTeX-verbatim-macro-boundaries))
      ...
, which looks like to call LaTeX-mode specific function
`LaTeX-verbatim-macro-boundaries' in other modes as well... Well, I'd
accept that if you place there some FIXME comments which warns that this
should be fixed when other mode implements non-trivial
`TeX-verbatim-p-function'.

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine
#Gaza #StopMassiveKilling #CeasefireNOW

Message #26 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Ikumi Keita <ikumi <at> ikumi.que.jp>
Cc: Arash Esbati <arash <at> gnu.org>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Thu, 19 Sep 2024 22:17:16 +0200

[Message part 1 (text/plain, inline)]

Hi Ikumi,

OK, let me know how the attached looks.

By the way, to clear up some of the confusion I expressed in an
earlier message, I don't know whether a macro can have multiple
verbatim arguments (some of which may be optional), and if so, then I
don't know whether (LaTeX-verbatim-macro-boundaries) does the
appropriate thing here (return the end of the macro argument, rather
than the end of the entire macro).

Thanks, best,

Paul

[0001-Fix-folding-of-verbatim-macros.patch (application/octet-stream, attachment)]

Message #29 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Ikumi Keita <ikumi <at> ikumi.que.jp>
Cc: Arash Esbati <arash <at> gnu.org>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Fri, 20 Sep 2024 06:40:18 +0200

[Message part 1 (text/plain, inline)]

Hi again,

I noticed just now that my patch can break the folding of
\end{verbatim} environments when called via TeX-fold-region.  Steps to
reproduce:

1. After loading AUCTeX and tex-fold, evaluate

   (add-to-list 'TeX-fold-macro-spec-list '(identity ("end")))

2. Create a LaTeX file containing

\begin{verbatim}
blah
\end{verbatim}

3. Make the region contain the line "\end{verbatim}", but none of the
line "\begin{verbatim}"

4. C-c C-o C-r (TeX-fold-region).

This results in [Error: No content or function found], resulting from
a failure for TeX-fold-macro-nth-arg to recover the first argument to
\end{verbatim}.


One can trace the issue to the following discrepancy:

a. With point on the "{" in "\end{verbatim}", (TeX-verbatim-p) returns
nil (as expected).

b. If we repeat (a) after narrowing to a region that contains the line
"\end{verbatim}" but none of the line "\begin{verbatim}", then it
instead returns t.


As a fix, we could either:

(i) Do (save-restriction (widen) ...) in TeX-fold-macro-nth-arg before
(TeX-verbatim-p).

(ii) Do (save-restriction (widen) ...) in LaTeX-verbatim-p.


The advantage of (ii) is that it arguably fixes the root cause rather
than just patching it at the source.  The disadvantage is that I don't
know off hand whether this would affect other calls to
LaTeX-verbatim-p in an undesired way.  I lean in favor of (i) for now,
and have updated my patch accordingly, but would welcome other
feedback.

Paul

[0001-Fix-folding-of-verbatim-macros.patch (application/octet-stream, attachment)]

Message #32 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: Paul Nelson <ultrono <at> gmail.com>
Cc: Arash Esbati <arash <at> gnu.org>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Fri, 20 Sep 2024 16:43:00 +0900

Hi Paul,

>>>>> Paul Nelson <ultrono <at> gmail.com> writes:
> I noticed just now that my patch can break the folding of
> \end{verbatim} environments when called via TeX-fold-region.  Steps to
> reproduce:

> 1. After loading AUCTeX and tex-fold, evaluate

>    (add-to-list 'TeX-fold-macro-spec-list '(identity ("end")))

> 2. Create a LaTeX file containing

> \begin{verbatim}
> blah
> \end{verbatim}

> 3. Make the region contain the line "\end{verbatim}", but none of the
> line "\begin{verbatim}"

> 4. C-c C-o C-r (TeX-fold-region).

> This results in [Error: No content or function found], resulting from
> a failure for TeX-fold-macro-nth-arg to recover the first argument to
> \end{verbatim}.

Confirmed.

> One can trace the issue to the following discrepancy:

> a. With point on the "{" in "\end{verbatim}", (TeX-verbatim-p) returns
> nil (as expected).

> b. If we repeat (a) after narrowing to a region that contains the line
> "\end{verbatim}" but none of the line "\begin{verbatim}", then it
> instead returns t.

Indeed.

> As a fix, we could either:

> (i) Do (save-restriction (widen) ...) in TeX-fold-macro-nth-arg before
> (TeX-verbatim-p).

> (ii) Do (save-restriction (widen) ...) in LaTeX-verbatim-p.

> The advantage of (ii) is that it arguably fixes the root cause rather
> than just patching it at the source.  The disadvantage is that I don't
> know off hand whether this would affect other calls to
> LaTeX-verbatim-p in an undesired way.  I lean in favor of (i) for now,
> and have updated my patch accordingly, but would welcome other
> feedback.

I don't have particular preference, so I think we can install your
change, provided that no objection is raised.

Thank you for your contribution!

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine
#Gaza #StopMassiveKilling #CeasefireNOW

Message #35 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Ikumi Keita <ikumi <at> ikumi.que.jp>
Cc: 73302 <at> debbugs.gnu.org, Paul Nelson <ultrono <at> gmail.com>
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Sat, 21 Sep 2024 09:30:02 +0200

Ikumi Keita <ikumi <at> ikumi.que.jp> writes:

>>>>>> Paul Nelson <ultrono <at> gmail.com> writes:
>> As a fix, we could either:
>
>> (i) Do (save-restriction (widen) ...) in TeX-fold-macro-nth-arg before
>> (TeX-verbatim-p).
>
>> (ii) Do (save-restriction (widen) ...) in LaTeX-verbatim-p.
>
>> The advantage of (ii) is that it arguably fixes the root cause rather
>> than just patching it at the source.  The disadvantage is that I don't
>> know off hand whether this would affect other calls to
>> LaTeX-verbatim-p in an undesired way.  I lean in favor of (i) for now,
>> and have updated my patch accordingly, but would welcome other
>> feedback.
>
> I don't have particular preference,

I think we should go with (i) and let the caller decide.

> so I think we can install your change, provided that no objection is
> raised.

@Paul: How does you change work in a file like this:

--8<---------------cut here---------------start------------->8---
\documentclass{article}

\usepackage{fvextra}

\begin{document}

Vanilla \verb|LaTeX|

With \Verb|fvextra|, or \Verb[fontsize=\scriptsize]|fvextra|.

With \Verb{fvextra}, or \Verb[fontsize=\scriptsize]{fvextra}.

\end{document}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:
--8<---------------cut here---------------end--------------->8---

> Thank you for your contribution!

+1.

Best, Arash

Message #38 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Arash Esbati <arash <at> gnu.org>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Sat, 21 Sep 2024 10:47:18 +0200

> @Paul: How does you change work in a file like this:
>
> --8<---------------cut here---------------start------------->8---
> \documentclass{article}
>
> \usepackage{fvextra}
>
> \begin{document}
>
> Vanilla \verb|LaTeX|
>
> With \Verb|fvextra|, or \Verb[fontsize=\scriptsize]|fvextra|.
>
> With \Verb{fvextra}, or \Verb[fontsize=\scriptsize]{fvextra}.
>
> \end{document}
>
> %%% Local Variables:
> %%% mode: latex
> %%% TeX-master: t
> %%% End:
> --8<---------------cut here---------------end--------------->8---


One issue that shows up when attempting to fold these examples via a
string display spec is that (TeX-find-macro-end), called with point at
the beginning of \Verb|LaTeX|, returns the end of \Verb rather than
the complete macro.  As a result, using tex-fold with a string display
spec never folds the complete macro, but instead just the part
appearing before the first "|".  Perhaps this should be changed?  In
any event, it's an issue orthogonal to my patch.

Modulo this issue, I think everything in the example you provided
works as intended.  For example, with the spec ("[1]:{1}!||{1}@||[1]#"
("Verb"), the tex code

\Verb[fontsize=\scriptsize]{fvextra}
\Verb{fvextra}
\Verb[fontsize=\scriptsize]|fvextra|

folds to something that looks like

fontsize=\scriptsize:fvextra!
fvextra@
fontsize=\scriptsize#|fvextra|

which I think is the expected behavior, modulo the oddity that in the
third example, "|fvextra|" appears at the end, due to the behavior of
(TeX-find-macro-end) noted above.

Note that the string display specs in tex-fold don't support "|...|"
arguments.  I do provide folding support for "\verb|...|" in
https://github.com/ultronozm/czm-tex-fold.el, but it's done there
using a function display spec, ultimately using the regexp
"\\\\verb|\\([^|]*\\)|" rather than anything built-in.  I guess I
could extend it to work also with \Verb.

Alternatively, one could view the contents of "|...|" as a "required
argument" and extend the string display spec folding to work with
"|...|" in addition to "{...}", but I don't see the immediate payoff
from doing so.

Message #41 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Arash Esbati <arash <at> gnu.org>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Sat, 21 Sep 2024 10:50:05 +0200

On an unrelated note, I did encounter another issue with this change:
it sporadically breaks some of the folding carried out by my package
https://github.com/ultronozm/czm-tex-mint.el, due to a weird bug that
I'm having trouble diagnosing.

The issue concerns detecting verbatim macros in code immediately after
moving out of an mmm-mode block in a LaTeX buffer.  Specifically, with
point at the beginning of the final line of such a block, evaluating

  (progn (forward-line) (TeX-verbatim-p))

returns non-nil, but should instead return nil.  For instance, if we
interactively do (forward-line) followed by (TeX-verbatim-p), then we
get nil.

As a result, (TeX-verbatim-p) returns t in cases where
(LaTeX-verbatim-macro-boundaries) doesn't return anything useful, and
so TeX-fold-macro-nth-arg fails to detect arguments.

The issue really boils down to

  (progn (forward-line) (syntax-propertize (point)) (nth 3 (syntax-ppss)))

The issue disappears when tested via edebug, which makes it a bit
tricky for me to debug further.  I've noticed that the issue does NOT
arise with

  (progn (forward-line) (mmm-update-submode-region) (syntax-propertize
(point)) (nth 3 (syntax-ppss)))

but DOES still arise when putting the same lines in my code (rather
than running it via M-:).

Any quick thoughts/suggestions?

Is there some function that "very forcefully" updates whatever
internals (syntax-propertize (point)) (nth 3 (syntax-ppss)) depend
upon?

Is one forced to resort to idle timer trickery?  That would be a bit
painful in the case I'm considering.

Message #44 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: Paul Nelson <ultrono <at> gmail.com>
Cc: Arash Esbati <arash <at> gnu.org>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Sun, 22 Sep 2024 00:35:47 +0900

Hi Paul,

>>>>> Paul Nelson <ultrono <at> gmail.com> writes:
> The issue concerns detecting verbatim macros in code immediately after
> moving out of an mmm-mode block in a LaTeX buffer.

There is a TODO entry in syntax.el which says
;; - interaction with mmm-mode.
, so maybe syntax-related facilities including syntax-ppss and
syntax-propertize-function have something incompatible with mmm-mode.

However, I don't know anything concrete with respect to such conflict.

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine
#Gaza #StopMassiveKilling #CeasefireNOW

Message #47 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Paul Nelson <ultrono <at> gmail.com>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Tue, 24 Sep 2024 22:57:26 +0200

Paul Nelson <ultrono <at> gmail.com> writes:

> One issue that shows up when attempting to fold these examples via a
> string display spec is that (TeX-find-macro-end), called with point at
> the beginning of \Verb|LaTeX|, returns the end of \Verb rather than
> the complete macro.  As a result, using tex-fold with a string display
> spec never folds the complete macro, but instead just the part
> appearing before the first "|".  Perhaps this should be changed?

Thanks for testing.  I installed your change; I suggest we try to fix
the issue you describe later.  Again, thanks for the patch.

> Note that the string display specs in tex-fold don't support "|...|"
> arguments.  I do provide folding support for "\verb|...|" in
> https://github.com/ultronozm/czm-tex-fold.el, but it's done there
> using a function display spec, ultimately using the regexp
> "\\\\verb|\\([^|]*\\)|" rather than anything built-in.  I guess I
> could extend it to work also with \Verb.
>
> Alternatively, one could view the contents of "|...|" as a "required
> argument" and extend the string display spec folding to work with
> "|...|" in addition to "{...}", but I don't see the immediate payoff
> from doing so.

What do others think?

Best, Arash

Message #50 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: Arash Esbati <arash <at> gnu.org>
Cc: 73302 <at> debbugs.gnu.org, Paul Nelson <ultrono <at> gmail.com>
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Thu, 26 Sep 2024 14:04:54 +0900

>>>>> Arash Esbati <arash <at> gnu.org> writes:
>> Alternatively, one could view the contents of "|...|" as a "required
>> argument" and extend the string display spec folding to work with
>> "|...|" in addition to "{...}", but I don't see the immediate payoff
>> from doing so.

> What do others think?

It seems a reasonable idea to me, provided that it is limited to
\verb-like commands. It would be nice if it is easily extended for
non-LaTeX mode constructs. (I imagine that ConTeXt has some \verb-like
commands.)

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine
#Gaza #StopMassiveKilling #CeasefireNOW

Message #53 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Ikumi Keita <ikumi <at> ikumi.que.jp>
Cc: Arash Esbati <arash <at> gnu.org>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Thu, 26 Sep 2024 20:39:41 +0200

For whatever it's worth, I updated my package czm-tex-fold to support
the variety of verbatim formats from Arash's earlier email.  This is
less flexible than what we would get by extending the display spec
format, but has the advantage that it's been implemented.

If you want to try it without setting up my package, just evaluate the
following code, with the final line evaluated in your tex buffer.
Then run folding commands (e.g., C-c C-o C-b) as usual.

(defun czm-tex-fold--create-misc-overlay (start end str spec)
  "Create an overlay to fold quotes between START and END with STR and SPEC."
  (let ((priority (TeX-overlay-prioritize start end))
        (ov (make-overlay start end)))
    (overlay-put ov 'category 'TeX-fold)
    (overlay-put ov 'priority priority)
    (overlay-put ov 'evaporate t) ; Remove the overlay when the text
is modified.
    (overlay-put ov 'display str)
    (overlay-put ov 'TeX-fold-display-string-spec spec)))

(defvar czm-tex-fold--verb-regex
  "\\\\\\(Verb\\|verb\\)\\(?:\\[[^]]*\\]\\)?[|{]\\([^|}]*\\)[|}]")

(defun czm-tex-fold-verbs (start end)
  "Fold `\\verb|...|', `\\Verb|...|', and `\\Verb{...}' macros between
START and END."
  (save-excursion
    (goto-char start)
    (while (re-search-forward czm-tex-fold--verb-regex end t)
      (let* ((verb-start (match-beginning 0))
             (verb-end (match-end 0))
             (verb-content (match-string 2))
             (spec (lambda (&rest _args)
                     (when (looking-at czm-tex-fold--verb-regex)
                       (match-string 2)))))
        (czm-tex-fold--create-misc-overlay verb-start verb-end
verb-content spec)))))

(add-hook 'TeX-fold-region-functions #'czm-tex-fold-verbs nil t)

On Thu, Sep 26, 2024 at 7:04 AM Ikumi Keita <ikumi <at> ikumi.que.jp> wrote:
>
> >>>>> Arash Esbati <arash <at> gnu.org> writes:
> >> Alternatively, one could view the contents of "|...|" as a "required
> >> argument" and extend the string display spec folding to work with
> >> "|...|" in addition to "{...}", but I don't see the immediate payoff
> >> from doing so.
>
> > What do others think?
>
> It seems a reasonable idea to me, provided that it is limited to
> \verb-like commands. It would be nice if it is easily extended for
> non-LaTeX mode constructs. (I imagine that ConTeXt has some \verb-like
> commands.)
>
> Regards,
> Ikumi Keita
> #StandWithUkraine #StopWarInUkraine
> #Gaza #StopMassiveKilling #CeasefireNOW

Message #56 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: Paul Nelson <ultrono <at> gmail.com>
Cc: Arash Esbati <arash <at> gnu.org>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Fri, 27 Sep 2024 15:05:29 +0900

Hi Paul,

>>>>> Paul Nelson <ultrono <at> gmail.com> writes:
> For whatever it's worth, I updated my package czm-tex-fold to support
> the variety of verbatim formats from Arash's earlier email.  This is
> less flexible than what we would get by extending the display spec
> format, but has the advantage that it's been implemented.

> If you want to try it without setting up my package, just evaluate the
> following code, with the final line evaluated in your tex buffer.
> Then run folding commands (e.g., C-c C-o C-b) as usual.

Thanks, it works well for me.

Regards,
Ikumi Keita
#StandWithUkraine #StopWarInUkraine
#Gaza #StopMassiveKilling #CeasefireNOW

Message #59 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Paul Nelson <ultrono <at> gmail.com>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Fri, 27 Sep 2024 19:22:12 +0200

Paul Nelson <ultrono <at> gmail.com> writes:

> For whatever it's worth, I updated my package czm-tex-fold to support
> the variety of verbatim formats from Arash's earlier email.  This is
> less flexible than what we would get by extending the display spec
> format, but has the advantage that it's been implemented.

> (defvar czm-tex-fold--verb-regex
>   "\\\\\\(Verb\\|verb\\)\\(?:\\[[^]]*\\]\\)?[|{]\\([^|}]*\\)[|}]")

A little simpler:

  "\\\\\\([Vv]verb\\)\\(?:\\[[^]]*\\]\\)?[|{]\\([^|}]*\\)[|}]"

How does your code work on this example:

  \verb|}|

?  I admit, this is a mess, but maybe we can come up with a solution
within AUCTeX; I have to think about it, though.

Best, Arash

Message #62 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Arash Esbati <arash <at> gnu.org>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Fri, 27 Sep 2024 20:36:10 +0200

Hi Arash,

> How does your code work on this example:
>
>   \verb|}|
>
> ?  I admit, this is a mess, but maybe we can come up with a solution
> within AUCTeX; I have to think about it, though.
>
> Best, Arash

That example motivated me to take a proper look at the verbatim
functions in AUCTeX, which is indeed relevant.  The following modified
code passes all tests mentioned thus far in this thread, although the
way the verbatim argument is found might be a bit brittle -- I guess
you could have a commented "{" in the middle of a multi-line
\Verb{...}, but multi-line \Verb{...} seems like illegal latex anyway,
so maybe it's all OK?

(defun czm-tex-fold--verb-data (&rest _args)
  "Return data for a verbatim macro.
Returns a list of the form (START END CONTENT)."
  (when-let* ((boundaries (LaTeX-verbatim-macro-boundaries))
              (bound-start (car boundaries))
              (bound-end (cdr boundaries))
              (end-delim-char (char-before bound-end))
              (start-delim-char
               (cond
                ((eq end-delim-char ?}) ?{)
                ((eq end-delim-char ?|) ?|)))
              (start-delim (char-to-string start-delim-char)))
    (goto-char (1- bound-end))
    (when (search-backward start-delim bound-start t)
      (let* ((verb-arg-start (1+ (point)))
             (verb-arg-end (1- bound-end)))
        (list
         bound-start
         bound-end
         (buffer-substring verb-arg-start verb-arg-end))))))

(defun czm-tex-fold-verbs (start end)
  "Fold verbatim macros between START and END."
  (save-excursion
    (goto-char start)
    (let ((re (concat "\\\\" (regexp-opt
                              (append
                               (LaTeX-verbatim-macros-with-braces)
                               (LaTeX-verbatim-macros-with-delims))))))
      (while (re-search-forward re end t)
        (when-let* ((data (czm-tex-fold--verb-data))
                    (verb-start (nth 0 data))
                    (verb-end (nth 1 data))
                    (content (nth 2 data))
                    (spec (lambda (&rest _args)
                            (nth 2 (czm-tex-fold--verb-content)))))
          (czm-tex-fold--create-misc-overlay verb-start verb-end
content spec))))))

Message #65 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Paul Nelson <ultrono <at> gmail.com>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Mon, 30 Sep 2024 20:52:44 +0200

Hi Paul,

Paul Nelson <ultrono <at> gmail.com> writes:

> That example motivated me to take a proper look at the verbatim
> functions in AUCTeX, which is indeed relevant.  The following modified
> code passes all tests mentioned thus far in this thread, although the
> way the verbatim argument is found might be a bit brittle

Does the code work on this example:

  \Verb{foo{b}ar}

> I guess you could have a commented "{" in the middle of a multi-line
> \Verb{...}, but multi-line \Verb{...} seems like illegal latex anyway,
> so maybe it's all OK?

I don't think you can have a commented "{" in the argument of
\Verb{...}, only balanced pair of braces like above.  Multi-line \verb
are illegal in vanilla LaTeX, yes, but with fvextra v1.8.0, you can have
them[1]:

  * By default, `\Verb` and `\SaveVerb` now take multi-line (but not
    multi-paragraph) verbatim arguments.  The old behavior of requiring
    verbatim arguments to be on a single line can be restored by setting
    the new option `vargsingleline` to `true`.

But within AUCTeX, we disallow multi-line verb args.

Overall, I think the following (100% untested) changes should make your
code more robust:

--8<---------------cut here---------------start------------->8---
(defun czm-tex-fold--verb-data (&rest _args)
  "Return data for a verbatim macro.
Returns a list of the form (START END CONTENT)."
  (when-let* ((boundaries (LaTeX-verbatim-macro-boundaries))
              (bound-start (car boundaries))
              (bound-end (cdr boundaries))
              (end-delim-char (char-before bound-end))
              (start-delim-char (if (= end-delim-char ?\})
                                    ?\{
                                  end-delim-char))
              (start-delim (char-to-string start-delim-char)))
    (goto-char (1- bound-end))
    (when (if (string= start-delim TeX-grop)
              (and (backward-sexp) (point))
            (search-backward start-delim bound-start t))
      (let* ((verb-arg-start (1+ (point)))
             (verb-arg-end (1- bound-end)))
        (list bound-start
              bound-end
              (buffer-substring-no-properties verb-arg-start
                                              verb-arg-end))))))

(defun czm-tex-fold-verbs (start end)
  "Fold verbatim macros between START and END."
  (save-excursion
    (goto-char start)
    (let ((re (concat "\\\\" (regexp-opt
                              (append
                               (LaTeX-verbatim-macros-with-braces)
                               (LaTeX-verbatim-macros-with-delims))))))
      (while (let ((case-fold-search nil))
               (re-search-forward re end t))
        (when-let* ((data (czm-tex-fold--verb-data))
                    (verb-start (nth 0 data))
                    (verb-end (nth 1 data))
                    (content (nth 2 data))
                    (spec (lambda (&rest _args)
                            (nth 2 (czm-tex-fold--verb-content)))))
          (czm-tex-fold--create-misc-overlay verb-start verb-end
                                             content spec))))))
--8<---------------cut here---------------end--------------->8---

Best, Arash

Footnotes:
[1]  https://ctan.net/macros/latex/contrib/fvextra/CHANGELOG.md

Message #68 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Arash Esbati <arash <at> gnu.org>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Mon, 30 Sep 2024 20:06:15 +0100

Hi Arash, thanks, the modified code you supplied (after a couple
tweaks) addresses all examples considered thus far, and indeed feels
less brittle.  Pushed to my package repo, copying for below in case of
curiosity.

(defun czm-tex-fold--verb-data (&rest _args)
  "Return data for a verbatim macro.
Returns a list of the form (START END CONTENT)."
  (when-let* ((boundaries (LaTeX-verbatim-macro-boundaries))
              (bound-start (car boundaries))
              (bound-end (cdr boundaries))
              (end-delim-char (char-before bound-end))
              (start-delim-char (if (= end-delim-char ?\})
                                    ?\{
                                  end-delim-char))
              (start-delim (char-to-string start-delim-char))
              (verb-arg-start
               (1+ (progn
                     (goto-char bound-end)
                     (if (string= start-delim TeX-grop)
                         (progn (backward-sexp) (point))
                       (forward-char -1)
                       (search-backward start-delim bound-start t)))))
              (verb-arg-end (1- bound-end)))
    (list bound-start
          bound-end
          (buffer-substring-no-properties verb-arg-start
                                          verb-arg-end))))

(defun czm-tex-fold-verbs (start end)
  "Fold verbatim macros between START and END."
  (save-excursion
    (goto-char start)
    (let ((re (concat "\\\\" (regexp-opt
                              (append
                               (LaTeX-verbatim-macros-with-braces)
                               (LaTeX-verbatim-macros-with-delims))))))
      (while (let ((case-fold-search nil))
               (re-search-forward re end t))
        (when-let* ((data (czm-tex-fold--verb-data))
                    (spec (lambda (&rest _args)
                            (nth 2 (czm-tex-fold--verb-data)))))
          (apply #'czm-tex-fold--create-misc-overlay (append data
(list spec))))))))

Message #71 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Paul Nelson <ultrono <at> gmail.com>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Mon, 30 Sep 2024 21:28:17 +0200

Paul Nelson <ultrono <at> gmail.com> writes:

> Hi Arash, thanks, the modified code you supplied (after a couple
> tweaks) addresses all examples considered thus far, and indeed feels
> less brittle.

Hi Paul, you're welcome.  Now that the exercise is completed, do you see
a chance to implement this also in tex-fold.el and make this feature
available to all AUCTeX users?

Best, Arash

Message #74 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Arash Esbati <arash <at> gnu.org>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Mon, 30 Sep 2024 21:36:00 +0100

[Message part 1 (text/plain, inline)]

Hi Arash,

> Now that the exercise is completed, do you see
> a chance to implement this also in tex-fold.el and make this feature
> available to all AUCTeX users?

Yes, you're one step ahead.  Indeed, the motivation for the bug fix
that started this thread was that I've been polishing
https://github.com/ultronozm/czm-tex-fold.el lately, with the aim of
sharing it here and pushing it in some form to AUCTeX or ELPA
(ideally, from my POV, the package would "cease to exist").

Besides verbatim environments, the package adds folding support for
quotes, dashes and begin/end environments (and maybe a couple more
opinionated defaults of lesser importance).

The folding for verbatim environments that we just discussed could be
incorporated by something like the attached patch.  The other features
could be discussed separately.  Any thoughts would be appreciated.

Thanks, best,

Paul

[0001-Improved-verbatim-folding.patch (application/octet-stream, attachment)]

Message #77 received at 73302 <at> debbugs.gnu.org (full text, mbox):

From: Paul Nelson <ultrono <at> gmail.com>
To: Arash Esbati <arash <at> gnu.org>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, 73302 <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Mon, 30 Sep 2024 21:44:44 +0100

[Message part 1 (text/plain, inline)]

Apologies for the spam, but I've amended the patch from my previous
email because I noticed that the change from "buffer-substring" to
"buffer-substring-no-properties" (from your previous email) resulted
in some odd behavior with colors when the user edits an already folded
verbatim environment and then exits the fold.

[0001-Improved-verbatim-folding.patch (application/octet-stream, attachment)]

Message #82 received at 73302-done <at> debbugs.gnu.org (full text, mbox):

From: Arash Esbati <arash <at> gnu.org>
To: Paul Nelson <ultrono <at> gmail.com>
Cc: Ikumi Keita <ikumi <at> ikumi.que.jp>, 73302-done <at> debbugs.gnu.org
Subject: Re: bug#73302: TeX-find-closing-brace and verbatim macros
Date: Wed, 02 Oct 2024 00:02:11 +0200

Hi Paul,

Paul Nelson <ultrono <at> gmail.com> writes:

> Indeed, the motivation for the bug fix that started this thread was
> that I've been polishing https://github.com/ultronozm/czm-tex-fold.el
> lately, with the aim of sharing it here and pushing it in some form to
> AUCTeX or ELPA (ideally, from my POV, the package would "cease to
> exist").

Yes, pushing the maintainance to others is always a good strategy ;-)

> Besides verbatim environments, the package adds folding support for
> quotes, dashes and begin/end environments (and maybe a couple more
> opinionated defaults of lesser importance).
>
> The folding for verbatim environments that we just discussed could be
> incorporated by something like the attached patch.

Many thanks for the patch, I installed that change (incl. the change
from your next mail).

> The other features could be discussed separately.  Any thoughts would
> be appreciated.

I suggest you prepare a patch for every feature you want to move into
AUCTeX and file a bug report, then we can discuss and install it.  For
now, I'm closing this report.

Best, Arash

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.