GNU bug report logs - #73641
30.0.90; Math in EWW/shr

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 73641 in the body.
You can then email your comments to 73641 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Augusto Stoffel <arstoffel <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.0.90; Math in EWW/shr
Date: Sat, 05 Oct 2024 16:01:36 +0200

shr has a tendency to display garbled math formulas.  For example
opening https://developer.mozilla.org/en-US/docs/Web/MathML/Examples/MathML_Pythagorean_Theorem
in EWW shows this:

> if a and b are the legs and c is the hypotenuse, then a2+b2=c2a^2 +
> b^2 = c^2.

This happens because <math> HTML elements tend to contain a MathML
description of the formula and possibly some "annotations", so what we
see above is "a2+b2=c2", the naive rendering of the MathML, concatenated
with "a^2 + b^2 = c^2" which is an annotation saying what the formula is
in TeX notation.

So in as a first improvement I would like to suggest adding this
definition:

  (defun shr-tag-math (dom)
    (shr-generic (thread-first
                   dom
                   (dom-child-by-tag 'semantics)
                   (dom-child-by-tag 'annotation)
                   (or dom))))

The above page then gets rendered like this:

> Specifically, if a and b are the legs and c is the hypotenuse, then
> a^2 + b^2 = c^2.

In other words, only the TeX annotation is displayed, which is as good
as it gets when trying to show math as plain text, IMO.

It is possible for the <math> element to include a SVG rendition of the
formula.  We could consider displaying that if available (not sure how
common it is, though).  Also, we might consider including infrastructure
to render MathML, but this would require external programs (there is no
reason to reinvent MathJax).  I've done that in one of my packages, see
this commit for the relevant code:

https://github.com/astoff/devdocs.el/commit/04a8793072f1fc7c23f115e2c7ee83ce669b6fa4

(Actually, I added a version of the above to two of my packages already,
so I think it's pretty clear Emacs should have some general
math-rendering infrastructure.)

Message #8 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Augusto Stoffel <arstoffel <at> gmail.com>
Cc: 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Sat, 05 Oct 2024 18:24:55 +0300

> From: Augusto Stoffel <arstoffel <at> gmail.com>
> Date: Sat, 05 Oct 2024 16:01:36 +0200
> 
> It is possible for the <math> element to include a SVG rendition of the
> formula.  We could consider displaying that if available (not sure how
> common it is, though).  Also, we might consider including infrastructure
> to render MathML, but this would require external programs (there is no
> reason to reinvent MathJax).  I've done that in one of my packages, see
> this commit for the relevant code:
> 
> https://github.com/astoff/devdocs.el/commit/04a8793072f1fc7c23f115e2c7ee83ce669b6fa4
> 
> (Actually, I added a version of the above to two of my packages already,
> so I think it's pretty clear Emacs should have some general
> math-rendering infrastructure.)

I agree that it would be good for Emacs to support math rendering, but
we need to find the best way of doing that, and then look for a
library or a program that implements that.  Maybe the first step would
be to conduct a survey of available methods and tools.

Message #11 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Augusto Stoffel <arstoffel <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Thu, 10 Oct 2024 08:41:32 +0200

On Sat,  5 Oct 2024 at 18:24, Eli Zaretskii wrote:

>> From: Augusto Stoffel <arstoffel <at> gmail.com>
>> Date: Sat, 05 Oct 2024 16:01:36 +0200
>> 
>> It is possible for the <math> element to include a SVG rendition of the
>> formula.  We could consider displaying that if available (not sure how
>> common it is, though).  Also, we might consider including infrastructure
>> to render MathML, but this would require external programs (there is no
>> reason to reinvent MathJax).  I've done that in one of my packages, see
>> this commit for the relevant code:
>> 
>> https://github.com/astoff/devdocs.el/commit/04a8793072f1fc7c23f115e2c7ee83ce669b6fa4
>> 
>> (Actually, I added a version of the above to two of my packages already,
>> so I think it's pretty clear Emacs should have some general
>> math-rendering infrastructure.)
>
> I agree that it would be good for Emacs to support math rendering, but
> we need to find the best way of doing that, and then look for a
> library or a program that implements that.  Maybe the first step would
> be to conduct a survey of available methods and tools.

Sure, but my suggestion above for a "baseline" implementation of the
math tag in shr makes sense anyway IMO.  Should I prepare a patch?

Now, about your question, I think there aren't too many options:

- Good old TeX: widely available among (and only among) scientific
  people, gigantic and bloated and slow, no MathML support, excellent at
  rendering TeX, has "shell escape" to worry about.

- LuaTeX: like TeX, but probably can do MathML as well.

- MathJax: needs JavaScript, otherwise can be bundled with Emacs (takes
  just 2MB), supports MathML, supports TeX notation but not arbitrary
  TeX packages, safe by default.

Message #14 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Augusto Stoffel <arstoffel <at> gmail.com>
Cc: 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Thu, 10 Oct 2024 10:43:53 +0300

> From: Augusto Stoffel <arstoffel <at> gmail.com>
> Cc: 73641 <at> debbugs.gnu.org
> Date: Thu, 10 Oct 2024 08:41:32 +0200
> 
> On Sat,  5 Oct 2024 at 18:24, Eli Zaretskii wrote:
> 
> >> From: Augusto Stoffel <arstoffel <at> gmail.com>
> >> Date: Sat, 05 Oct 2024 16:01:36 +0200
> >> 
> >> It is possible for the <math> element to include a SVG rendition of the
> >> formula.  We could consider displaying that if available (not sure how
> >> common it is, though).  Also, we might consider including infrastructure
> >> to render MathML, but this would require external programs (there is no
> >> reason to reinvent MathJax).  I've done that in one of my packages, see
> >> this commit for the relevant code:
> >> 
> >> https://github.com/astoff/devdocs.el/commit/04a8793072f1fc7c23f115e2c7ee83ce669b6fa4
> >> 
> >> (Actually, I added a version of the above to two of my packages already,
> >> so I think it's pretty clear Emacs should have some general
> >> math-rendering infrastructure.)
> >
> > I agree that it would be good for Emacs to support math rendering, but
> > we need to find the best way of doing that, and then look for a
> > library or a program that implements that.  Maybe the first step would
> > be to conduct a survey of available methods and tools.
> 
> Sure, but my suggestion above for a "baseline" implementation of the
> math tag in shr makes sense anyway IMO.  Should I prepare a patch?

I don't think I understand what you are proposing.  What is the
"baseline" implementation for math tags? what will it entail and what
will it allow/enable?

> Now, about your question, I think there aren't too many options:
> 
> - Good old TeX: widely available among (and only among) scientific
>   people, gigantic and bloated and slow, no MathML support, excellent at
>   rendering TeX, has "shell escape" to worry about.
> 
> - LuaTeX: like TeX, but probably can do MathML as well.
> 
> - MathJax: needs JavaScript, otherwise can be bundled with Emacs (takes
>   just 2MB), supports MathML, supports TeX notation but not arbitrary
>   TeX packages, safe by default.

The basic question is what these produce as output, and how can Emacs
display what they produce?  The best alternative for Emacs is to have
a tool that can generate characters, which could then be rendered
using a suitable font.  The (distant) second best is a tool that
generates an image.

Thanks.

Message #17 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Augusto Stoffel <arstoffel <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Thu, 10 Oct 2024 12:49:10 +0200

On Thu, 10 Oct 2024 at 10:43, Eli Zaretskii wrote:

> I don't think I understand what you are proposing.  What is the
> "baseline" implementation for math tags? what will it entail and what
> will it allow/enable?

I refer precisely to the code from the bug's first message, which also
has an example of the result.

>> Now, about your question, I think there aren't too many options:
>> 
>> - Good old TeX: widely available among (and only among) scientific
>>   people, gigantic and bloated and slow, no MathML support, excellent at
>>   rendering TeX, has "shell escape" to worry about.
>> 
>> - LuaTeX: like TeX, but probably can do MathML as well.
>> 
>> - MathJax: needs JavaScript, otherwise can be bundled with Emacs (takes
>>   just 2MB), supports MathML, supports TeX notation but not arbitrary
>>   TeX packages, safe by default.
>
> The basic question is what these produce as output, and how can Emacs
> display what they produce?  The best alternative for Emacs is to have
> a tool that can generate characters, which could then be rendered
> using a suitable font.

MathJax is this tool.  It can render formulas as "HTML with complicated
styling" to resize and move around characters and symbols.  I don't
think Emacs could display such things in general.

>  The (distant) second best is a tool that generates an image.

I think an image with readable text beneath it is quite alright.

Message #20 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Augusto Stoffel <arstoffel <at> gmail.com>
Cc: 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Thu, 10 Oct 2024 14:34:07 +0300

> From: Augusto Stoffel <arstoffel <at> gmail.com>
> Cc: 73641 <at> debbugs.gnu.org
> Date: Thu, 10 Oct 2024 12:49:10 +0200
> 
> On Thu, 10 Oct 2024 at 10:43, Eli Zaretskii wrote:
> 
> > I don't think I understand what you are proposing.  What is the
> > "baseline" implementation for math tags? what will it entail and what
> > will it allow/enable?
> 
> I refer precisely to the code from the bug's first message, which also
> has an example of the result.

Ah, okay.  That's fine, but please add some comment there that we fall
back to the text variant because we don't (yet) know how to do better.

> > The basic question is what these produce as output, and how can Emacs
> > display what they produce?  The best alternative for Emacs is to have
> > a tool that can generate characters, which could then be rendered
> > using a suitable font.
> 
> MathJax is this tool.  It can render formulas as "HTML with complicated
> styling" to resize and move around characters and symbols.  I don't
> think Emacs could display such things in general.

Do you know what is missing for us to be able to display that?

> >  The (distant) second best is a tool that generates an image.
> 
> I think an image with readable text beneath it is quite alright.

Display of inline images in Emacs has quite a few limitations that
make it less desirable for this purpose, IMO.

Thanks.

Message #23 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Jeremy Bryant <jb <at> jeremybryant.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Augusto Stoffel <arstoffel <at> gmail.com>, Sacha Chua <sacha <at> sachachua.com>,
 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Thu, 10 Oct 2024 21:24:31 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Augusto Stoffel <arstoffel <at> gmail.com>
>> Cc: 73641 <at> debbugs.gnu.org
>> Date: Thu, 10 Oct 2024 12:49:10 +0200
>> 
>> On Thu, 10 Oct 2024 at 10:43, Eli Zaretskii wrote:
>> 
>> > I don't think I understand what you are proposing.  What is the
>> > "baseline" implementation for math tags? what will it entail and what
>> > will it allow/enable?
>> 
>> I refer precisely to the code from the bug's first message, which also
>> has an example of the result.
>
> Ah, okay.  That's fine, but please add some comment there that we fall
> back to the text variant because we don't (yet) know how to do better.
>
>> > The basic question is what these produce as output, and how can Emacs
>> > display what they produce?  The best alternative for Emacs is to have
>> > a tool that can generate characters, which could then be rendered
>> > using a suitable font.
>> 
>> MathJax is this tool.  It can render formulas as "HTML with complicated
>> styling" to resize and move around characters and symbols.  I don't
>> think Emacs could display such things in general.
>
> Do you know what is missing for us to be able to display that?
>
>> >  The (distant) second best is a tool that generates an image.
>> 
>> I think an image with readable text beneath it is quite alright.
>
> Display of inline images in Emacs has quite a few limitations that
> make it less desirable for this purpose, IMO.
>
> Thanks.

As this is related to shr and displaying svg images, this reminds me of
a patch of Sacha earlier this year.

Perhaps Sacha you have a suggestion on this?

Message #26 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Augusto Stoffel <arstoffel <at> gmail.com>
To: Jeremy Bryant <jb <at> jeremybryant.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Sacha Chua <sacha <at> sachachua.com>,
 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Sat, 12 Oct 2024 16:24:12 +0200

[Message part 1 (text/plain, inline)]

On Thu, 10 Oct 2024 at 21:24, Jeremy Bryant wrote:

> As this is related to shr and displaying svg images, this reminds me of
> a patch of Sacha earlier this year.
>
> Perhaps Sacha you have a suggestion on this?

Independent of that, I've attached a patch containing the "baseline"
action that can be later refined.

[0001-shr.el-Define-rendering-rule-for-math-tag.patch (text/x-patch, attachment)]

Message #29 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Augusto Stoffel <arstoffel <at> gmail.com>
Cc: jb <at> jeremybryant.net, sacha <at> sachachua.com, 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Sat, 12 Oct 2024 18:07:41 +0300

> From: Augusto Stoffel <arstoffel <at> gmail.com>
> Cc: Eli Zaretskii <eliz <at> gnu.org>,  73641 <at> debbugs.gnu.org,  Sacha Chua
>  <sacha <at> sachachua.com>
> Date: Sat, 12 Oct 2024 16:24:12 +0200
> 
> On Thu, 10 Oct 2024 at 21:24, Jeremy Bryant wrote:
> 
> > As this is related to shr and displaying svg images, this reminds me of
> > a patch of Sacha earlier this year.
> >
> > Perhaps Sacha you have a suggestion on this?
> 
> Independent of that, I've attached a patch containing the "baseline"
> action that can be later refined.

Thanks, but can we please have a comment here explaining why we do
that?

Message #32 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Augusto Stoffel <arstoffel <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jb <at> jeremybryant.net, sacha <at> sachachua.com, 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Sat, 12 Oct 2024 19:13:48 +0200

[Message part 1 (text/plain, inline)]

On Sat, 12 Oct 2024 at 18:07, Eli Zaretskii wrote:

> Thanks, but can we please have a comment here explaining why we do
> that?

How about this?

[0001-shr.el-Define-rendering-rule-for-math-tag.patch (text/x-patch, attachment)]

Message #35 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Augusto Stoffel <arstoffel <at> gmail.com>
Cc: jb <at> jeremybryant.net, sacha <at> sachachua.com, 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Sat, 12 Oct 2024 21:16:35 +0300

> From: Augusto Stoffel <arstoffel <at> gmail.com>
> Cc: jb <at> jeremybryant.net,  73641 <at> debbugs.gnu.org,  sacha <at> sachachua.com
> Date: Sat, 12 Oct 2024 19:13:48 +0200
> 
> On Sat, 12 Oct 2024 at 18:07, Eli Zaretskii wrote:
> 
> > Thanks, but can we please have a comment here explaining why we do
> > that?
> 
> How about this?

LGTM, thanks.

Message #38 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Augusto Stoffel <arstoffel <at> gmail.com>
Cc: jb <at> jeremybryant.net, sacha <at> sachachua.com, 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Sun, 27 Oct 2024 12:25:26 +0200

> From: Augusto Stoffel <arstoffel <at> gmail.com>
> Cc: jb <at> jeremybryant.net,  73641 <at> debbugs.gnu.org,  sacha <at> sachachua.com
> Date: Sat, 12 Oct 2024 19:13:48 +0200
> 
> On Sat, 12 Oct 2024 at 18:07, Eli Zaretskii wrote:
> 
> > Thanks, but can we please have a comment here explaining why we do
> > that?
> 
> How about this?

As I said, this LGTM, but I'm unsure how you'd like to proceed.
Should I install this, and you will follow up with some code which
uses this new function?

Thanks.

Message #41 received at 73641 <at> debbugs.gnu.org (full text, mbox):

From: Augusto Stoffel <arstoffel <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jb <at> jeremybryant.net, sacha <at> sachachua.com, 73641 <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Tue, 29 Oct 2024 07:44:42 +0100

On Sun, 27 Oct 2024 at 12:25, Eli Zaretskii wrote:

>> From: Augusto Stoffel <arstoffel <at> gmail.com>
>> Cc: jb <at> jeremybryant.net,  73641 <at> debbugs.gnu.org,  sacha <at> sachachua.com
>> Date: Sat, 12 Oct 2024 19:13:48 +0200
>> 
>> On Sat, 12 Oct 2024 at 18:07, Eli Zaretskii wrote:
>> 
>> > Thanks, but can we please have a comment here explaining why we do
>> > that?
>> 
>> How about this?
>
> As I said, this LGTM, but I'm unsure how you'd like to proceed.
> Should I install this, and you will follow up with some code which
> uses this new function?

From my perspective it can be installed, and no follow up code is
needed.  At some point shr calls (intern (format "shr-tag-%s" ...)), so
the code that uses the new function is already there.

Message #46 received at 73641-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Augusto Stoffel <arstoffel <at> gmail.com>
Cc: jb <at> jeremybryant.net, sacha <at> sachachua.com, 73641-done <at> debbugs.gnu.org
Subject: Re: bug#73641: 30.0.90; Math in EWW/shr
Date: Thu, 31 Oct 2024 12:21:11 +0200

> From: Augusto Stoffel <arstoffel <at> gmail.com>
> Cc: jb <at> jeremybryant.net,  73641 <at> debbugs.gnu.org,  sacha <at> sachachua.com
> Date: Tue, 29 Oct 2024 07:44:42 +0100
> 
> On Sun, 27 Oct 2024 at 12:25, Eli Zaretskii wrote:
> 
> >> From: Augusto Stoffel <arstoffel <at> gmail.com>
> >> Cc: jb <at> jeremybryant.net,  73641 <at> debbugs.gnu.org,  sacha <at> sachachua.com
> >> Date: Sat, 12 Oct 2024 19:13:48 +0200
> >> 
> >> On Sat, 12 Oct 2024 at 18:07, Eli Zaretskii wrote:
> >> 
> >> > Thanks, but can we please have a comment here explaining why we do
> >> > that?
> >> 
> >> How about this?
> >
> > As I said, this LGTM, but I'm unsure how you'd like to proceed.
> > Should I install this, and you will follow up with some code which
> > uses this new function?
> 
> >From my perspective it can be installed, and no follow up code is
> needed.  At some point shr calls (intern (format "shr-tag-%s" ...)), so
> the code that uses the new function is already there.

Thanks, installed on master, and closing the bug.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.