GNU bug report logs - #22399
org-mode fontification

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 22399 in the body.
You can then email your comments to 22399 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: phillip.lord <at> russet.org.uk (Phillip Lord)
To: bug-gnu-emacs <at> gnu.org
Subject: org-mode fontification
Date: Mon, 18 Jan 2016 13:59:41 +0000

On emacs-25 ox-html, line 1847, org-html-fontify-code does this...


(setq code (with-temp-buffer
		       ;; Switch to language-specific mode.
		       (funcall lang-mode)
		       (insert code)
		       ;; Fontify buffer.
		       (org-font-lock-ensure)


But `org-font-lock-ensure' is defined like so....

(defalias 'org-font-lock-ensure
  (if (fboundp 'org-font-lock-ensure)
      #'font-lock-ensure
    (lambda (_beg _end) (font-lock-fontify-buffer))))


which takes two args, causing crashes when exporting to code to HTML.

Phil

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Rasmus <rasmus <at> gmx.us>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#22399: org-mode fontification
Date: Mon, 18 Jan 2016 15:16:48 +0100

Hi Phillip,

Thanks for your report.

phillip.lord <at> russet.org.uk (Phillip Lord) writes:

> On emacs-25 ox-html, line 1847, org-html-fontify-code does this...
>
> (setq code (with-temp-buffer
> 		       ;; Switch to language-specific mode.
> 		       (funcall lang-mode)
> 		       (insert code)
> 		       ;; Fontify buffer.
> 		       (org-font-lock-ensure)
>
>
> But `org-font-lock-ensure' is defined like so....
>
> (defalias 'org-font-lock-ensure
>   (if (fboundp 'org-font-lock-ensure)
>       #'font-lock-ensure
>     (lambda (_beg _end) (font-lock-fontify-buffer))))
>
>
> which takes two args, causing crashes when exporting to code to HTML.

This is fixed in the most recent stable version of Org, i.e. 8.3.3¹ ,
which, I guess, will be part of if Emacs 25.2 or 26.1 (whichever comes
first and allow new features).  It’s also fixed in 8.2.10².  I guess
Emacs-25 should switch v8.2.10.  [Honestly, when v8.3 couldn’t be merged I
did not cross my mind to upgrade to v8.2.10].  In case v8.2.10 it too
radical, a "local fix" must be applied to the version of Org in emacs.git.

Thanks,
Rasmus


Footnotes: 
¹   http://orgmode.org/cgit.cgi/org-mode.git/tree/lisp/ox-html.el?h=maint#n2086

²   http://orgmode.org/cgit.cgi/org-mode.git/tree/lisp/ox-html.el?id=fdd9b18598239dd0e8b18383ac930110d51ed22e#n1893

-- 
Warning: Everything saved will be lost

Message #11 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: phillip.lord <at> russet.org.uk (Phillip Lord)
To: Rasmus <rasmus <at> gmx.us>
Cc: 22399 <at> debbugs.gnu.org
Subject: Re: bug#22399: org-mode fontification
Date: Mon, 18 Jan 2016 18:05:25 +0000

Rasmus <rasmus <at> gmx.us> writes:

> Hi Phillip,
>
> Thanks for your report.
>
> phillip.lord <at> russet.org.uk (Phillip Lord) writes:
>
>> On emacs-25 ox-html, line 1847, org-html-fontify-code does this...
>>
>> (setq code (with-temp-buffer
>> 		       ;; Switch to language-specific mode.
>> 		       (funcall lang-mode)
>> 		       (insert code)
>> 		       ;; Fontify buffer.
>> 		       (org-font-lock-ensure)
>>
>>
>> But `org-font-lock-ensure' is defined like so....
>>
>> (defalias 'org-font-lock-ensure
>>   (if (fboundp 'org-font-lock-ensure)
>>       #'font-lock-ensure
>>     (lambda (_beg _end) (font-lock-fontify-buffer))))
>>
>>
>> which takes two args, causing crashes when exporting to code to HTML.
>
> This is fixed in the most recent stable version of Org, i.e. 8.3.3¹ ,
> which, I guess, will be part of if Emacs 25.2 or 26.1 (whichever comes
> first and allow new features).  It’s also fixed in 8.2.10².  I guess
> Emacs-25 should switch v8.2.10.  [Honestly, when v8.3 couldn’t be merged I
> did not cross my mind to upgrade to v8.2.10].  In case v8.2.10 it too
> radical, a "local fix" must be applied to the version of Org in emacs.git.


Well, the fix on emacs-25 appears to be trivial i.e.

(org-font-lock-ensure (point-min) (point-max))

But I don't know what the protocol is for org-mode as it's upstream.
AFAICT, this is a regression from emacs-24.5, so it would be a shame if
the bug gets into 25.1.

Let me know if you want me to do anything!

Phil

Message #14 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Rasmus <rasmus <at> gmx.us>
To: phillip.lord <at> russet.org.uk
Cc: 22399 <at> debbugs.gnu.org
Subject: Re: bug#22399: org-mode fontification
Date: Mon, 18 Jan 2016 19:48:17 +0100

phillip.lord <at> russet.org.uk (Phillip Lord) writes:

> Rasmus <rasmus <at> gmx.us> writes:
>
>> Hi Phillip,
>>
>> Thanks for your report.
>>
>> phillip.lord <at> russet.org.uk (Phillip Lord) writes:
>>
>>> On emacs-25 ox-html, line 1847, org-html-fontify-code does this...
>>>
>>> (setq code (with-temp-buffer
>>> 		       ;; Switch to language-specific mode.
>>> 		       (funcall lang-mode)
>>> 		       (insert code)
>>> 		       ;; Fontify buffer.
>>> 		       (org-font-lock-ensure)
>>>
>>>
>>> But `org-font-lock-ensure' is defined like so....
>>>
>>> (defalias 'org-font-lock-ensure
>>>   (if (fboundp 'org-font-lock-ensure)
>>>       #'font-lock-ensure
>>>     (lambda (_beg _end) (font-lock-fontify-buffer))))
>>>
>>>
>>> which takes two args, causing crashes when exporting to code to HTML.
>>
>> This is fixed in the most recent stable version of Org, i.e. 8.3.3¹ ,
>> which, I guess, will be part of if Emacs 25.2 or 26.1 (whichever comes
>> first and allow new features).  It’s also fixed in 8.2.10².  I guess
>> Emacs-25 should switch v8.2.10.  [Honestly, when v8.3 couldn’t be merged I
>> did not cross my mind to upgrade to v8.2.10].  In case v8.2.10 it too
>> radical, a "local fix" must be applied to the version of Org in emacs.git.
>
>
> Well, the fix on emacs-25 appears to be trivial i.e.
>
> (org-font-lock-ensure (point-min) (point-max))
>
> But I don't know what the protocol is for org-mode as it's upstream.
> AFAICT, this is a regression from emacs-24.5, so it would be a shame if
> the bug gets into 25.1.

Sorry, I managed to confuse myself before.  Emacs-25 has Org 8.2.10, but
this bug only lives in the core version.

Isn’t a better fix to change the signature of the alias to mirror that of
font-lock-ensure?  I.e.

    (defalias 'org-font-lock-ensure
      (if (fboundp 'org-font-lock-ensure)
          #'font-lock-ensure
        (lambda (&optional _beg _end) (font-lock-fontify-buffer))))

Since (org-font-lock-ensure) is called from many places.

Note, this change only lives it the Emacs version of Org.  It was added in
this commit:

       6711a21f1125c0047c56eb266eb374c1ec90a967
       Author:     Stefan Monnier <monnier <at> iro.umontreal.ca>
       AuthorDate: Wed May 28 23:45:29 2014 -0400

      * lisp/font-lock.el (font-lock-flush, font-lock-ensure): New functions.
      * lisp/org/org-compat.el (org-font-lock-ensure): New function.
      * lisp/org/ox-odt.el (org-odt-do-format-code):
      * lisp/org/ox-html.el (org-html-fontify-code):
      * lisp/org/org.el (org-fontify-like-in-org-mode):
      * lisp/org/org-src.el (org-src-font-lock-fontify-block):
      * lisp/org/org-clock.el (org-clock-get-clocktable): Use it.
      * lisp/org/ox-org.el (org-org-publish-to-org): Use it.  Avoid using find-file
      from Elisp.
      * test/automated/ruby-mode-tests.el (ruby-assert-face): Use font-lock-ensure.
      (ruby-interpolation-keeps-non-quote-syntax): Use syntax-propertize.


I’ll check on font-lock-ensure business and whether we can use that in
Org-9.

Thanks,
Rasmus

-- 
The right to be left alone is a human right

Message #17 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Kyle Meyer <kyle <at> kyleam.com>
To: phillip.lord <at> russet.org.uk
Cc: 22399 <at> debbugs.gnu.org, Org Mode <emacs-orgmode <at> gnu.org>
Subject: Re: bug#22399: org-mode fontification
Date: Mon, 18 Jan 2016 14:27:04 -0500

Hi Phillip and Rasmus,

Rasmus <rasmus <at> gmx.us> writes:

[...]

> Note, this change only lives it the Emacs version of Org.  It was added in
> this commit:
>
>        6711a21f1125c0047c56eb266eb374c1ec90a967
>        Author:     Stefan Monnier <monnier <at> iro.umontreal.ca>
>        AuthorDate: Wed May 28 23:45:29 2014 -0400
>
>       * lisp/font-lock.el (font-lock-flush, font-lock-ensure): New functions.
>       * lisp/org/org-compat.el (org-font-lock-ensure): New function.
>       * lisp/org/ox-odt.el (org-odt-do-format-code):
>       * lisp/org/ox-html.el (org-html-fontify-code):
>       * lisp/org/org.el (org-fontify-like-in-org-mode):
>       * lisp/org/org-src.el (org-src-font-lock-fontify-block):
>       * lisp/org/org-clock.el (org-clock-get-clocktable): Use it.
>       * lisp/org/ox-org.el (org-org-publish-to-org): Use it.  Avoid using find-file
>       from Elisp.
>       * test/automated/ruby-mode-tests.el (ruby-assert-face): Use font-lock-ensure.
>       (ruby-interpolation-keeps-non-quote-syntax): Use syntax-propertize.

This commit was backported in Org commit d81e6b5, and then followed up
by c115c7b (org-compat: Use different font-lock-ensure alias,
2015-07-26), which defined the alias as

    (unless (fboundp 'font-lock-ensure)
      (defalias 'font-lock-ensure 'font-lock-fontify-buffer))


--
Kyle

Message #20 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Kyle Meyer <kyle <at> kyleam.com>
To: phillip.lord <at> russet.org.uk
Cc: 22399 <at> debbugs.gnu.org, Org Mode <emacs-orgmode <at> gnu.org>
Subject: Re: bug#22399: org-mode fontification
Date: Tue, 19 Jan 2016 01:32:49 -0500

[Message part 1 (text/plain, inline)]

Kyle Meyer <kyle <at> kyleam.com> writes:
> Rasmus <rasmus <at> gmx.us> writes:
>
> [...]
>
>> Note, this change only lives it the Emacs version of Org.  It was added in
>> this commit:
>>
>>        6711a21f1125c0047c56eb266eb374c1ec90a967

[...]

>
> This commit was backported in Org commit d81e6b5, and then followed up
> by c115c7b (org-compat: Use different font-lock-ensure alias,
> 2015-07-26), which defined the alias as
>
>     (unless (fboundp 'font-lock-ensure)
>       (defalias 'font-lock-ensure 'font-lock-fontify-buffer))

I've attached two patches against emacs-25 that would get the Org and
Emacs repos in similar states wrt font-lock-ensure.

The first patch replaces the font-lock-ensure compatibility alias from
6711a21f1 with the one currently used in the Org repo (from commit
e6883dd03).  The new alias is limited to being called with no
arguments, but, at the moment, no Org code calls font-lock-ensure with
any arguments.

The second patch reverts a part of 6711a21f1 that modified
org-src-font-lock-fontify-block to use font-lock-ensure rather than
font-lock-fontify-buffer.  In this particular case, using
font-lock-ensure instead of font-lock-fontify-buffer fails to
highlight the source block.

[0001-org-compat-Use-different-font-lock-ensure-alias.patch (text/x-diff, attachment)]

[0002-org-src-Use-font-lock-fontify-buffer-for-blocks.patch (text/x-diff, attachment)]

[Message part 4 (text/plain, inline)]

--
Kyle

Message #23 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Rasmus <rasmus <at> gmx.us>
Cc: 22399 <at> debbugs.gnu.org
Subject: Re: bug#22399: org-mode fontification
Date: Tue, 19 Jan 2016 09:51:18 +0100

The following message is a courtesy copy of an article
that has been posted to gmane.emacs.orgmode as well.

Kyle Meyer <kyle <at> kyleam.com> writes:

> I've attached two patches against emacs-25 that would get the Org and
> Emacs repos in similar states wrt font-lock-ensure.

Thanks Kyle.

> The first patch replaces the font-lock-ensure compatibility alias from
> 6711a21f1 with the one currently used in the Org repo (from commit
> e6883dd03).  The new alias is limited to being called with no
> arguments, but, at the moment, no Org code calls font-lock-ensure with
> any arguments.

I just have one concern, which is pretty theoretical.  Isn’t it bad to
define the alias to be font-lock-ensure since this is an existing function
and people would have clear expectations of what they are calling?  I
realize that there’s no Emacs-25 that will not have this function (which
is why it’s very theoretical).  Also, if the alias were ever to point to
font-lock-fontify-buffer, the signature would change compared to the
expected signature of font-lock-ensure.

These considerations are probably too theoretical to consider further,
though.

> The second patch reverts a part of 6711a21f1 that modified
> org-src-font-lock-fontify-block to use font-lock-ensure rather than
> font-lock-fontify-buffer.  In this particular case, using
> font-lock-ensure instead of font-lock-fontify-buffer fails to
> highlight the source block.

Cool!

Cheers,
Rasmus

-- 
May contains speling mistake

Message #26 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: phillip.lord <at> russet.org.uk (Phillip Lord)
To: Kyle Meyer <kyle <at> kyleam.com>
Cc: 22399 <at> debbugs.gnu.org, Org Mode <emacs-orgmode <at> gnu.org>
Subject: Re: bug#22399: org-mode fontification
Date: Tue, 19 Jan 2016 11:25:01 +0000

Kyle Meyer <kyle <at> kyleam.com> writes:

> Kyle Meyer <kyle <at> kyleam.com> writes:
>> Rasmus <rasmus <at> gmx.us> writes:
>>
>> [...]
>>
>>> Note, this change only lives it the Emacs version of Org.  It was added in
>>> this commit:
>>>
>>>        6711a21f1125c0047c56eb266eb374c1ec90a967
>
> [...]
>
>>
>> This commit was backported in Org commit d81e6b5, and then followed up
>> by c115c7b (org-compat: Use different font-lock-ensure alias,
>> 2015-07-26), which defined the alias as
>>
>>     (unless (fboundp 'font-lock-ensure)
>>       (defalias 'font-lock-ensure 'font-lock-fontify-buffer))
>
> I've attached two patches against emacs-25 that would get the Org and
> Emacs repos in similar states wrt font-lock-ensure.
>
> The first patch replaces the font-lock-ensure compatibility alias from
> 6711a21f1 with the one currently used in the Org repo (from commit
> e6883dd03).  The new alias is limited to being called with no
> arguments, but, at the moment, no Org code calls font-lock-ensure with
> any arguments.
>
> The second patch reverts a part of 6711a21f1 that modified
> org-src-font-lock-fontify-block to use font-lock-ensure rather than
> font-lock-fontify-buffer.  In this particular case, using
> font-lock-ensure instead of font-lock-fontify-buffer fails to
> highlight the source block.


I've tested these on emacs-25 and can confirm that they solve the
problem.

Thanks!

Phil

Message #29 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Kyle Meyer <kyle <at> kyleam.com>
To: emacs-orgmode <at> gnu.org
Cc: 22399 <at> debbugs.gnu.org
Subject: Re: bug#22399: org-mode fontification
Date: Tue, 19 Jan 2016 10:33:12 -0500

Rasmus <rasmus <at> gmx.us> writes:

> Kyle Meyer <kyle <at> kyleam.com> writes:

[...]

>> The first patch replaces the font-lock-ensure compatibility alias from
>> 6711a21f1 with the one currently used in the Org repo (from commit
>> e6883dd03).  The new alias is limited to being called with no
>> arguments, but, at the moment, no Org code calls font-lock-ensure with
>> any arguments.
>
> I just have one concern, which is pretty theoretical.  Isn’t it bad to
> define the alias to be font-lock-ensure since this is an existing function
> and people would have clear expectations of what they are calling?  I
> realize that there’s no Emacs-25 that will not have this function (which
> is why it’s very theoretical).

I agree, but, like you, I'm not sure it will matter in practice.  Again,
this was the solution that was already in Org's repo at the time I
backported Emacs's 6711a21f1.  However, it's easy enough to switch to
using an org-font-lock-ensure variant that avoids these issues, so I
will do that (in an updated patch and on Org's maint).

--
Kyle

Message #32 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Kyle Meyer <kyle <at> kyleam.com>
To: emacs-orgmode <at> gnu.org
Cc: 22399 <at> debbugs.gnu.org
Subject: Re: bug#22399: org-mode fontification
Date: Tue, 19 Jan 2016 13:04:15 -0500

[Message part 1 (text/plain, inline)]

Kyle Meyer <kyle <at> kyleam.com> writes:

[...]

> However, it's easy enough to switch to using an org-font-lock-ensure
> variant that avoids these issues, so I will do that (in an updated
> patch and on Org's maint).

I've attached updated patches against emacs-25.  If these are applied to
the Emacs repo, I'll backport the new font-lock-ensure alias to Org's
maint.

[0001-org-compat-Fix-font-lock-ensure-alias.patch (text/x-patch, attachment)]

[0002-org-src-Use-font-lock-fontify-buffer-for-blocks.patch (text/x-patch, attachment)]

[Message part 4 (text/plain, inline)]

--
Kyle

Message #35 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Rasmus <rasmus <at> gmx.us>
Cc: 22399 <at> debbugs.gnu.org
Subject: Re: bug#22399: org-mode fontification
Date: Tue, 19 Jan 2016 20:05:32 +0100

The following message is a courtesy copy of an article
that has been posted to gmane.emacs.orgmode as well.

Hi Kyle,

Kyle Meyer <kyle <at> kyleam.com> writes:

>> I just have one concern, which is pretty theoretical.  Isn’t it bad to
>> define the alias to be font-lock-ensure since this is an existing function
>> and people would have clear expectations of what they are calling?  I
>> realize that there’s no Emacs-25 that will not have this function (which
>> is why it’s very theoretical).
>
> I agree, but, like you, I'm not sure it will matter in practice.  Again,
> this was the solution that was already in Org's repo at the time I
> backported Emacs's 6711a21f1.  However, it's easy enough to switch to
> using an org-font-lock-ensure variant that avoids these issues, so I
> will do that (in an updated patch and on Org's maint).

Thanks.

Kyle Meyer <kyle <at> kyleam.com> writes:

>> However, it's easy enough to switch to using an org-font-lock-ensure
>> variant that avoids these issues, so I will do that (in an updated
>> patch and on Org's maint).
>
> I've attached updated patches against emacs-25.  If these are applied to
> the Emacs repo, I'll backport the new font-lock-ensure alias to Org's
> maint.

Looks good, thanks.

Rasmus

-- 
Er du tosset for noge' lårt!

Message #38 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: phillip.lord <at> russet.org.uk (Phillip Lord)
To: Rasmus <rasmus <at> gmx.us>
Cc: 22399 <at> debbugs.gnu.org, emacs-orgmode <at> gnu.org
Subject: Re: [O] bug#22399: org-mode fontification
Date: Wed, 20 Jan 2016 11:39:33 +0000

Rasmus <rasmus <at> gmx.us> writes:
>
> Kyle Meyer <kyle <at> kyleam.com> writes:
>
>>> However, it's easy enough to switch to using an org-font-lock-ensure
>>> variant that avoids these issues, so I will do that (in an updated
>>> patch and on Org's maint).
>>
>> I've attached updated patches against emacs-25.  If these are applied to
>> the Emacs repo, I'll backport the new font-lock-ensure alias to Org's
>> maint.
>
> Looks good, thanks.


Just checking whether on who is going to apply the patches! They don't
seem to be on emacs-25 yet.

Phil

Message #41 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Kyle Meyer <kyle <at> kyleam.com>
To: phillip.lord <at> russet.org.uk (Phillip Lord)
Cc: 22399 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>,
 Rasmus <rasmus <at> gmx.us>
Subject: Re: bug#22399: org-mode fontification
Date: Wed, 20 Jan 2016 12:18:42 -0500

[Message part 1 (text/plain, inline)]

phillip.lord <at> russet.org.uk (Phillip Lord) writes:

> Rasmus <rasmus <at> gmx.us> writes:
>>
>> Kyle Meyer <kyle <at> kyleam.com> writes:
>>
>>>> However, it's easy enough to switch to using an org-font-lock-ensure
>>>> variant that avoids these issues, so I will do that (in an updated
>>>> patch and on Org's maint).
>>>
>>> I've attached updated patches against emacs-25.  If these are applied to
>>> the Emacs repo, I'll backport the new font-lock-ensure alias to Org's
>>> maint.
>>
>> Looks good, thanks.
>
>
> Just checking whether on who is going to apply the patches! They don't
> seem to be on emacs-25 yet.

I have copyright assignment with the FSF (for Org mode contributions),
but I haven't contributed to Emacs directly before.  I'm cc'ing Stefan
because these patches are touching his changes.

I've re-attached the most recent set of patches
(http://permalink.gmane.org/gmane.emacs.bugs/111756).

[0001-org-compat-Fix-font-lock-ensure-alias.patch (text/x-patch, attachment)]

[0002-org-src-Use-font-lock-fontify-buffer-for-blocks.patch (text/x-patch, attachment)]

[Message part 4 (text/plain, inline)]

--
Kyle

Message #44 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Kyle Meyer <kyle <at> kyleam.com>
Cc: 22399 <at> debbugs.gnu.org, Rasmus <rasmus <at> gmx.us>,
 Phillip Lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#22399: org-mode fontification
Date: Wed, 20 Jan 2016 17:02:15 -0500

>  (defalias 'org-font-lock-ensure
> -  (if (fboundp 'org-font-lock-ensure)
> +  (if (fboundp 'font-lock-ensure)
>        #'font-lock-ensure
> -    (lambda (_beg _end) (font-lock-fontify-buffer))))
> +    (lambda (&optional _beg _end) (font-lock-fontify-buffer))))

Looks good.

>  	    (unless (eq major-mode lang-mode) (funcall lang-mode))
> -            (org-font-lock-ensure)
> +	    ;; Avoid `font-lock-ensure', which does not display fonts in
> +	    ;; source block.
> +	    (font-lock-fontify-buffer)

Intuitively, this looks like a wrong fix.  Do you have a reproducible
recipe showing the problem that you're trying to fix?


        Stefan

Message #47 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Kyle Meyer <kyle <at> kyleam.com>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: 22399 <at> debbugs.gnu.org, Rasmus <rasmus <at> gmx.us>,
 Phillip Lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#22399: org-mode fontification
Date: Wed, 20 Jan 2016 19:44:58 -0500

Thanks for having a look.

Stefan Monnier <monnier <at> IRO.UMontreal.CA> writes:

>>  	    (unless (eq major-mode lang-mode) (funcall lang-mode))
>> -            (org-font-lock-ensure)
>> +	    ;; Avoid `font-lock-ensure', which does not display fonts in
>> +	    ;; source block.
>> +	    (font-lock-fontify-buffer)
>
> Intuitively, this looks like a wrong fix.  Do you have a reproducible
> recipe showing the problem that you're trying to fix?

Sure.  With

--8<---------------cut here---------------start------------->8---

#+BEGIN_SRC elisp
  (require 'cl-lib)
  (message "test")
#+END_SRC
--8<---------------cut here---------------end--------------->8---

in /tmp/scratch.org, run

  emacs -Q --eval "(setq org-src-fontify-natively t)" --visit /tmp/scratch.org

Go to the code block and hit C-c ' (org-edit-special), and then exit
with another C-c '.  Without this change, the source block is no longer
highlighted as elisp code when I return to the buffer.

I don't think it's just something odd on my end because, when we
backported 6711a21, a user reported that he lost highlighting in source
blocks.

  http://thread.gmane.org/gmane.emacs.orgmode/99495

--
Kyle

Message #50 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Kyle Meyer <kyle <at> kyleam.com>
Cc: 22399 <at> debbugs.gnu.org, Rasmus <rasmus <at> gmx.us>,
 Phillip Lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#22399: org-mode fontification
Date: Mon, 25 Jan 2016 09:30:37 -0500

> Sure.  With
>
> --8<---------------cut here---------------start------------->8---
>
> #+BEGIN_SRC elisp
>   (require 'cl-lib)
>   (message "test")
> #+END_SRC
> --8<---------------cut here---------------end--------------->8---
>
> in /tmp/scratch.org, run
>
>   emacs -Q --eval "(setq org-src-fontify-natively t)" --visit /tmp/scratch.org
>
> Go to the code block and hit C-c ' (org-edit-special), and then exit
> with another C-c '.  Without this change, the source block is no longer
> highlighted as elisp code when I return to the buffer.

I see the problem.  org-src-font-lock-fontify-block is using buffers
named " org-src-fontification:<major-mode>" in an unusual way:
they're updated via normal buffer modifications, but they're not put in
font-lock-mode, so all the font-lock machinery which tries to keep the
fontification up-to-date is short-circuited, so it triggers a bug in
font-lock-ensure where we made incorrect assumptions.

I installed the patch below into emacs-25, which should fix this
problem,


        Stefan


--- a/lisp/font-lock.el
+++ b/lisp/font-lock.el
@@ -1074,7 +1074,13 @@ font-lock-flush
 
 (defvar font-lock-ensure-function
   (lambda (_beg _end)
-    (unless font-lock-fontified (font-lock-default-fontify-buffer)))
+    (unless font-lock-fontified
+      (font-lock-default-fontify-buffer)
+      (unless font-lock-mode
+        ;; If font-lock is not enabled, we don't have the hooks in place to
+        ;; track modifications, so a subsequent call to font-lock-ensure can't
+        ;; assume that the fontification is still valid.
+        (setq font-lock-fontified nil))))
   "Function to make sure a region has been fontified.
 Called with two arguments BEG and END.")
 

Message #53 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Kyle Meyer <kyle <at> kyleam.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 22399 <at> debbugs.gnu.org, Rasmus <rasmus <at> gmx.us>,
 Phillip Lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#22399: org-mode fontification
Date: Tue, 26 Jan 2016 01:58:59 -0500

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

[...]

> I see the problem.  org-src-font-lock-fontify-block is using buffers
> named " org-src-fontification:<major-mode>" in an unusual way:
> they're updated via normal buffer modifications, but they're not put in
> font-lock-mode, so all the font-lock machinery which tries to keep the
> fontification up-to-date is short-circuited, so it triggers a bug in
> font-lock-ensure where we made incorrect assumptions.
>
> I installed the patch below into emacs-25, which should fix this
> problem,

Thank you.  That seems to work nicely.

Is there anything else I need to do for the first patch?

-- 
Kyle

Message #56 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Kyle Meyer <kyle <at> kyleam.com>
Cc: 22399 <at> debbugs.gnu.org, Rasmus <rasmus <at> gmx.us>,
 Phillip Lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#22399: org-mode fontification
Date: Tue, 26 Jan 2016 08:06:29 -0500

>> I see the problem.  org-src-font-lock-fontify-block is using buffers
>> named " org-src-fontification:<major-mode>" in an unusual way:
>> they're updated via normal buffer modifications, but they're not put in
>> font-lock-mode, so all the font-lock machinery which tries to keep the
>> fontification up-to-date is short-circuited, so it triggers a bug in
>> font-lock-ensure where we made incorrect assumptions.
>> I installed the patch below into emacs-25, which should fix this
>> problem,
> Thank you.  That seems to work nicely.
> Is there anything else I need to do for the first patch?

No, it's good to go.  If you don't have commit rights, I can install it
for you into emacs-25, unless the Org guys prefer to do it via the
Org tree.


        Stefan

Message #59 received at 22399 <at> debbugs.gnu.org (full text, mbox):

From: Kyle Meyer <kyle <at> kyleam.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 22399 <at> debbugs.gnu.org, Rasmus <rasmus <at> gmx.us>,
 Phillip Lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#22399: org-mode fontification
Date: Tue, 26 Jan 2016 10:27:42 -0500

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>> Thank you.  That seems to work nicely.
>> Is there anything else I need to do for the first patch?
>
> No, it's good to go.  If you don't have commit rights, I can install it
> for you into emacs-25, unless the Org guys prefer to do it via the
> Org tree.

No, I don't have commit rights.  I'll update the Org side, but, for
issues that were introduced in the Emacs tree post 24.5, I don't know if
it's preferable to also update Emacs instead of waiting for the changes
to come over with a full sync.

(There's a similar situation in bug#22469.  Some issues were introduced
by changes to Org in the Emacs repo, and these were fixed in the Org
repo after the patch was backported.)

-- 
Kyle

Message #64 received at 22399-done <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Kyle Meyer <kyle <at> kyleam.com>
Cc: 22399-done <at> debbugs.gnu.org, Rasmus <rasmus <at> gmx.us>,
 Phillip Lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#22399: org-mode fontification
Date: Mon, 01 Feb 2016 08:50:11 -0500

> No, I don't have commit rights.  I'll update the Org side, but, for
> issues that were introduced in the Emacs tree post 24.5, I don't know if
> it's preferable to also update Emacs instead of waiting for the changes
> to come over with a full sync.

Looks like there's no Org people following this, so I installed it into
emacs-25 (IOW it should be fixed in the next (25.0.91) pretest).


        Stefan

Message #67 received at 22399-done <at> debbugs.gnu.org (full text, mbox):

From: Kyle Meyer <kyle <at> kyleam.com>
To: Bastien Guerry <bastien.guerry <at> free.fr>
Cc: 22399-done <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>,
 Rasmus <rasmus <at> gmx.us>, Phillip Lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#22399: org-mode fontification
Date: Wed, 03 Feb 2016 01:51:52 -0500

Bastien Guerry <bastien.guerry <at> free.fr> writes:

> Hi Stefan and Kyle,
>
> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>
>>> No, I don't have commit rights.  I'll update the Org side, but, for
>>> issues that were introduced in the Emacs tree post 24.5, I don't know if
>>> it's preferable to also update Emacs instead of waiting for the changes
>>> to come over with a full sync.
>>
>> Looks like there's no Org people following this, so I installed it into
>> emacs-25 (IOW it should be fixed in the next (25.0.91) pretest).
>
> Thanks for installing this -- Kyle, do you confirm you installed the
> change in Org's git repo?

Yes, in a4edee4.

-- 
Kyle

Message #70 received at 22399-done <at> debbugs.gnu.org (full text, mbox):

From: Bastien Guerry <bastien.guerry <at> free.fr>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Kyle Meyer <kyle <at> kyleam.com>, 22399-done <at> debbugs.gnu.org,
 Rasmus <rasmus <at> gmx.us>, Phillip Lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#22399: org-mode fontification
Date: Wed, 03 Feb 2016 07:38:34 +0100

Hi Stefan and Kyle,

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>> No, I don't have commit rights.  I'll update the Org side, but, for
>> issues that were introduced in the Emacs tree post 24.5, I don't know if
>> it's preferable to also update Emacs instead of waiting for the changes
>> to come over with a full sync.
>
> Looks like there's no Org people following this, so I installed it into
> emacs-25 (IOW it should be fixed in the next (25.0.91) pretest).

Thanks for installing this -- Kyle, do you confirm you installed the
change in Org's git repo?

Message #73 received at 22399-done <at> debbugs.gnu.org (full text, mbox):

From: Bastien Guerry <bastien.guerry <at> free.fr>
To: Kyle Meyer <kyle <at> kyleam.com>
Cc: 22399-done <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>,
 Rasmus <rasmus <at> gmx.us>, Phillip Lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#22399: org-mode fontification
Date: Wed, 03 Feb 2016 07:55:50 +0100

Kyle Meyer <kyle <at> kyleam.com> writes:

>> Thanks for installing this -- Kyle, do you confirm you installed the
>> change in Org's git repo?
>
> Yes, in a4edee4.

Thanks!

-- 
 Bastien

Message #76 received at 22399-done <at> debbugs.gnu.org (full text, mbox):

From: phillip.lord <at> russet.org.uk (Phillip Lord)
To: Bastien Guerry <bastien.guerry <at> free.fr>
Cc: Kyle Meyer <kyle <at> kyleam.com>, 22399-done <at> debbugs.gnu.org,
 Rasmus <rasmus <at> gmx.us>, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#22399: org-mode fontification
Date: Thu, 11 Feb 2016 18:18:09 +0000

Bastien Guerry <bastien.guerry <at> free.fr> writes:

> Kyle Meyer <kyle <at> kyleam.com> writes:
>
>>> Thanks for installing this -- Kyle, do you confirm you installed the
>>> change in Org's git repo?
>>
>> Yes, in a4edee4.
>
> Thanks!


And thank you everyone for fixing!

Phil

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.