GNU bug report logs - #31750
simplify and tune Emacs stack-related allocation

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 31750 in the body.
You can then email your comments to 31750 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Emacs bugs <bug-gnu-emacs <at> gnu.org>
Subject: simplify and tune Emacs stack-related allocation
Date: Thu, 7 Jun 2018 19:27:52 -0700

[Message part 1 (text/plain, inline)]

While looking into the Emacs garbage collector's stack-overflow problems 
I noticed several opportunities for improving performance and 
simplifying the code in minor ways. I plan to install the attached 
patches to take advantage of these opportunities; although they don't 
address stack overflow, they can be thought of as a first, cleanup step 
toward addressing stack overflow.

Overall these patches trim about 250 lines from the C source code, and 
should improve efficiency by not creating objects on the heap to handle 
stack-related events such as implementing save-excursion. This patch 
also removes primitives like 'free_misc' that ask for trouble by 
second-guessing the garbage collector. I'm filing this as a bug report 
in order to give the patches a bit of time to cool before installing. 
Comments welcome, as usual.

[0001-New-mint_ptr-representation-for-C-pointers.patch (text/x-patch, attachment)]

[0002-Simplify-init_module_assertions.patch (text/x-patch, attachment)]

[0003-Avoid-Lisp_Misc-allocation-if-C-stack-suffices.patch (text/x-patch, attachment)]

[0004-Use-record_unwind_protect_ptr-to-avoid-allocation.patch (text/x-patch, attachment)]

[0005-Avoid-allocating-a-Lisp_Save_Value-in-ftfont.c.patch (text/x-patch, attachment)]

[0006-Just-use-cons-in-macfont_descriptor_entity.patch (text/x-patch, attachment)]

[0007-Avoid-allocating-Lisp_Save_Value-for-excursions.patch (text/x-patch, attachment)]

[0008-Avoid-allocating-Lisp_Save_Value-for-arrays.patch (text/x-patch, attachment)]

[0009-New-type-Lisp_Misc_Ptr.patch (text/x-patch, attachment)]

[0010-Remove-Lisp_Misc_Save_Value.patch (text/x-patch, attachment)]

Message #10 received at 31750 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>, 31750 <at> debbugs.gnu.org
Subject: Re: bug#31750: simplify and tune Emacs stack-related allocation
Date: Thu, 7 Jun 2018 20:20:59 -0700

On 06/07/2018 07:27 PM, Paul Eggert wrote:
> While looking into the Emacs garbage collector's stack-overflow problems 
> I noticed several opportunities for improving performance and 
> simplifying the code in minor ways. I plan to install the attached 
> patches to take advantage of these opportunities; although they don't 
> address stack overflow,

One can also dream of the lisp evaluator doing escape analysis and stack 
allocation.

> they can be thought of as a first, cleanup step 
> toward addressing stack overflow.

Are you thinking along the same lines I was thinking of in my proposal 
to address GC safety?

> Overall these patches trim about 250 lines from the C source code, and 
> should improve efficiency by not creating objects on the heap to handle 
> stack-related events such as implementing save-excursion. This patch 
> also removes primitives like 'free_misc' that ask for trouble by 
> second-guessing the garbage collector. I'm filing this as a bug report 
> in order to give the patches a bit of time to cool before installing. 
> Comments welcome, as usual.
> 

Message #13 received at 31750 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Daniel Colascione <dancol <at> dancol.org>, 31750 <at> debbugs.gnu.org
Subject: Re: bug#31750: simplify and tune Emacs stack-related allocation
Date: Fri, 8 Jun 2018 08:42:50 -0700

Daniel Colascione wrote:
> Are you thinking along the same lines I was thinking of in my proposal to 
> address GC safety?

Quite possibly, though I expect my thoughts are along more-gradual and 
more-modest lines. Can you remind me what that proposal is, with a bug# or a URL 
or something? I remember our discussion in 2016 about signal handling, but I 
don't recall exactly what the proposed fixes were, other than to eliminate the 
longjmp when the C stack overflows.

Message #16 received at 31750 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 31750 <at> debbugs.gnu.org
Subject: Re: bug#31750: simplify and tune Emacs stack-related allocation
Date: Sat, 09 Jun 2018 23:30:30 -0400

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> Subject: [PATCH 01/10] New mint_ptr representation for C pointers
>
> * src/lisp.h (make_mint_ptr, mint_ptrp, xmint_pointer): New functions.

> Use mint pointers instead of merely save pointers.

I think it would be helpful to add a brief summary of the tradeoffs
between save pointers and mint pointers to the commit message.


> +/* A mint_ptr object OBJ represents a pointer P.  OBJ is preferably a
> +   Lisp integer I such that XINTPTR (i) == P, as this is more efficient.
                   ^                    ^
                   I guess one of these is in the wrong case.

Message #19 received at 31750 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 31750 <at> debbugs.gnu.org
Subject: Re: bug#31750: simplify and tune Emacs stack-related allocation
Date: Sat, 9 Jun 2018 21:03:25 -0700

Noam Postavsky wrote:
> Paul Eggert <eggert <at> cs.ucla.edu> writes:
> 
>> Subject: [PATCH 01/10] New mint_ptr representation for C pointers
>>
>> * src/lisp.h (make_mint_ptr, mint_ptrp, xmint_pointer): New functions.
> 
>> Use mint pointers instead of merely save pointers.
> 
> I think it would be helpful to add a brief summary of the tradeoffs
> between save pointers and mint pointers to the commit message.

Thanks for looking at it. Here's a quick attempt at a summary comment for 
make_mint_ptr. This wording compares mint_ptr values to Lisp_User_Ptr values, 
not to save pointers, in order simplify the later patch that removes save pointers.

/* A mint_ptr object OBJ represents a C-language pointer P efficiently.
   Preferably (and typically), OBJ is a Lisp integer I such that
   XINTPTR (I) == P, as this represents P within a single Lisp value
   without requiring any auxiliary memory.  However, if P would be
   damaged by being tagged as an integer and then untagged via
   XINTPTR, then OBJ is a Lisp_Save_Value with pointer component P.

   mint_ptr objects are efficiency hacks that are intended for use
   only within C code.  C code should never blindly accept a mint_ptr
   object from Lisp code, as that would allow Lisp code to coin
   pointers from integers and could lead to crashes.  A C pointer that
   needs to be packaged into a Lisp-visible object can be made part of
   a Lisp_Misc type instead; see Lisp_User_Ptr for an example.  */

Message #22 received at 31750 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: npostavs <at> gmail.com, 31750 <at> debbugs.gnu.org
Subject: Re: bug#31750: simplify and tune Emacs stack-related allocation
Date: Sun, 10 Jun 2018 18:07:23 +0300

> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Sat, 9 Jun 2018 21:03:25 -0700
> Cc: 31750 <at> debbugs.gnu.org
> 
> /* A mint_ptr object OBJ represents a C-language pointer P efficiently.
>     Preferably (and typically), OBJ is a Lisp integer I such that
>     XINTPTR (I) == P, as this represents P within a single Lisp value
>     without requiring any auxiliary memory.  However, if P would be
>     damaged by being tagged as an integer and then untagged via
>     XINTPTR, then OBJ is a Lisp_Save_Value with pointer component P.
> 
>     mint_ptr objects are efficiency hacks that are intended for use
>     only within C code.  C code should never blindly accept a mint_ptr
>     object from Lisp code, as that would allow Lisp code to coin
>     pointers from integers and could lead to crashes.  A C pointer that
>     needs to be packaged into a Lisp-visible object can be made part of
>     a Lisp_Misc type instead; see Lisp_User_Ptr for an example.  */

I'm not sure I agree with the last part, at least not in such
categorical terms.  E.g., w32notify.c does exactly what the above says
not to do, and AFAICT is a very good candidate for using mint_ptr
objects.

OTOH, maybe I do agree, as long as "blindly" is an essential part of
that rule ;-)

Message #25 received at 31750 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: npostavs <at> gmail.com, 31750 <at> debbugs.gnu.org
Subject: Re: bug#31750: simplify and tune Emacs stack-related allocation
Date: Sun, 10 Jun 2018 09:12:33 -0700

Eli Zaretskii wrote:
> OTOH, maybe I do agree, as long as "blindly" is an essential part of
> that rule ;-)

Yes, the "blindly" is essential. It's OK to use xmint_ptr on any mint_ptr, so 
long as you don't blindly trust the result to be a valid C pointer of the kind 
that you want; that is, you have to know that your C code created the mint_ptr 
in question and that no Lisp code can have replaced the mint_ptr with some other 
object (perhaps also a mint_ptr). This is like XSAVE_POINTER, where you also 
have to know that Lisp code has not replaced the Lisp_Save_Value object with 
some other object (perhaps also of type Lisp_Save_Value and with a pointer payload).

The main difference is that Lisp code can easily coin a mint_ptr (simply by 
using a fixnum) whereas it can't easily coin a Lisp_Save_Value, so there's less 
runtime checking to catch bugs in the C code. However, if two or more 
Lisp_Save_Value objects are exposed to Lisp code then the Lisp code can pass one 
where the other is expected and this can cause the same sort of crash as passing 
a fixnum where a mint_ptr is expected, so introducing mint_ptr doesn't make 
things that much more dangerous in principle than they were before.

Message #30 received at 31750-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 31750-done <at> debbugs.gnu.org
Subject: Re: bug#31750: simplify and tune Emacs stack-related allocation
Date: Fri, 15 Jun 2018 00:46:58 -0700

I installed the patches into master after tweaking the mint_ptr comments a bit, 
and am marking this as done.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.