GNU bug report logs - #69952
[PATCH] Support pdumping compiled queries by dumping their source

Previous Next

Package: emacs;

Reported by: Sergey Vinokurov <serg.foo <at> gmail.com>

Date: Sat, 23 Mar 2024 03:29:01 UTC

Severity: normal

Tags: patch

Done: Yuan Fu <casouri <at> gmail.com>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 69952 in the body.
You can then email your comments to 69952 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 23 Mar 2024 03:29:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Sergey Vinokurov <serg.foo <at> gmail.com>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Sat, 23 Mar 2024 03:29:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sergey Vinokurov <serg.foo <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [PATCH] Support pdumping compiled queries by dumping their source
Date: Sat, 23 Mar 2024 03:27:18 +0000
[Message part 1 (text/plain, inline)]
Hello,

This patch aims to allow more things to be dumped. In particular, 
compiled treesitter queries can be easily dumped by storing their source 
so that they will be recompiled on load.

I noticed that in my config compiled quires are created somewhere which 
prevents me dumping with standard Emacs build. But with this patch I can 
dump successfully and not bother finding out who produced the queries.

It seems like there should be no drawbacks in allowing more things to be 
dumped, please correct me if I'm wrong.

Regards,
Sergey
[pdump.patch (text/x-patch, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 23 Mar 2024 07:10:03 GMT) Full text and rfc822 format available.

Message #8 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Sergey Vinokurov <serg.foo <at> gmail.com>
Cc: 69952 <at> debbugs.gnu.org, Daniel Colascione <dancol <at> dancol.org>,
 Yuan Fu <casouri <at> gmail.com>
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 23 Mar 2024 09:08:21 +0200
> Date: Sat, 23 Mar 2024 03:27:18 +0000
> From: Sergey Vinokurov <serg.foo <at> gmail.com>
> 
> This patch aims to allow more things to be dumped. In particular, 
> compiled treesitter queries can be easily dumped by storing their source 
> so that they will be recompiled on load.
> 
> I noticed that in my config compiled quires are created somewhere which 
> prevents me dumping with standard Emacs build. But with this patch I can 
> dump successfully and not bother finding out who produced the queries.
> 
> It seems like there should be no drawbacks in allowing more things to be 
> dumped, please correct me if I'm wrong.

Thanks.  I added Daniel and Yuan to the discussion, in case they have
comments.

And I have a question: what happens if the pdumper file dumped with
tree-sitter available is loaded by an Emacs session in which
tree-sitter is not available?  That can happen on Windows, for
example, if the tree-sitter library or the grammar library required
for recomputing the query is not available.  I think we need some code
to prevent Emacs from crashing on startup in that case.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 23 Mar 2024 14:11:03 GMT) Full text and rfc822 format available.

Message #11 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Sergey Vinokurov <serg.foo <at> gmail.com>
Cc: 69952 <at> debbugs.gnu.org, dancol <at> dancol.org, casouri <at> gmail.com
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 23 Mar 2024 15:09:30 +0200
> Date: Sat, 23 Mar 2024 12:53:37 +0000
> Cc: 69952 <at> debbugs.gnu.org, Daniel Colascione <dancol <at> dancol.org>,
>  Yuan Fu <casouri <at> gmail.com>
> From: Sergey Vinokurov <serg.foo <at> gmail.com>
> 
> On 23/03/2024 07:08, Eli Zaretskii wrote:
> >> Date: Sat, 23 Mar 2024 03:27:18 +0000
> >> From: Sergey Vinokurov <serg.foo <at> gmail.com>
> >>
> >> This patch aims to allow more things to be dumped. In particular,
> >> compiled treesitter queries can be easily dumped by storing their source
> >> so that they will be recompiled on load.
> >>
> >> I noticed that in my config compiled quires are created somewhere which
> >> prevents me dumping with standard Emacs build. But with this patch I can
> >> dump successfully and not bother finding out who produced the queries.
> >>
> >> It seems like there should be no drawbacks in allowing more things to be
> >> dumped, please correct me if I'm wrong.
> > 
> > Thanks.  I added Daniel and Yuan to the discussion, in case they have
> > comments.
> > 
> > And I have a question: what happens if the pdumper file dumped with
> > tree-sitter available is loaded by an Emacs session in which
> > tree-sitter is not available?  That can happen on Windows, for
> > example, if the tree-sitter library or the grammar library required
> > for recomputing the query is not available.  I think we need some code
> > to prevent Emacs from crashing on startup in that case.
> 
> According to c2ecb08775dc24618de507d2d1ce0f9b0debe17e, treesitter 
> queries are compiled lazily on fist use. When loading dump file in an 
> Emacs with missing grammar library the user should encounter an error on 
> first use of the query.
> 
> Quick testing shows that dump file can be loaded even with grammar 
> definitions removed. Compiled queries are still there in elisp and 
> there's no error during load.

OK, thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 23 Mar 2024 14:43:02 GMT) Full text and rfc822 format available.

Message #14 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Sergey Vinokurov <serg.foo <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 69952 <at> debbugs.gnu.org, Daniel Colascione <dancol <at> dancol.org>,
 Yuan Fu <casouri <at> gmail.com>
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 23 Mar 2024 12:53:37 +0000
On 23/03/2024 07:08, Eli Zaretskii wrote:
>> Date: Sat, 23 Mar 2024 03:27:18 +0000
>> From: Sergey Vinokurov <serg.foo <at> gmail.com>
>>
>> This patch aims to allow more things to be dumped. In particular,
>> compiled treesitter queries can be easily dumped by storing their source
>> so that they will be recompiled on load.
>>
>> I noticed that in my config compiled quires are created somewhere which
>> prevents me dumping with standard Emacs build. But with this patch I can
>> dump successfully and not bother finding out who produced the queries.
>>
>> It seems like there should be no drawbacks in allowing more things to be
>> dumped, please correct me if I'm wrong.
> 
> Thanks.  I added Daniel and Yuan to the discussion, in case they have
> comments.
> 
> And I have a question: what happens if the pdumper file dumped with
> tree-sitter available is loaded by an Emacs session in which
> tree-sitter is not available?  That can happen on Windows, for
> example, if the tree-sitter library or the grammar library required
> for recomputing the query is not available.  I think we need some code
> to prevent Emacs from crashing on startup in that case.

According to c2ecb08775dc24618de507d2d1ce0f9b0debe17e, treesitter 
queries are compiled lazily on fist use. When loading dump file in an 
Emacs with missing grammar library the user should encounter an error on 
first use of the query.

Quick testing shows that dump file can be loaded even with grammar 
definitions removed. Compiled queries are still there in elisp and 
there's no error during load.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 13 Apr 2024 07:43:04 GMT) Full text and rfc822 format available.

Message #17 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: dancol <at> dancol.org, casouri <at> gmail.com
Cc: serg.foo <at> gmail.com, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 13 Apr 2024 10:41:48 +0300
Ping!  Daniel and Yuan, any comments on this proposal?

> Cc: 69952 <at> debbugs.gnu.org, Daniel Colascione <dancol <at> dancol.org>,
>  Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 23 Mar 2024 09:08:21 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> 
> > Date: Sat, 23 Mar 2024 03:27:18 +0000
> > From: Sergey Vinokurov <serg.foo <at> gmail.com>
> > 
> > This patch aims to allow more things to be dumped. In particular, 
> > compiled treesitter queries can be easily dumped by storing their source 
> > so that they will be recompiled on load.
> > 
> > I noticed that in my config compiled quires are created somewhere which 
> > prevents me dumping with standard Emacs build. But with this patch I can 
> > dump successfully and not bother finding out who produced the queries.
> > 
> > It seems like there should be no drawbacks in allowing more things to be 
> > dumped, please correct me if I'm wrong.
> 
> Thanks.  I added Daniel and Yuan to the discussion, in case they have
> comments.
> 
> And I have a question: what happens if the pdumper file dumped with
> tree-sitter available is loaded by an Emacs session in which
> tree-sitter is not available?  That can happen on Windows, for
> example, if the tree-sitter library or the grammar library required
> for recomputing the query is not available.  I think we need some code
> to prevent Emacs from crashing on startup in that case.
> 
> 
> 
> 




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 20 Apr 2024 08:09:02 GMT) Full text and rfc822 format available.

Message #20 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 20 Apr 2024 01:08:15 -0700

> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> Ping!  Daniel and Yuan, any comments on this proposal?
> 
>> Cc: 69952 <at> debbugs.gnu.org, Daniel Colascione <dancol <at> dancol.org>,
>> Yuan Fu <casouri <at> gmail.com>
>> Date: Sat, 23 Mar 2024 09:08:21 +0200
>> From: Eli Zaretskii <eliz <at> gnu.org>
>> 
>>> Date: Sat, 23 Mar 2024 03:27:18 +0000
>>> From: Sergey Vinokurov <serg.foo <at> gmail.com>
>>> 
>>> This patch aims to allow more things to be dumped. In particular, 
>>> compiled treesitter queries can be easily dumped by storing their source 
>>> so that they will be recompiled on load.
>>> 
>>> I noticed that in my config compiled quires are created somewhere which 
>>> prevents me dumping with standard Emacs build. But with this patch I can 
>>> dump successfully and not bother finding out who produced the queries.
>>> 
>>> It seems like there should be no drawbacks in allowing more things to be 
>>> dumped, please correct me if I'm wrong.
>> 
>> Thanks.  I added Daniel and Yuan to the discussion, in case they have
>> comments.
>> 
>> And I have a question: what happens if the pdumper file dumped with
>> tree-sitter available is loaded by an Emacs session in which
>> tree-sitter is not available?  That can happen on Windows, for
>> example, if the tree-sitter library or the grammar library required
>> for recomputing the query is not available.  I think we need some code
>> to prevent Emacs from crashing on startup in that case.

Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.

Yuan



Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 20 Apr 2024 09:34:08 GMT) Full text and rfc822 format available.

Message #23 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 20 Apr 2024 12:32:57 +0300
> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 20 Apr 2024 01:08:15 -0700
> Cc: dancol <at> dancol.org,
>  serg.foo <at> gmail.com,
>  69952 <at> debbugs.gnu.org
> 
> 
> 
> > On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >> 
> >> And I have a question: what happens if the pdumper file dumped with
> >> tree-sitter available is loaded by an Emacs session in which
> >> tree-sitter is not available?  That can happen on Windows, for
> >> example, if the tree-sitter library or the grammar library required
> >> for recomputing the query is not available.  I think we need some code
> >> to prevent Emacs from crashing on startup in that case.
> 
> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.

Giving up: no.  But I think the code which loads the queries from the
pdumper file should be protected from crashing in those cases.  Can
you suggest how to do that?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 20 Apr 2024 22:52:02 GMT) Full text and rfc822 format available.

Message #26 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, Daniel Colascione <dancol <at> dancol.org>,
 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 20 Apr 2024 15:50:36 -0700

> On Apr 20, 2024, at 2:32 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sat, 20 Apr 2024 01:08:15 -0700
>> Cc: dancol <at> dancol.org,
>> serg.foo <at> gmail.com,
>> 69952 <at> debbugs.gnu.org
>> 
>> 
>> 
>>> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>>> 
>>>> And I have a question: what happens if the pdumper file dumped with
>>>> tree-sitter available is loaded by an Emacs session in which
>>>> tree-sitter is not available?  That can happen on Windows, for
>>>> example, if the tree-sitter library or the grammar library required
>>>> for recomputing the query is not available.  I think we need some code
>>>> to prevent Emacs from crashing on startup in that case.
>> 
>> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.
> 
> Giving up: no.  But I think the code which loads the queries from the
> pdumper file should be protected from crashing in those cases.  Can
> you suggest how to do that?

Would Emacs crash? If the dump file just contains the query (which is a string), then Emacs would just compile the query with treesit-query-compile, it could fail (due to the reasons I mentioned), but shouldn’t crash Emacs.

Yuan



Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sun, 21 Apr 2024 04:55:02 GMT) Full text and rfc822 format available.

Message #29 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sun, 21 Apr 2024 07:53:46 +0300
> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 20 Apr 2024 15:50:36 -0700
> Cc: Daniel Colascione <dancol <at> dancol.org>,
>  serg.foo <at> gmail.com,
>  69952 <at> debbugs.gnu.org
> 
> >>> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >>>> 
> >>>> And I have a question: what happens if the pdumper file dumped with
> >>>> tree-sitter available is loaded by an Emacs session in which
> >>>> tree-sitter is not available?  That can happen on Windows, for
> >>>> example, if the tree-sitter library or the grammar library required
> >>>> for recomputing the query is not available.  I think we need some code
> >>>> to prevent Emacs from crashing on startup in that case.
> >> 
> >> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.
> > 
> > Giving up: no.  But I think the code which loads the queries from the
> > pdumper file should be protected from crashing in those cases.  Can
> > you suggest how to do that?
> 
> Would Emacs crash? If the dump file just contains the query (which is a string), then Emacs would just compile the query with treesit-query-compile, it could fail (due to the reasons I mentioned), but shouldn’t crash Emacs.

Can Emacs compile a query if the tree-sitter shared library and/or the
grammar library is not available?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sun, 21 Apr 2024 23:43:02 GMT) Full text and rfc822 format available.

Message #32 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, Daniel Colascione <dancol <at> dancol.org>,
 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sun, 21 Apr 2024 16:41:54 -0700

> On Apr 20, 2024, at 9:53 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sat, 20 Apr 2024 15:50:36 -0700
>> Cc: Daniel Colascione <dancol <at> dancol.org>,
>> serg.foo <at> gmail.com,
>> 69952 <at> debbugs.gnu.org
>> 
>>>>> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>>>>> 
>>>>>> And I have a question: what happens if the pdumper file dumped with
>>>>>> tree-sitter available is loaded by an Emacs session in which
>>>>>> tree-sitter is not available?  That can happen on Windows, for
>>>>>> example, if the tree-sitter library or the grammar library required
>>>>>> for recomputing the query is not available.  I think we need some code
>>>>>> to prevent Emacs from crashing on startup in that case.
>>>> 
>>>> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.
>>> 
>>> Giving up: no.  But I think the code which loads the queries from the
>>> pdumper file should be protected from crashing in those cases.  Can
>>> you suggest how to do that?
>> 
>> Would Emacs crash? If the dump file just contains the query (which is a string), then Emacs would just compile the query with treesit-query-compile, it could fail (due to the reasons I mentioned), but shouldn’t crash Emacs.
> 
> Can Emacs compile a query if the tree-sitter shared library and/or the
> grammar library is not available?

No. You need both tree-sitter library and the grammar library.

Yuan



Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Mon, 22 Apr 2024 05:48:04 GMT) Full text and rfc822 format available.

Message #35 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Mon, 22 Apr 2024 08:47:31 +0300
> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sun, 21 Apr 2024 16:41:54 -0700
> Cc: Daniel Colascione <dancol <at> dancol.org>,
>  serg.foo <at> gmail.com,
>  69952 <at> debbugs.gnu.org
> 
> 
> 
> > On Apr 20, 2024, at 9:53 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > 
> >> From: Yuan Fu <casouri <at> gmail.com>
> >> Date: Sat, 20 Apr 2024 15:50:36 -0700
> >> Cc: Daniel Colascione <dancol <at> dancol.org>,
> >> serg.foo <at> gmail.com,
> >> 69952 <at> debbugs.gnu.org
> >> 
> >>>>> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >>>>>> 
> >>>>>> And I have a question: what happens if the pdumper file dumped with
> >>>>>> tree-sitter available is loaded by an Emacs session in which
> >>>>>> tree-sitter is not available?  That can happen on Windows, for
> >>>>>> example, if the tree-sitter library or the grammar library required
> >>>>>> for recomputing the query is not available.  I think we need some code
> >>>>>> to prevent Emacs from crashing on startup in that case.
> >>>> 
> >>>> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.
> >>> 
> >>> Giving up: no.  But I think the code which loads the queries from the
> >>> pdumper file should be protected from crashing in those cases.  Can
> >>> you suggest how to do that?
> >> 
> >> Would Emacs crash? If the dump file just contains the query (which is a string), then Emacs would just compile the query with treesit-query-compile, it could fail (due to the reasons I mentioned), but shouldn’t crash Emacs.
> > 
> > Can Emacs compile a query if the tree-sitter shared library and/or the
> > grammar library is not available?
> 
> No. You need both tree-sitter library and the grammar library.

That's what I thought.  So starting Emacs in that case will attempt to
call a function from the tree-sitter library, and will segfault,
right?  If so, we should have some protection in the code in
pdumper.c that loads queries, which tests that tree-sitter is
availabale, and if not does something to prevent the segfault, like
not loading the query, perhaps?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Mon, 22 Apr 2024 06:06:04 GMT) Full text and rfc822 format available.

Message #38 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, Daniel Colascione <dancol <at> dancol.org>,
 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sun, 21 Apr 2024 23:04:47 -0700

> On Apr 21, 2024, at 10:47 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sun, 21 Apr 2024 16:41:54 -0700
>> Cc: Daniel Colascione <dancol <at> dancol.org>,
>> serg.foo <at> gmail.com,
>> 69952 <at> debbugs.gnu.org
>> 
>> 
>> 
>>> On Apr 20, 2024, at 9:53 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>> 
>>>> From: Yuan Fu <casouri <at> gmail.com>
>>>> Date: Sat, 20 Apr 2024 15:50:36 -0700
>>>> Cc: Daniel Colascione <dancol <at> dancol.org>,
>>>> serg.foo <at> gmail.com,
>>>> 69952 <at> debbugs.gnu.org
>>>> 
>>>>>>> On Apr 13, 2024, at 12:41 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>>>>>>> 
>>>>>>>> And I have a question: what happens if the pdumper file dumped with
>>>>>>>> tree-sitter available is loaded by an Emacs session in which
>>>>>>>> tree-sitter is not available?  That can happen on Windows, for
>>>>>>>> example, if the tree-sitter library or the grammar library required
>>>>>>>> for recomputing the query is not available.  I think we need some code
>>>>>>>> to prevent Emacs from crashing on startup in that case.
>>>>>> 
>>>>>> Yeah, being able to dump queries is certainly nice. There will be problems if the Emacs session that later loads the query either have a different grammar version loaded, or a difference grammar library for the same language, or outright doesn’t have tree-sitter like Eli mentioned. I don’t know if they are big enough problems to give up dumping queries, though.
>>>>> 
>>>>> Giving up: no.  But I think the code which loads the queries from the
>>>>> pdumper file should be protected from crashing in those cases.  Can
>>>>> you suggest how to do that?
>>>> 
>>>> Would Emacs crash? If the dump file just contains the query (which is a string), then Emacs would just compile the query with treesit-query-compile, it could fail (due to the reasons I mentioned), but shouldn’t crash Emacs.
>>> 
>>> Can Emacs compile a query if the tree-sitter shared library and/or the
>>> grammar library is not available?
>> 
>> No. You need both tree-sitter library and the grammar library.
> 
> That's what I thought.  So starting Emacs in that case will attempt to
> call a function from the tree-sitter library, and will segfault,
> right?  If so, we should have some protection in the code in
> pdumper.c that loads queries, which tests that tree-sitter is
> availabale, and if not does something to prevent the segfault, like
> not loading the query, perhaps?

Right, it can use Ftreesit_available_p (we can define a C equivalent if pdumper can’t call lisp), and decide whether to compile the query or just keep the query as a string, or signal an error?

Yuan





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Mon, 22 Apr 2024 06:20:02 GMT) Full text and rfc822 format available.

Message #41 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Mon, 22 Apr 2024 09:18:49 +0300
> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sun, 21 Apr 2024 23:04:47 -0700
> Cc: Daniel Colascione <dancol <at> dancol.org>,
>  serg.foo <at> gmail.com,
>  69952 <at> debbugs.gnu.org
> 
> >> No. You need both tree-sitter library and the grammar library.
> > 
> > That's what I thought.  So starting Emacs in that case will attempt to
> > call a function from the tree-sitter library, and will segfault,
> > right?  If so, we should have some protection in the code in
> > pdumper.c that loads queries, which tests that tree-sitter is
> > availabale, and if not does something to prevent the segfault, like
> > not loading the query, perhaps?
> 
> Right, it can use Ftreesit_available_p (we can define a C equivalent if pdumper can’t call lisp), and decide whether to compile the query or just keep the query as a string, or signal an error?

We cannot safely signal an error at that point, I think, so just
keeping the query as a string should do, as it will then signal an
error at run time when that query is used, is that right?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Mon, 22 Apr 2024 06:26:02 GMT) Full text and rfc822 format available.

Message #44 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sun, 21 Apr 2024 23:25:10 -0700

> On Apr 21, 2024, at 11:18 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sun, 21 Apr 2024 23:04:47 -0700
>> Cc: Daniel Colascione <dancol <at> dancol.org>,
>> serg.foo <at> gmail.com,
>> 69952 <at> debbugs.gnu.org
>> 
>>>> No. You need both tree-sitter library and the grammar library.
>>> 
>>> That's what I thought.  So starting Emacs in that case will attempt to
>>> call a function from the tree-sitter library, and will segfault,
>>> right?  If so, we should have some protection in the code in
>>> pdumper.c that loads queries, which tests that tree-sitter is
>>> availabale, and if not does something to prevent the segfault, like
>>> not loading the query, perhaps?
>> 
>> Right, it can use Ftreesit_available_p (we can define a C equivalent if pdumper can’t call lisp), and decide whether to compile the query or just keep the query as a string, or signal an error?
> 
> We cannot safely signal an error at that point, I think, so just
> keeping the query as a string should do, as it will then signal an
> error at run time when that query is used, is that right?

Yes, most likely a function-undefined signal, since all the treesit.c functions like treesit-query-capture or treesit-query-compile will be nonexistent. And usually the Lisp program trying to use the query would check for tree-sitter availability with treesit-available-p before trying to use any tree-sitter functions; so that signal will be usually avoided as well.

Yuan



Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 04 May 2024 09:40:02 GMT) Full text and rfc822 format available.

Message #47 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>, serg.foo <at> gmail.com
Cc: 69952 <at> debbugs.gnu.org, dancol <at> dancol.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 04 May 2024 12:39:05 +0300
> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sun, 21 Apr 2024 23:25:10 -0700
> Cc: dancol <at> dancol.org,
>  serg.foo <at> gmail.com,
>  69952 <at> debbugs.gnu.org
> 
> > On Apr 21, 2024, at 11:18 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > 
> > 
> > We cannot safely signal an error at that point, I think, so just
> > keeping the query as a string should do, as it will then signal an
> > error at run time when that query is used, is that right?
> 
> Yes, most likely a function-undefined signal, since all the treesit.c functions like treesit-query-capture or treesit-query-compile will be nonexistent. And usually the Lisp program trying to use the query would check for tree-sitter availability with treesit-available-p before trying to use any tree-sitter functions; so that signal will be usually avoided as well.

Can you suggest such an addition to the patch?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 04 May 2024 21:30:01 GMT) Full text and rfc822 format available.

Message #50 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 4 May 2024 14:29:06 -0700

> On May 4, 2024, at 2:39 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sun, 21 Apr 2024 23:25:10 -0700
>> Cc: dancol <at> dancol.org,
>> serg.foo <at> gmail.com,
>> 69952 <at> debbugs.gnu.org
>> 
>>> On Apr 21, 2024, at 11:18 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>> 
>>> 
>>> We cannot safely signal an error at that point, I think, so just
>>> keeping the query as a string should do, as it will then signal an
>>> error at run time when that query is used, is that right?
>> 
>> Yes, most likely a function-undefined signal, since all the treesit.c functions like treesit-query-capture or treesit-query-compile will be nonexistent. And usually the Lisp program trying to use the query would check for tree-sitter availability with treesit-available-p before trying to use any tree-sitter functions; so that signal will be usually avoided as well.
> 
> Can you suggest such an addition to the patch?

Let me take a look.

Yuan



Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 18 May 2024 08:39:01 GMT) Full text and rfc822 format available.

Message #53 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 18 May 2024 11:38:11 +0300
Ping!

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 4 May 2024 14:29:06 -0700
> Cc: serg.foo <at> gmail.com,
>  dancol <at> dancol.org,
>  69952 <at> debbugs.gnu.org
> 
> 
> 
> > On May 4, 2024, at 2:39 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > 
> >> From: Yuan Fu <casouri <at> gmail.com>
> >> Date: Sun, 21 Apr 2024 23:25:10 -0700
> >> Cc: dancol <at> dancol.org,
> >> serg.foo <at> gmail.com,
> >> 69952 <at> debbugs.gnu.org
> >> 
> >>> On Apr 21, 2024, at 11:18 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >>> 
> >>> 
> >>> We cannot safely signal an error at that point, I think, so just
> >>> keeping the query as a string should do, as it will then signal an
> >>> error at run time when that query is used, is that right?
> >> 
> >> Yes, most likely a function-undefined signal, since all the treesit.c functions like treesit-query-capture or treesit-query-compile will be nonexistent. And usually the Lisp program trying to use the query would check for tree-sitter availability with treesit-available-p before trying to use any tree-sitter functions; so that signal will be usually avoided as well.
> > 
> > Can you suggest such an addition to the patch?
> 
> Let me take a look.
> 
> Yuan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Wed, 22 May 2024 06:39:01 GMT) Full text and rfc822 format available.

Message #56 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Tue, 21 May 2024 23:36:47 -0700

> On May 18, 2024, at 1:38 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> Ping!
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sat, 4 May 2024 14:29:06 -0700
>> Cc: serg.foo <at> gmail.com,
>> dancol <at> dancol.org,
>> 69952 <at> debbugs.gnu.org
>> 
>> 
>> 
>>> On May 4, 2024, at 2:39 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>> 
>>>> From: Yuan Fu <casouri <at> gmail.com>
>>>> Date: Sun, 21 Apr 2024 23:25:10 -0700
>>>> Cc: dancol <at> dancol.org,
>>>> serg.foo <at> gmail.com,
>>>> 69952 <at> debbugs.gnu.org
>>>> 
>>>>> On Apr 21, 2024, at 11:18 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>>>> 
>>>>> 
>>>>> We cannot safely signal an error at that point, I think, so just
>>>>> keeping the query as a string should do, as it will then signal an
>>>>> error at run time when that query is used, is that right?
>>>> 
>>>> Yes, most likely a function-undefined signal, since all the treesit.c functions like treesit-query-capture or treesit-query-compile will be nonexistent. And usually the Lisp program trying to use the query would check for tree-sitter availability with treesit-available-p before trying to use any tree-sitter functions; so that signal will be usually avoided as well.
>>> 
>>> Can you suggest such an addition to the patch?
>> 
>> Let me take a look.
>> 
>> Yuan

Am I missing something? It seems the patch doesn’t include anything about loading a dumped query? I guess that’s the addition you’re talking about? If I want to add a special loader, where should I start?

Yuan



Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Wed, 22 May 2024 12:57:02 GMT) Full text and rfc822 format available.

Message #59 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>, Andrea Corallo <acorallo <at> gnu.org>
Cc: serg.foo <at> gmail.com, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Wed, 22 May 2024 15:55:50 +0300
> From: Yuan Fu <casouri <at> gmail.com>
> Date: Tue, 21 May 2024 23:36:47 -0700
> Cc: serg.foo <at> gmail.com,
>  dancol <at> dancol.org,
>  69952 <at> debbugs.gnu.org
> 
> >>>> Yes, most likely a function-undefined signal, since all the treesit.c functions like treesit-query-capture or treesit-query-compile will be nonexistent. And usually the Lisp program trying to use the query would check for tree-sitter availability with treesit-available-p before trying to use any tree-sitter functions; so that signal will be usually avoided as well.
> >>> 
> >>> Can you suggest such an addition to the patch?
> >> 
> >> Let me take a look.
> >> 
> >> Yuan
> 
> Am I missing something? It seems the patch doesn’t include anything about loading a dumped query?

Looks like that, yes.

> I guess that’s the addition you’re talking about? If I want to add a special loader, where should I start?

I guess in dump_do_emacs_relocation and/or dump_do_dump_relocation?

Alternatively, perhaps it's better to define a hook via
pdumper_do_now_and_after_load_impl or
pdumper_do_now_and_after_late_load_impl, in which case the hook will
be run after loading the dump file.  This sounds easier if you can
access all the loaded queries one by one, instead of catching them
as they are being loaded.

Andrea, can you help Yuan here?  You have dealt with some of this when
you added dumping and restoring of native-compiled CUs, so I guess you
know your way better in pdumper.c.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Wed, 29 May 2024 16:36:02 GMT) Full text and rfc822 format available.

Message #62 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <acorallo <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Yuan Fu <casouri <at> gmail.com>, dancol <at> dancol.org, 69952 <at> debbugs.gnu.org,
 serg.foo <at> gmail.com
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Wed, 29 May 2024 12:35:24 -0400
Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Tue, 21 May 2024 23:36:47 -0700
>> Cc: serg.foo <at> gmail.com,
>>  dancol <at> dancol.org,
>>  69952 <at> debbugs.gnu.org
>> 
>> >>>> Yes, most likely a function-undefined signal, since all the
>> >>> treesit.c functions like treesit-query-capture or
>> >>> treesit-query-compile will be nonexistent. And usually the Lisp
>> >>> program trying to use the query would check for tree-sitter
>> >>> availability with treesit-available-p before trying to use any
>> >>> tree-sitter functions; so that signal will be usually avoided as
>> >>> well.
>> >>> 
>> >>> Can you suggest such an addition to the patch?
>> >> 
>> >> Let me take a look.
>> >> 
>> >> Yuan
>> 
>> Am I missing something? It seems the patch doesn’t include anything about loading a dumped query?
>
> Looks like that, yes.
>
>> I guess that’s the addition you’re talking about? If I want to add a special loader, where should I start?
>
> I guess in dump_do_emacs_relocation and/or dump_do_dump_relocation?

Yes that's correct.

> Alternatively, perhaps it's better to define a hook via
> pdumper_do_now_and_after_load_impl or
> pdumper_do_now_and_after_late_load_impl, in which case the hook will
> be run after loading the dump file.  This sounds easier if you can
> access all the loaded queries one by one, instead of catching them
> as they are being loaded.

Note also that if one decides to execute code at LATE_RELOCS or
VERY_LATE_RELOCS time he has the lisp machinery at disposal (including
allocation), but native code becomes functional only after
VERY_LATE_RELOCS relocs are done (not sure this can impact you here).

On this might be necessary if Yuan goes for the hook and needs native
Lisp code to be working, to add
'pdumper_do_now_and_after_very_late_load_impl' (which I think I didn't
bothered at the time).

  Andrea




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 01 Jun 2024 17:10:01 GMT) Full text and rfc822 format available.

Message #65 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Andrea Corallo <acorallo <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Daniel Colascione <dancol <at> dancol.org>,
 69952 <at> debbugs.gnu.org, serg.foo <at> gmail.com
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 1 Jun 2024 10:07:38 -0700

> On May 29, 2024, at 9:35 AM, Andrea Corallo <acorallo <at> gnu.org> wrote:
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
>>> From: Yuan Fu <casouri <at> gmail.com>
>>> Date: Tue, 21 May 2024 23:36:47 -0700
>>> Cc: serg.foo <at> gmail.com,
>>> dancol <at> dancol.org,
>>> 69952 <at> debbugs.gnu.org
>>> 
>>>>>>> Yes, most likely a function-undefined signal, since all the
>>>>>> treesit.c functions like treesit-query-capture or
>>>>>> treesit-query-compile will be nonexistent. And usually the Lisp
>>>>>> program trying to use the query would check for tree-sitter
>>>>>> availability with treesit-available-p before trying to use any
>>>>>> tree-sitter functions; so that signal will be usually avoided as
>>>>>> well.
>>>>>> 
>>>>>> Can you suggest such an addition to the patch?
>>>>> 
>>>>> Let me take a look.
>>>>> 
>>>>> Yuan
>>> 
>>> Am I missing something? It seems the patch doesn’t include anything about loading a dumped query?
>> 
>> Looks like that, yes.
>> 
>>> I guess that’s the addition you’re talking about? If I want to add a special loader, where should I start?
>> 
>> I guess in dump_do_emacs_relocation and/or dump_do_dump_relocation?
> 
> Yes that's correct.
> 
>> Alternatively, perhaps it's better to define a hook via
>> pdumper_do_now_and_after_load_impl or
>> pdumper_do_now_and_after_late_load_impl, in which case the hook will
>> be run after loading the dump file.  This sounds easier if you can
>> access all the loaded queries one by one, instead of catching them
>> as they are being loaded.
> 
> Note also that if one decides to execute code at LATE_RELOCS or
> VERY_LATE_RELOCS time he has the lisp machinery at disposal (including
> allocation), but native code becomes functional only after
> VERY_LATE_RELOCS relocs are done (not sure this can impact you here).
> 
> On this might be necessary if Yuan goes for the hook and needs native
> Lisp code to be working, to add
> 'pdumper_do_now_and_after_very_late_load_impl' (which I think I didn't
> bothered at the time).
> 
>  Andrea

Thanks to you both! I’ll see what I can come up with.

Yuan





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 15 Jun 2024 08:00:02 GMT) Full text and rfc822 format available.

Message #68 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, acorallo <at> gnu.org, dancol <at> dancol.org,
 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 15 Jun 2024 10:59:22 +0300
Ping!  Any progress there?

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 1 Jun 2024 10:07:38 -0700
> Cc: Eli Zaretskii <eliz <at> gnu.org>,
>  serg.foo <at> gmail.com,
>  Daniel Colascione <dancol <at> dancol.org>,
>  69952 <at> debbugs.gnu.org
> 
> 
> 
> > On May 29, 2024, at 9:35 AM, Andrea Corallo <acorallo <at> gnu.org> wrote:
> > 
> > Eli Zaretskii <eliz <at> gnu.org> writes:
> > 
> >>> From: Yuan Fu <casouri <at> gmail.com>
> >>> Date: Tue, 21 May 2024 23:36:47 -0700
> >>> Cc: serg.foo <at> gmail.com,
> >>> dancol <at> dancol.org,
> >>> 69952 <at> debbugs.gnu.org
> >>> 
> >>>>>>> Yes, most likely a function-undefined signal, since all the
> >>>>>> treesit.c functions like treesit-query-capture or
> >>>>>> treesit-query-compile will be nonexistent. And usually the Lisp
> >>>>>> program trying to use the query would check for tree-sitter
> >>>>>> availability with treesit-available-p before trying to use any
> >>>>>> tree-sitter functions; so that signal will be usually avoided as
> >>>>>> well.
> >>>>>> 
> >>>>>> Can you suggest such an addition to the patch?
> >>>>> 
> >>>>> Let me take a look.
> >>>>> 
> >>>>> Yuan
> >>> 
> >>> Am I missing something? It seems the patch doesn’t include anything about loading a dumped query?
> >> 
> >> Looks like that, yes.
> >> 
> >>> I guess that’s the addition you’re talking about? If I want to add a special loader, where should I start?
> >> 
> >> I guess in dump_do_emacs_relocation and/or dump_do_dump_relocation?
> > 
> > Yes that's correct.
> > 
> >> Alternatively, perhaps it's better to define a hook via
> >> pdumper_do_now_and_after_load_impl or
> >> pdumper_do_now_and_after_late_load_impl, in which case the hook will
> >> be run after loading the dump file.  This sounds easier if you can
> >> access all the loaded queries one by one, instead of catching them
> >> as they are being loaded.
> > 
> > Note also that if one decides to execute code at LATE_RELOCS or
> > VERY_LATE_RELOCS time he has the lisp machinery at disposal (including
> > allocation), but native code becomes functional only after
> > VERY_LATE_RELOCS relocs are done (not sure this can impact you here).
> > 
> > On this might be necessary if Yuan goes for the hook and needs native
> > Lisp code to be working, to add
> > 'pdumper_do_now_and_after_very_late_load_impl' (which I think I didn't
> > bothered at the time).
> > 
> >  Andrea
> 
> Thanks to you both! I’ll see what I can come up with.
> 
> Yuan
> 
> 




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Wed, 19 Jun 2024 06:28:01 GMT) Full text and rfc822 format available.

Message #71 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, Andrea Corallo <acorallo <at> gnu.org>,
 Daniel Colascione <dancol <at> dancol.org>, 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Tue, 18 Jun 2024 23:25:45 -0700

> On Jun 15, 2024, at 12:59 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> Ping!  Any progress there?

Unfortunately I haven’t make much progress. I’ll report back when I do. Thanks for the reminder!

Yuan



Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 06 Jul 2024 07:25:02 GMT) Full text and rfc822 format available.

Message #74 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, acorallo <at> gnu.org, dancol <at> dancol.org,
 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 06 Jul 2024 10:24:32 +0300
> From: Yuan Fu <casouri <at> gmail.com>
> Date: Tue, 18 Jun 2024 23:25:45 -0700
> Cc: Andrea Corallo <acorallo <at> gnu.org>,
>  serg.foo <at> gmail.com,
>  Daniel Colascione <dancol <at> dancol.org>,
>  69952 <at> debbugs.gnu.org
> 
> 
> 
> > On Jun 15, 2024, at 12:59 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > 
> > Ping!  Any progress there?
> 
> Unfortunately I haven’t make much progress. I’ll report back when I do. Thanks for the reminder!

Any news with this?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sat, 20 Jul 2024 09:44:02 GMT) Full text and rfc822 format available.

Message #77 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: casouri <at> gmail.com
Cc: serg.foo <at> gmail.com, acorallo <at> gnu.org, dancol <at> dancol.org,
 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 20 Jul 2024 12:43:11 +0300
Ping!

> Cc: serg.foo <at> gmail.com, acorallo <at> gnu.org, dancol <at> dancol.org,
>  69952 <at> debbugs.gnu.org
> Date: Sat, 06 Jul 2024 10:24:32 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> 
> > From: Yuan Fu <casouri <at> gmail.com>
> > Date: Tue, 18 Jun 2024 23:25:45 -0700
> > Cc: Andrea Corallo <acorallo <at> gnu.org>,
> >  serg.foo <at> gmail.com,
> >  Daniel Colascione <dancol <at> dancol.org>,
> >  69952 <at> debbugs.gnu.org
> > 
> > 
> > 
> > > On Jun 15, 2024, at 12:59 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > > 
> > > Ping!  Any progress there?
> > 
> > Unfortunately I haven’t make much progress. I’ll report back when I do. Thanks for the reminder!
> 
> Any news with this?
> 
> 
> 
> 




Reply sent to Yuan Fu <casouri <at> gmail.com>:
You have taken responsibility. (Sat, 20 Jul 2024 21:08:01 GMT) Full text and rfc822 format available.

Notification sent to Sergey Vinokurov <serg.foo <at> gmail.com>:
bug acknowledged by developer. (Sat, 20 Jul 2024 21:08:02 GMT) Full text and rfc822 format available.

Message #82 received at 69952-done <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: serg.foo <at> gmail.com, Andrea Corallo <acorallo <at> gnu.org>,
 Daniel Colascione <dancol <at> dancol.org>, 69952-done <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sat, 20 Jul 2024 14:06:41 -0700
Hey, sorry for the delay. It took me several attempts, but it’s actually pretty simple once I understand it. I just pushed the original patch plus my patch to master. Dumping query and loading it back now works pretty well! 

Yuan

> On Jul 20, 2024, at 2:43 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> Ping!
> 
>> Cc: serg.foo <at> gmail.com, acorallo <at> gnu.org, dancol <at> dancol.org,
>> 69952 <at> debbugs.gnu.org
>> Date: Sat, 06 Jul 2024 10:24:32 +0300
>> From: Eli Zaretskii <eliz <at> gnu.org>
>> 
>>> From: Yuan Fu <casouri <at> gmail.com>
>>> Date: Tue, 18 Jun 2024 23:25:45 -0700
>>> Cc: Andrea Corallo <acorallo <at> gnu.org>,
>>> serg.foo <at> gmail.com,
>>> Daniel Colascione <dancol <at> dancol.org>,
>>> 69952 <at> debbugs.gnu.org
>>> 
>>> 
>>> 
>>>> On Jun 15, 2024, at 12:59 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>>> 
>>>> Ping!  Any progress there?
>>> 
>>> Unfortunately I haven’t make much progress. I’ll report back when I do. Thanks for the reminder!
>> 
>> Any news with this?
>> 
>> 
>> 
>> 





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#69952; Package emacs. (Sun, 21 Jul 2024 04:31:01 GMT) Full text and rfc822 format available.

Message #85 received at 69952 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: serg.foo <at> gmail.com, acorallo <at> gnu.org, dancol <at> dancol.org,
 69952 <at> debbugs.gnu.org
Subject: Re: bug#69952: [PATCH] Support pdumping compiled queries by dumping
 their source
Date: Sun, 21 Jul 2024 07:30:18 +0300
> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 20 Jul 2024 14:06:41 -0700
> Cc: serg.foo <at> gmail.com,
>  Andrea Corallo <acorallo <at> gnu.org>,
>  Daniel Colascione <dancol <at> dancol.org>,
>  69952-done <at> debbugs.gnu.org
> 
> Hey, sorry for the delay. It took me several attempts, but it’s actually pretty simple once I understand it. I just pushed the original patch plus my patch to master. Dumping query and loading it back now works pretty well! 

Great, thanks.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sun, 18 Aug 2024 11:24:07 GMT) Full text and rfc822 format available.

This bug report was last modified 301 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.