From debbugs-submit-bounces@debbugs.gnu.org Sat May 07 16:05:47 2022 Received: (at submit) by debbugs.gnu.org; 7 May 2022 20:05:47 +0000 Received: from localhost ([127.0.0.1]:52994 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nnQgV-0001wo-4D for submit@debbugs.gnu.org; Sat, 07 May 2022 16:05:47 -0400 Received: from lists.gnu.org ([209.51.188.17]:38972) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nnQgS-0001wg-T8 for submit@debbugs.gnu.org; Sat, 07 May 2022 16:05:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60048) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nnQgS-0005Q6-PZ for bug-gnu-emacs@gnu.org; Sat, 07 May 2022 16:05:44 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:43434) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nnQgS-0001Bq-Gw for bug-gnu-emacs@gnu.org; Sat, 07 May 2022 16:05:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=To:Subject:Date:From:MIME-Version:in-reply-to: references; bh=BDXrmO3+Lun2MfJ+as+hYSqcYu+zih8KyzJqNgaKS2g=; b=rA8dNfsh2e97Ef +0N/amy7uvimcqWNVr4F1yHRrzEDGEohpHgqFT3ynbELilYU8/77Gnxb/LJydK5Ol+qYti3GYvXAE CH/sjHAn6VXstYmcZ6t8lBVcv2TMTqIjhc52R7+MJOM+e3mo4iH6+HnY3Evkx7lxKlWJeVXLb3eL6 NysWGqI9oNtd8nwqXxMnNWsxlKc8sD7pnFfhc5Xr/r7BFf1xCQ4OEFS0LHeKKlMIwuT/815ezhkHJ Ft+5JmABiWFBX6uGw8hSKIh354eDYLCU7GjhHUo+8UsjNZVFRj/RUF/eupyXOT2Iu3AyhIwic7H47 WlzeAdWNySZv3ulG/Ylw==; Received: from mail-vk1-f176.google.com ([209.85.221.176]:41981) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nnQgR-0005jM-QP for bug-gnu-emacs@gnu.org; Sat, 07 May 2022 16:05:44 -0400 Received: by mail-vk1-f176.google.com with SMTP id y27so5140905vkl.8 for ; Sat, 07 May 2022 13:05:43 -0700 (PDT) X-Gm-Message-State: AOAM5331VIlrwz0FvDFjVT27mZgTxZ7Me8zmdT2e+QPOy7OKV6Op5vIu +ru0xjzxNo3qQZQ+kLwxCUe4hFSDoXBOAhqDWh8= X-Google-Smtp-Source: ABdhPJxC7yv57ySgAyqCUuaIrBPON/3s5JMbpMSa8sFSS7bLzBoMgtgC74J8k+2Xyjc/5l3LGH/NaJbKUhbsUUkMoII= X-Received: by 2002:ac5:cb64:0:b0:351:c285:68fa with SMTP id l4-20020ac5cb64000000b00351c28568famr5414695vkn.6.1651953943269; Sat, 07 May 2022 13:05:43 -0700 (PDT) MIME-Version: 1.0 From: Robert Weiner Date: Sat, 7 May 2022 16:05:17 -0400 X-Gmail-Original-Message-ID: Message-ID: Subject: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation To: bug-gnu-emacs@gnu.org Content-Type: multipart/alternative; boundary="000000000000f9560e05de717f0c" X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: rswgnu@gmail.com Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --000000000000f9560e05de717f0c Content-Type: text/plain; charset="UTF-8" Tested under Emacs 28.1 and a recent tip of the Emacs git repo for Emacs 29 with asynchronous native compilation enabled: M-x package-install RET hyperbole RET fails to load the hyperbole-autoloads.el file before the async native compiler and byte compiler produce these errors since the autoloaded var:append function is not defined: Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-em-but.el: Error: Symbol's function definition is void var:append Disable showing Disable logging Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-mouse.el: Error: Symbol's function definition is void var:append Disable showing Disable logging Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hbut.el: Error: Symbol's function definition is void var:append Disable showing Disable logging The package manager definitely generates hyperbole-autoloads.el at some point though I do not know if it is before these errors are produced. Thanks for any help with this. -- Bob --000000000000f9560e05de717f0c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Tested under Emacs 28.1 and a recent tip of the Emacs git repo= for Emacs
29 with asynchronous native compilation enabled:

M-x p= ackage-install RET hyperbole RET

fails to load the hyperbole-autoloa= ds.el file before the
async native compiler and byte compiler produce th= ese errors since
the autoloaded var:append function is not defined:
<= br>Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-em-but.el: Error: Sy= mbol's function definition is void var:append Disable showing Disable l= ogging
Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-mouse.el: Err= or: Symbol's function definition is void var:append Disable showing Dis= able logging
Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hbut.el: Er= ror: Symbol's function definition is void var:append Disable showing Di= sable logging

The package manager definitely generates hyperbole-aut= oloads.el at some
point though I do not know if it is before these error= s are produced.

Thanks for any help with this.

-- Bob
--000000000000f9560e05de717f0c-- From debbugs-submit-bounces@debbugs.gnu.org Sun May 08 01:09:56 2022 Received: (at 55305) by debbugs.gnu.org; 8 May 2022 05:09:56 +0000 Received: from localhost ([127.0.0.1]:53245 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nnZB2-000106-Us for submit@debbugs.gnu.org; Sun, 08 May 2022 01:09:56 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35056) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nnZB1-0000zs-Vq for 55305@debbugs.gnu.org; Sun, 08 May 2022 01:09:52 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49884) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nnZAw-00062X-K7; Sun, 08 May 2022 01:09:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=Pn22fPbh6OR/79to2u/ZjBH70k/tidAcJ19p55cT4Tw=; b=Qrw+PxyYjoxj inl77Pug+ggpKi11EbkPUxX+Pg2iPdRC5Pr2wY0ccpYwCegyhsJ8UjnrBZVtZf6DYsexa5/42qxUW 6bdj5n6612lE000ug6u6AOyGvarRiUmBnONjh8n+ycemoEW56kksnCfl5N4AaotpGrHUriQIOJImj zMoe7ce3deYOlWADRgBOQ8Yniw6IrwgyiX5f2Y4VL1lY+tZujdciU/MeO1tyk9Lv21rRy1EXrTibg SpAmyNErDocgxPtzofLviUHgAdEGFdTagNiYPqjj7FfQ3Oc1tBe2SGDX0fBASFGvKQ2JzwY7BGoqg gFRQ/NUmGDjs9a0L38XEZw==; Received: from [87.69.77.57] (port=2566 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nnZAl-00036i-Mt; Sun, 08 May 2022 01:09:44 -0400 Date: Sun, 08 May 2022 08:09:28 +0300 Message-Id: <8335hky5iv.fsf@gnu.org> From: Eli Zaretskii To: rswgnu@gmail.com In-Reply-To: (message from Robert Weiner on Sat, 7 May 2022 16:05:17 -0400) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: 55305@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Robert Weiner > Date: Sat, 7 May 2022 16:05:17 -0400 > > Tested under Emacs 28.1 and a recent tip of the Emacs git repo for Emacs > 29 with asynchronous native compilation enabled: > > M-x package-install RET hyperbole RET > > fails to load the hyperbole-autoloads.el file before the > async native compiler and byte compiler produce these errors since > the autoloaded var:append function is not defined: Hyperbole is not part of Emacs, so this problem should first be taken up with the Hyperbole developers. > Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-em-but.el: Error: Symbol's function definition is void > var:append Disable showing Disable logging > Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-mouse.el: Error: Symbol's function definition is void > var:append Disable showing Disable logging > Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hbut.el: Error: Symbol's function definition is void > var:append Disable showing Disable logging > > The package manager definitely generates hyperbole-autoloads.el at some > point though I do not know if it is before these errors are produced. Do the *.el files that produce the error 'require' or 'load' hyperbole-autoloads? If not, how would compilation know to load that file? Asynchronous native compilation runs in a separate pristine Emacs session, so it needs every dependency explicitly spelled out, or it will fail. But again, I think this is a matter for the Hyperbole developers to look into first. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sun May 08 07:45:35 2022 Received: (at control) by debbugs.gnu.org; 8 May 2022 11:45:35 +0000 Received: from localhost ([127.0.0.1]:53511 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nnfLy-0004wW-U7 for submit@debbugs.gnu.org; Sun, 08 May 2022 07:45:35 -0400 Received: from quimby.gnus.org ([95.216.78.240]:57966) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nnfLx-0004wI-A3 for control@debbugs.gnu.org; Sun, 08 May 2022 07:45:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Subject:From:To:Message-Id:Date:Sender:Reply-To:Cc: MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=74r/fGVDedcKF0CoECMj29Ymak7vpA1ZRlUCrJNAmn8=; b=BBQFWyq/Z3EDk1iMSnJjQvbwFq sKtCYG8FESVvnFp6nTDYMZ7tz8GSiyTZI8Z5m/tbimUaj1uHt7YXhZAO476q67KsVKWlx28BZogsD YByyjeGSdkYVGjwdPJ6AAIWQwcA+Ym6JojFGqOdsT2Vi3k90v5RmM6jimQZb/3A2ETY0=; Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nnfLp-0007tq-K2 for control@debbugs.gnu.org; Sun, 08 May 2022 13:45:27 +0200 Date: Sun, 08 May 2022 13:45:25 +0200 Message-Id: <87tua0jlii.fsf@gnus.org> To: control@debbugs.gnu.org From: Lars Ingebrigtsen Subject: control message for bug #55305 X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: tags 55305 + moreinfo quit Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) tags 55305 + moreinfo quit From debbugs-submit-bounces@debbugs.gnu.org Thu May 12 01:15:33 2022 Received: (at 55305) by debbugs.gnu.org; 12 May 2022 05:15:33 +0000 Received: from localhost ([127.0.0.1]:39172 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1np1Ai-0006AF-Lr for submit@debbugs.gnu.org; Thu, 12 May 2022 01:15:33 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59522) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1np1Ag-00069w-S4 for 55305@debbugs.gnu.org; Thu, 12 May 2022 01:15:31 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57036) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1np1Ab-0002Wn-M7 for 55305@debbugs.gnu.org; Thu, 12 May 2022 01:15:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=To:Subject:Date:From:In-Reply-To:References: MIME-Version; bh=xutqzU5ZR05ChkkxCxxgExJRdYYliKsNcvdAk4SsLuY=; b=Xx9rWuUgRroE 6HZJzFIDbB0Gz4CHEcDDodS2jfASgjiM5uj1DRoxdBHb1mc77moI6Kpz1IqQiTcZsxyfz111rliz6 F5HPR9n3eQvX7bOZArIrZdi+oH04thT/XtxG8YJ8URwXK2yoRpkX8Mh7UDDSnglDdiByTacrpAwrR r0hrSF4yR6pUBb27u5w/0lfK1kPLkjzRFUnO2G4ZfBIxoF1J7etU+n+T3ALeV4xl1NW3XYK3Z7UY8 CMiUzH2R2Z/C2ATKeiSvPmpE1CNkn5zX3luLPVOfvKq4jryY0gD4nGn0eXx0unx65Yt5cWS/4Qyws 6yUew9ZC65geBcSahT8dBw==; Received: from mail-vk1-f169.google.com ([209.85.221.169]:42743) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1np1Aa-00037Q-R7 for 55305@debbugs.gnu.org; Thu, 12 May 2022 01:15:24 -0400 Received: by mail-vk1-f169.google.com with SMTP id e144so2128499vke.9 for <55305@debbugs.gnu.org>; Wed, 11 May 2022 22:15:24 -0700 (PDT) X-Gm-Message-State: AOAM531mNkfxpPZsuI9MsD3g9v7QRgpL1DgxcVBWMMfnu+1qm6nNjGoD cnEJWgMObfKAJLuirertyhNrw355Uwxo/EEIzGI= X-Google-Smtp-Source: ABdhPJwyOB30yNLPoSbrMLELg9DhWVQiKrC7GY2T9NMwt0gsLgZ7iAqUEp6Bi+aqVggbELBxM6fFObNbqUlls+ynzDQ= X-Received: by 2002:a05:6122:a12:b0:351:c28f:674 with SMTP id 18-20020a0561220a1200b00351c28f0674mr15365990vkn.3.1652332524238; Wed, 11 May 2022 22:15:24 -0700 (PDT) MIME-Version: 1.0 References: <8335hky5iv.fsf@gnu.org> In-Reply-To: <8335hky5iv.fsf@gnu.org> From: Robert Weiner Date: Thu, 12 May 2022 01:14:58 -0400 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation To: Eli Zaretskii Content-Type: multipart/alternative; boundary="000000000000287c4b05dec9a5c1" X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 55305 Cc: 55305@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: rswgnu@gmail.com Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --000000000000287c4b05dec9a5c1 Content-Type: text/plain; charset="UTF-8" On Sun, May 8, 2022 at 1:09 AM Eli Zaretskii wrote: > > From: Robert Weiner > > Date: Sat, 7 May 2022 16:05:17 -0400 > > > > Tested under Emacs 28.1 and a recent tip of the Emacs git repo for Emacs > > 29 with asynchronous native compilation enabled: > > > > M-x package-install RET hyperbole RET > > > > fails to load the hyperbole-autoloads.el file before the > > async native compiler and byte compiler produce these errors since > > the autoloaded var:append function is not defined: > > Hyperbole is not part of Emacs, so this problem should first be taken > up with the Hyperbole developers. > Hi Eli: Thanks for the response. Two initial points: 1. I am the lead Hyperbole developer, so I will discuss it with myself :-) 2. I long ago was told that Elpa packages are considered part of Emacs although they don't ship with it; this is why you see a number of Hyperbole issues brought up on this list. It would be good if you and other core Emacs developers all held the same yes/no opinion on this so others could follow whatever is decided. > > Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-em-but.el: Error: > Symbol's function definition is void > > var:append Disable showing Disable logging > > Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-mouse.el: Error: > Symbol's function definition is void > > var:append Disable showing Disable logging > > Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hbut.el: Error: Symbol's > function definition is void > > var:append Disable showing Disable logging > > > > The package manager definitely generates hyperbole-autoloads.el at some > > point though I do not know if it is before these errors are produced. > > Do the *.el files that produce the error 'require' or 'load' > hyperbole-autoloads? No, the Emacs package manager installation process generates that file from the ;;;###autoload annotations in Hyperbole lisp files (see the 'package--make-autoloads-and-stuff' function in "package.el"). The package manager also sets up at package activation time to have those autoloads loaded prior to loading any other code from the Hyperbole package. It would not make sense to require hyperbole-autoloads in other Lisp files since that file does not exist when the package distribution is generated and cannot be referenced prior to making a release; it would also defeat the purpose of autoloads. > If not, how would compilation know to load that > file? Asynchronous native compilation runs in a separate pristine > Emacs session, so it needs every dependency explicitly spelled out, or > it will fail. > This is the issue that I am bringing up that I am surprised does not affect or has not been reported for other packages. There needs to be a persistent, cross-session Emacs hook that runs prior to native compilation of packages that loads the autoloads file for the package. > But again, I think this is a matter for the Hyperbole developers to > look into first. > I have resolved this for Hyperbole in an upcoming pre-release by moving a number of previously autoloaded definitions into a file that can be 'required' by each Hyperbole module instead. This was a good bit of work and does not address the more general problem. I hope you and others will consider this a bit more and look into how it can be resolved for the good of all packages, as it seems to be a disconnect between the Emacs package manager and the native compilation code. Best regards, Bob --000000000000287c4b05dec9a5c1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sun, May 8, 2022 at 1:09 AM Eli Zaretskii <eliz@gnu.org> wrote:
> Fr= om: Robert Weiner <rsw@= gnu.org>
> Date: Sat, 7 May 2022 16:05:17 -0400
>
> Tested under Emacs 28.1 and a recent tip of the Emacs git repo for Ema= cs
> 29 with asynchronous native compilation enabled:
>
> M-x package-install RET hyperbole RET
>
> fails to load the hyperbole-autoloads.el file before the
> async native compiler and byte compiler produce these errors since
> the autoloaded var:append function is not defined:

Hyperbole is not part of Emacs, so this problem should first be taken
up with the Hyperbole developers.

Hi Eli:
<= div class=3D"gmail_default" style=3D"font-family:monospace,monospace">
<= /div>
Thanks for the response.=C2=A0 Two initial points:

1. I am the le= ad Hyperbole developer, so I will discuss it with myself :-)

2. I lo= ng ago was=C2=A0told=C2=A0that Elpa packages are considered part of Emacs a= lthough they don't ship with it; this is why you see a number of Hyperb= ole issues brought up on this list.=C2=A0 It would be good if you and other= core Emacs developers all held the same yes/no opinion on this so others c= ould follow whatever is decided.


> Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-em-but.el: Error: = Symbol's function definition is void
> var:append Disable showing Disable logging
> Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-mouse.el: Error: S= ymbol's function definition is void
> var:append Disable showing Disable logging
> Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hbut.el: Error: Symbol= 's function definition is void
> var:append Disable showing Disable logging
>
> The package manager definitely generates hyperbole-autoloads.el at som= e
> point though I do not know if it is before these errors are produced.<= br>
Do the *.el files that produce the error 'require' or 'load'= ;
hyperbole-autoloads?

No, the Emacs package manager i= nstallation process generates that file from the ;;;###autoload annotations= in Hyperbole lisp files (see the 'package--make-autoloads-and-stuff= 9; function in "package.el").=C2=A0 The package manager also sets= up at package=C2=A0activation time to have those autoloads loaded prior to= loading any other code from the Hyperbole package.=C2=A0 It would not make= sense to require hyperbole-autoloads in other Lisp files since that file d= oes=C2=A0not exist when the package distribution is generated and cannot be= referenced prior to making a release; it would also defeat the purpose of = autoloads.
=C2=A0 = If not, how would compilation know to load that
file?=C2=A0 Asynchronous native compilation runs in a separate pristine
Emacs session, so it needs every dependency explicitly spelled out, or
it will fail.

This is the issue that I am bringi= ng up that I am surprised does not affect or has not been reported for othe= r packages.=C2=A0 There needs to be a persistent, cross-session Emacs hook = that runs prior to native compilation of packages that loads the autoloads = file for the package.


But again, I think this is a matter for the Hyperbole developers to
look into first.

I have resolved this for Hyperb= ole in an upcoming pre-release by moving a number of previously autoloaded = definitions into a file that can be 'required' by each Hyperbole mo= dule instead.=C2=A0 This was a good bit of work and does not address the mo= re general problem.=C2=A0 I hope you and others will consider this a bit mo= re and look into how it can be resolved for the good of all packages, as it= seems to be a disconnect between the Emacs package manager and the native = compilation code.

Best regards,

Bob
--000000000000287c4b05dec9a5c1-- From debbugs-submit-bounces@debbugs.gnu.org Thu May 12 01:51:51 2022 Received: (at 55305) by debbugs.gnu.org; 12 May 2022 05:51:51 +0000 Received: from localhost ([127.0.0.1]:39208 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1np1jq-0007Md-Os for submit@debbugs.gnu.org; Thu, 12 May 2022 01:51:50 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35456) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1np1jp-0007MM-EJ for 55305@debbugs.gnu.org; Thu, 12 May 2022 01:51:49 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57446) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1np1jk-0007kY-2l; Thu, 12 May 2022 01:51:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=aqUpVHEDyCo1Gm0x6yhJj4sfmEmEQ4Nm1c08dzdGC80=; b=REGf0dNype6Z raQhTRiCifqtdAxKM+CrtX0gisbaeRDwWCVKzsMChkA9/x5dnMk659nD7N532tyQsTwNvUO+7LnL/ Q+wuxXMKs41GyZzUZoxOHCIAllUXlrQzAQ6VqJciwxYJfke4dr7XE29u2nAy7Hcs98wjLxmJi1RKK xGAXTLYEJyuw2hJMwrr6FeMxlFFKShEk/VUO3ostlNMPVaVe31YMV9VV07mBRBFfp/ejf2Py2DUsH uPtv4aV3sjmtRULVYNFPhQ2tZvmXnd24CGUNCsxvo+rTrX98/9eHKxCV/so5ymqrZENhlbUNwMUlp 4jLK6GQ5UYGWUlUxXz4vCA==; Received: from [87.69.77.57] (port=2042 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1np1jj-00080q-I5; Thu, 12 May 2022 01:51:43 -0400 Date: Thu, 12 May 2022 08:51:45 +0300 Message-Id: <83zgjnpaby.fsf@gnu.org> From: Eli Zaretskii To: rswgnu@gmail.com In-Reply-To: (message from Robert Weiner on Thu, 12 May 2022 01:14:58 -0400) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: <8335hky5iv.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: 55305@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Robert Weiner > Date: Thu, 12 May 2022 01:14:58 -0400 > Cc: 55305@debbugs.gnu.org > > Do the *.el files that produce the error 'require' or 'load' > hyperbole-autoloads? > > No, the Emacs package manager installation process generates that file from the ;;;###autoload annotations > in Hyperbole lisp files (see the 'package--make-autoloads-and-stuff' function in "package.el"). The package > manager also sets up at package activation time to have those autoloads loaded prior to loading any other > code from the Hyperbole package. Does this last fact mean there's an assumption in Hyperbole that the package is always activated before its *.el files are compiled? If so, perhaps this is why it fails during native-compilation, where the package is not activated prior to the compilation? And why would such an assumption, if it exist, make sense? It seems to me like the ability to compile a .el file should require activation of any package, or what am I missing? From debbugs-submit-bounces@debbugs.gnu.org Thu May 12 02:22:24 2022 Received: (at 55305) by debbugs.gnu.org; 12 May 2022 06:22:24 +0000 Received: from localhost ([127.0.0.1]:39287 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1np2DC-0008I7-Bd for submit@debbugs.gnu.org; Thu, 12 May 2022 02:22:24 -0400 Received: from eggs.gnu.org ([209.51.188.92]:40002) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1np2DA-0008Hr-Jy for 55305@debbugs.gnu.org; Thu, 12 May 2022 02:22:09 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57762) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1np2D5-0003nv-Cr for 55305@debbugs.gnu.org; Thu, 12 May 2022 02:22:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=To:Subject:Date:From:In-Reply-To:References: MIME-Version; bh=L0Yh43xH2jIBgiixd02JPm97gFM0txSk9E6l6Y3pfOE=; b=doavipZYo26U /A7lrvrc/COtBR3p69YoGgQxDdUNQv6Q4YEpHbEtyEpn0tl1Cls9ia7Rw11hd1Iko/1FCzdm4a+Rf DV7Rw2AXdY+fUoWrviPkgdJWySMiD6E/RgmEr/qKg6cgA/UuaF6rVChvtUfLV9wiM8g4ANp+vM+gt aOYl7DYR3hqb742D5eKvAkbY3E624ZF+HpHCWEtXd9anTkfoAF+XNssKZbwm8W5QiJqb7zzi1xPOC yT9aiik8PekWVF3xu8R+hNdGg97Kba/fYurmPXmPcgBdxHJbsjYQ/U1f3yO8iXjikdlA1ifo+vAxT TBkZ7whirRzWHNkJ5oQhhA==; Received: from mail-vk1-f179.google.com ([209.85.221.179]:33341) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1np2D5-0008Jq-7j for 55305@debbugs.gnu.org; Thu, 12 May 2022 02:22:03 -0400 Received: by mail-vk1-f179.google.com with SMTP id d132so2204737vke.0 for <55305@debbugs.gnu.org>; Wed, 11 May 2022 23:22:03 -0700 (PDT) X-Gm-Message-State: AOAM532RIafzXjZwF1wfFce4mL8RxmXRgbokur145KSP0VmRFay3c3yS PiRntTodskoUJHaRfka5rR5U8IQ8tEDak21Pbhg= X-Google-Smtp-Source: ABdhPJxmHGR+HcVqu7fROzoKH93PqhFUhb1DhS5mzb51Zlx7BGrPyMduFxzXD+g/BqkKsNGLINjya8nn2Chqg5p7D/Q= X-Received: by 2002:ac5:cb64:0:b0:351:c285:68fa with SMTP id l4-20020ac5cb64000000b00351c28568famr15601872vkn.6.1652336522604; Wed, 11 May 2022 23:22:02 -0700 (PDT) MIME-Version: 1.0 References: <8335hky5iv.fsf@gnu.org> <83zgjnpaby.fsf@gnu.org> In-Reply-To: <83zgjnpaby.fsf@gnu.org> From: Robert Weiner Date: Thu, 12 May 2022 02:21:37 -0400 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation To: Eli Zaretskii Content-Type: multipart/alternative; boundary="0000000000007ab4af05deca9376" X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 55305 Cc: 55305@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: rswgnu@gmail.com Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --0000000000007ab4af05deca9376 Content-Type: text/plain; charset="UTF-8" On Thu, May 12, 2022 at 1:51 AM Eli Zaretskii wrote: > > From: Robert Weiner > > Date: Thu, 12 May 2022 01:14:58 -0400 > > Cc: 55305@debbugs.gnu.org > > > > Do the *.el files that produce the error 'require' or 'load' > > hyperbole-autoloads? > > > > No, the Emacs package manager installation process generates that file > from the ;;;###autoload annotations > > in Hyperbole lisp files (see the 'package--make-autoloads-and-stuff' > function in "package.el"). The package > > manager also sets up at package activation time to have those autoloads > loaded prior to loading any other > > code from the Hyperbole package. > > Does this last fact mean there's an assumption in Hyperbole that the > package is always activated before its *.el files are compiled? If > so, perhaps this is why it fails during native-compilation, where the > package is not activated prior to the compilation? > Said another way, there is an assumption that the hyperbole-autoloads.el file is loaded prior to any compilation, yes. This is similar to assumptions that loaddefs.el are loaded prior to their reference in other Emacs Lisp files. The point of the autoloads file is to include definitions that must exist in the Lisp environment prior to their reference in any Lisp files, whether this is during package use or package build-time. > And why would such an assumption, if it exist, make sense? It seems > to me like the ability to compile a .el file should require activation > of any package, or what am I missing? > I am not saying that the package must be activated prior to compilation but just that there must be an additional hook provided that forces loading of the autoloads prior to any build/compilation of the package (whether byte compilation or native compilation). Otherwise the build process will generate errors because the autoload definitions will not exist, e.g. maybe an autoloaded variable meant to be global to the package is referenced at the top-level of a package Lisp file. Complex packages have complex dependencies that I would say cannot all be handled with requires; otherwise, there would be no need for the autoload mechanism. Or am I missing something? Bob --0000000000007ab4af05deca9376 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Thu, May 12, 2022 at 1:51 AM Eli Zaretskii <eliz@gnu.org> wrote:
> F= rom: Robert Weiner <rsw= @gnu.org>
> Date: Thu, 12 May 2022 01:14:58 -0400
> Cc: 55305@d= ebbugs.gnu.org
>
>=C2=A0 Do the *.el files that produce the error 'require' or &#= 39;load'
>=C2=A0 hyperbole-autoloads?
>
> No, the Emacs package manager installation process generates that file= from the ;;;###autoload annotations
> in Hyperbole lisp files (see the 'package--make-autoloads-and-stuf= f' function in "package.el").=C2=A0 The package
> manager also sets up at package activation time to have those autoload= s loaded prior to loading any other
> code from the Hyperbole package.

Does this last fact mean there's an assumption in Hyperbole that the package is always activated before its *.el files are compiled?=C2=A0 If so, perhaps this is why it fails during native-compilation, where the
package is not activated prior to the compilation?
Said another way, there is an assumption that the hyperbole-autoloads.el= file is loaded prior to any compilation, yes.=C2=A0 This is similar to ass= umptions that loaddefs.el are loaded prior to their reference in other Emac= s Lisp files.=C2=A0 The point of the autoloads file is to include definitio= ns that must exist in the Lisp environment prior to their reference in any = Lisp files, whether this is during package use or package build-time.
=

=

And why would such an assumption, if it exist, make sense?=C2=A0 It seems to me like the ability to compile a .el file should require activation
of any package, or what am I missing?

I am not s= aying that the package must be activated prior to compilation but just that= there must be an additional hook provided that forces loading of the autol= oads prior to any build/compilation of the package (whether byte compilatio= n or native compilation).=C2=A0 Otherwise the build process will generate e= rrors because the autoload definitions will not exist, e.g. maybe an autolo= aded variable meant to be global to the package is referenced at the top-le= vel of a package Lisp file.=C2=A0 Complex packages have complex dependencie= s that I would say cannot all be handled with requires; otherwise, there wo= uld be no need for the autoload mechanism.=C2=A0 Or am I missing something?=

Bob
--0000000000007ab4af05deca9376-- From debbugs-submit-bounces@debbugs.gnu.org Thu May 12 03:22:54 2022 Received: (at 55305) by debbugs.gnu.org; 12 May 2022 07:22:54 +0000 Received: from localhost ([127.0.0.1]:39377 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1np39y-0003uh-Fq for submit@debbugs.gnu.org; Thu, 12 May 2022 03:22:54 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49218) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1np39w-0003uS-F1 for 55305@debbugs.gnu.org; Thu, 12 May 2022 03:22:52 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58510) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1np39q-00044d-Cj; Thu, 12 May 2022 03:22:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=sm8PQtXSevWDLQ1eHYcs+NsZ3d9X696OGXyKbf/ZcRw=; b=PT3JSDnClcz8 DuyhpCnKj3nl0Sokh54CunEMlRRJ4xIGykiy5fGMAtTLdOCSPW+1PIwYMGBULEZlQP9KXfy7Q2mlN z0nm9YOnRkcz+hf/qHeF2qZ3uCktx0oFtUPJFCfSlkHy4g2V59/aKYUpwWV0BpD9+EBv8TqlbcnN4 d2gT1cpCVUWr7DudORupmhF9LMUjVm4RLBKegbXS/RfmSontCS6omngBvGQexaesK5z2QKjqV7OSO srF+CNmrhpxEMe6n3pOuDOXap57E4XXQVPUyTTzwclmgzR6sCON1rhhn+zB3qS9eHrT0P1I+iQkcV QtwxgR+khmktpEHM3d/XaQ==; Received: from [87.69.77.57] (port=3671 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1np39p-00036l-Kg; Thu, 12 May 2022 03:22:45 -0400 Date: Thu, 12 May 2022 10:22:47 +0300 Message-Id: <83tu9vp648.fsf@gnu.org> From: Eli Zaretskii To: rswgnu@gmail.com In-Reply-To: (message from Robert Weiner on Thu, 12 May 2022 02:21:37 -0400) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: <8335hky5iv.fsf@gnu.org> <83zgjnpaby.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: 55305@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Robert Weiner > Date: Thu, 12 May 2022 02:21:37 -0400 > Cc: 55305@debbugs.gnu.org > > Does this last fact mean there's an assumption in Hyperbole that the > package is always activated before its *.el files are compiled? If > so, perhaps this is why it fails during native-compilation, where the > package is not activated prior to the compilation? > > Said another way, there is an assumption that the hyperbole-autoloads.el file is loaded prior to any > compilation, yes. This is similar to assumptions that loaddefs.el are loaded prior to their reference in other > Emacs Lisp files. loaddefs.el is preloaded into Emacs when it is built, so the analogy doesn't work in practice. I think you should look at bundled packages like Calc. Calc has calc-loaddefs.el, but I just now forced Emacs 28.1 to native-compile Calc and didn't see any problems. And I see that calc.el does say explicitly ;;;; (Autoloads here) (load "calc-loaddefs.el" nil t) > The point of the autoloads file is to include definitions that must exist in the Lisp > environment prior to their reference in any Lisp files, whether this is during package use or package > build-time. That is true, but AFAIU packages that have their own separate autoloads file should proactively do something to make sure those autoloads are loaded before they are needed. And this is not related to native-compilation in any way: the same will happen if one tries to byte-compile Hyperbole files without first loading its autoloads. Right? From debbugs-submit-bounces@debbugs.gnu.org Sat May 14 10:48:14 2022 Received: (at 55305) by debbugs.gnu.org; 14 May 2022 14:48:14 +0000 Received: from localhost ([127.0.0.1]:47212 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1npt41-00013F-IG for submit@debbugs.gnu.org; Sat, 14 May 2022 10:48:14 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47574) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1npt3z-00012o-UA for 55305@debbugs.gnu.org; Sat, 14 May 2022 10:48:12 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:55762) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1npt3u-0003Oz-G3 for 55305@debbugs.gnu.org; Sat, 14 May 2022 10:48:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=To:Subject:Date:From:In-Reply-To:References: MIME-Version; bh=SbPMVETI+4yUkz6ghGNUYEmk6pgiJhDVAMa03rzk3hY=; b=OSv2omvW//0p V0L5D7nGlW8UAtb06s2ZLqNVr+o1Vmc7NN/4nutehfP9oy4i+RnobXUrEWsWNss1e9P44hdJ5c330 tXlMA6nVu+f5/HCXmA1dDl5/+Mk+2b2txOjzPpEaTGEY6S1R8y5aPSLSkq4JmpzLg9lvRmTRaryTv JQVUgkIImpETUs+gLa7EtL/L+f+dEq1oP3yXx1vGqH8o2PVhBW4bvqp1kBUttT0u+lLKEBZG6kkli 4AI+WFIPwYww76ejGzb2Ra3flRC3WMCgUca6/6q3xB9qKzDeUZdvE40LbCqW8rkKMlZSIFEyR1wdY Q+1yiDcI36pdVX8Wgcz4fQ==; Received: from mail-vs1-f46.google.com ([209.85.217.46]:44841) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1npt3u-0001PZ-8N for 55305@debbugs.gnu.org; Sat, 14 May 2022 10:48:06 -0400 Received: by mail-vs1-f46.google.com with SMTP id x8so11216625vsg.11 for <55305@debbugs.gnu.org>; Sat, 14 May 2022 07:48:06 -0700 (PDT) X-Gm-Message-State: AOAM533KhEaM9yGNOJNp4BA4J4p8JGHPZi7xSQtVcgxM4H0fmI85qrUD O1+/iRA5v0+mGD3MxB1iv9wHKSb0F8CRNniHRVk= X-Google-Smtp-Source: ABdhPJyGyfs+BlWA55JQmVNYuDwQ0tzEBuC3+C0WanBJArlHc8LPP+0JVNmBTrmK8e+fGzyElT5aqkRTIN0d+l+2zf8= X-Received: by 2002:a67:f544:0:b0:32c:fd7d:b898 with SMTP id z4-20020a67f544000000b0032cfd7db898mr3778781vsn.77.1652539685636; Sat, 14 May 2022 07:48:05 -0700 (PDT) MIME-Version: 1.0 References: <8335hky5iv.fsf@gnu.org> <83zgjnpaby.fsf@gnu.org> <83tu9vp648.fsf@gnu.org> In-Reply-To: <83tu9vp648.fsf@gnu.org> From: Robert Weiner Date: Sat, 14 May 2022 10:47:40 -0400 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation To: Eli Zaretskii Content-Type: multipart/alternative; boundary="000000000000f0902305def9e033" X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 55305 Cc: 55305@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: rswgnu@gmail.com Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --000000000000f0902305def9e033 Content-Type: text/plain; charset="UTF-8" On Thu, May 12, 2022 at 3:22 AM Eli Zaretskii wrote: > > From: Robert Weiner > > Date: Thu, 12 May 2022 02:21:37 -0400 > > Cc: 55305@debbugs.gnu.org > > > > Does this last fact mean there's an assumption in Hyperbole that the > > package is always activated before its *.el files are compiled? If > > so, perhaps this is why it fails during native-compilation, where the > > package is not activated prior to the compilation? > > > > Said another way, there is an assumption that the hyperbole-autoloads.el > file is loaded prior to any > > compilation, yes. This is similar to assumptions that loaddefs.el are > loaded prior to their reference in other > > Emacs Lisp files. Hi Eli: Again, thanks for your feedback. I don't expect any change to be made on this in Emacs at this point but wanted to finish the discussion with a few final thoughts. > loaddefs.el is preloaded into Emacs when it is built, so the analogy > doesn't work in practice. > Emacs loads autoloads from a file when it is built (prior to dumping its image) and I am simply suggesting that both the package manager and the native compiler do the same for packages. > I think you should look at bundled packages like Calc. Calc has > calc-loaddefs.el, but I just now forced Emacs 28.1 to native-compile > Calc and didn't see any problems. And I see that calc.el does say > explicitly > > ;;;; (Autoloads here) > (load "calc-loaddefs.el" nil t) > Exactly, calc.el works around this missing feature by explicitly loading the loaddefs and then having every other calc module require the 'calc' library. This is equivalent to a manual load of the autoloads in every module of the package, i.e. there is no autoloading since the autoload definitions are required everywhere. Any package can do this but then nothing is autoloaded at build time when a definition is referenced. The calc package goes further and adds a hack at the end of certain files: ;; Local variables: ;; generated-autoload-file: "calc-loaddefs.el" ;; End: to force magic ;;;###autoload definitions to be written to the calc-loaddefs.el file. All of this is necessary because certain automated handling of the default package autoloading file is missing from Emacs. > > The point of the autoloads file is to include definitions that must > exist in the Lisp > > environment prior to their reference in any Lisp files, whether this is > during package use or package > > build-time. > > That is true, but AFAIU packages that have their own separate > autoloads file should proactively do something to make sure those > autoloads are loaded before they are needed. > My question is why? If we want definitions within packages autoloaded just as they can be outside of packages, why do we not want to simply fix the issue so that each package's autoload file is actually autoloaded by the package manager and the native compiler at both build and package activation time (the latter already being done)? We have a standardized naming for such files, package-.el. They are generated by the package manager but presently not autoloaded at build initialization. It is immensely more work for each large package to require this in each of their files and makes little sense since such a file does not exist at least for Elpa packages until build time. And this is not related to native-compilation in any way: the same > will happen if one tries to byte-compile Hyperbole files without first > loading its autoloads. Right? > Yes, if you manually byte-compile a package file, you have to ensure its autoloads have been loaded, but this is for a manual process. I am suggesting that in an automated context of package building that this too should be automated and autoloads should automatically be loaded by the build automation systems. Otherwise, my argument is that these are not treated as autoload files at the package level but are simply Lisp libraries that have to be manually loaded by every library in the package, a tedious affair for large packages. Certainly not the end of the world but difficult to manage and get right all the time. --000000000000f0902305def9e033 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Thu, May 12, 2022 at 3:22 AM Eli Zaretskii <eliz@gnu.org> wrote:
> F= rom: Robert Weiner <rsw= @gnu.org>
> Date: Thu, 12 May 2022 02:21:37 -0400
> Cc: 55305@d= ebbugs.gnu.org
>
>=C2=A0 Does this last fact mean there's an assumption in Hyperbole = that the
>=C2=A0 package is always activated before its *.el files are compiled?= =C2=A0 If
>=C2=A0 so, perhaps this is why it fails during native-compilation, wher= e the
>=C2=A0 package is not activated prior to the compilation?
>
> Said another way, there is an assumption that the hyperbole-autoloads.= el file is loaded prior to any
> compilation, yes.=C2=A0 This is similar to assumptions that loaddefs.e= l are loaded prior to their reference in other
> Emacs Lisp files.
Hi Eli:

Again, thanks for you= r feedback.=C2=A0 I don't expect any change to be made on this in Emacs= at this point but wanted to finish the discussion with a few final thought= s.
loaddefs.el is preloaded into Emacs when it is built, so the analogy
doesn't work in practice.

Emacs loads autolo= ads from a file when it is built (prior to dumping its image) and I am simp= ly suggesting that both the package manager and the native compiler do the = same for packages.
=C2=A0
I think you should look at bundled packages like Calc.= =C2=A0 Calc has
calc-loaddefs.el, but I just now forced Emacs 28.1 to native-compile
Calc and didn't see any problems.=C2=A0 And I see that calc.el does say=
explicitly

=C2=A0 ;;;; (Autoloads here)
=C2=A0 (load "calc-loaddefs.el" nil t)

<= /div>
Exactly, calc.el works around this missing feature by explicitly loading t= he loaddefs and then having every other calc module require the 'calc&#= 39; library.=C2=A0 This is equivalent to a manual load of the autoloads in = every module of the package, i.e. there is no autoloading since the autoloa= d definitions are required everywhere.=C2=A0 Any package can do this but th= en nothing is autoloaded at build time when a definition is referenced.=C2= =A0 The calc package goes further and adds a hack at the end of certain fil= es:

;; Local variables:
;; generated-autoload-file: "calc-lo= addefs.el"
;; End:

to force magic ;;;###autoload definit= ions to be written to the calc-loaddefs.el file.=C2=A0 All of this is neces= sary because certain automated handling of the default package autoloading = file is missing from Emacs.


> The point of the autoloads file is to include definitions that must ex= ist in the Lisp
> environment prior to their reference in any Lisp files, whether this i= s during package use or package
> build-time.

That is true, but AFAIU packages that have their own separate
autoloads file should proactively do something to make sure those
autoloads are loaded before they are needed.

My = question is why?=C2=A0 If we want definitions within packages autoloaded ju= st as they can be outside of packages, why do we not want to simply fix the= issue so that each package's autoload file is actually autoloaded by t= he package manager and the native compiler at both build=C2=A0and package a= ctivation time (the latter already being done)?=C2=A0 We have a standardize= d naming for such files, package-<package-name>.el.=C2=A0 They are ge= nerated by the package manager but presently not autoloaded at build initia= lization.=C2=A0 It is immensely more work for each large package to require= this in each of their files and makes little sense since such a file does = not exist at least for Elpa packages until build time.

And this is not related to native-= compilation in any way: the same
will happen if one tries to byte-compile Hyperbole files without first
loading its autoloads.=C2=A0 Right?

Yes, if= you manually byte-compile a package file, you have to ensure its autoloads= have been loaded, but this is for a manual process.=C2=A0 I am suggesting = that in an automated context of package building that this too should be au= tomated and autoloads should automatically be loaded by the build automatio= n systems.=C2=A0 Otherwise, my argument is that these are not treated as au= toload files at the package level but are simply Lisp libraries that have t= o be manually loaded by every library in the package, a tedious affair for = large packages.=C2=A0 Certainly not the end of the world but difficult to m= anage and get right all the time.


--000000000000f0902305def9e033-- From debbugs-submit-bounces@debbugs.gnu.org Sat May 14 11:06:02 2022 Received: (at 55305) by debbugs.gnu.org; 14 May 2022 15:06:02 +0000 Received: from localhost ([127.0.0.1]:47218 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nptLG-0001Yh-Fw for submit@debbugs.gnu.org; Sat, 14 May 2022 11:06:02 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49994) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nptLE-0001YF-Qz for 55305@debbugs.gnu.org; Sat, 14 May 2022 11:06:01 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:55894) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nptL9-0000OR-I9; Sat, 14 May 2022 11:05:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=VrQ0FGKV70tApVvx5Om653GlhtcgiblxA64XlUHCxE8=; b=V29cNAaFia/j +METilvMM4ErTea1s38W4yfc+gVjs0/6bVnKTUEBUxNJW8gqde1mLCDVU/1z9z5zuIc8fVUy2oUGB rlD9efKvzgHidYvsQ/n+NZZ93pQp/5BvHD86hjj3iD7vxbJpWifbHzvmvv23vQy8U9F2y2oUYktMi Cn6Hw0qjX0oJQe6a+Xh3FtBETUQXb3Hs9ZUWmclVlS8oVghaxH82eMjr9YhAKQF6Pvs7HGA2OSAzJ Elz+j2W8N8gm0GR0E9/7Ir2xxTcRxFOmQruYKY9TMkZr90UJa5xAz6HCaT3rpUaDpn/RpuLGAzjbv Qdk9uVcKUbzJdCfczoTR4g==; Received: from [87.69.77.57] (port=1922 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nptL9-0001ZC-1a; Sat, 14 May 2022 11:05:55 -0400 Date: Sat, 14 May 2022 18:05:44 +0300 Message-Id: <83ee0wkvcn.fsf@gnu.org> From: Eli Zaretskii To: rswgnu@gmail.com In-Reply-To: (message from Robert Weiner on Sat, 14 May 2022 10:47:40 -0400) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: <8335hky5iv.fsf@gnu.org> <83zgjnpaby.fsf@gnu.org> <83tu9vp648.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: 55305@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Robert Weiner > Date: Sat, 14 May 2022 10:47:40 -0400 > Cc: 55305@debbugs.gnu.org > > > loaddefs.el is preloaded into Emacs when it is built, so the analogy > > doesn't work in practice. > > Emacs loads autoloads from a file when it is built (prior to dumping its > image) and I am simply suggesting that both the package manager and the > native compiler do the same for packages. How can a native compiler or a byte compiler know that a given .el file needs some loadefs file to be loaded before it's compiled? Most Lisp files don't have and don't need any loaddefs files to be compiled. For that matter, how can a compiler know that a given .el files _has_ a loaddefs file, and if so, what is its name? Loaddefs files are needed for when a file is loaded, not when it's compiled. For compilation, we have 'require' and 'eval-when-compile', and always had them. Are you saying that 'require' and 'eval-when-compile' should also be replaced by some automation? if so, how can this work even in principle without some meta-data available somewhere for the compiler to find and use? And if we need meta-data, what is wrong with having it in the form of 'require' and 'eval-when-compile' in the file itself? > > I think you should look at bundled packages like Calc. Calc has > > calc-loaddefs.el, but I just now forced Emacs 28.1 to native-compile > > Calc and didn't see any problems. And I see that calc.el does say > > explicitly > > > > ;;;; (Autoloads here) > > (load "calc-loaddefs.el" nil t) > > > > Exactly, calc.el works around this missing feature by explicitly loading > the loaddefs and then having every other calc module require the 'calc' > library. But it isn't just calc.el: every single FOO-loaddefs.el file in the Emacs tree is loaded like that. This is a de-facto standard in how we use the separate loaddefs files of bundled packages. And again, there's nothing new here related to the native compiler: it works the same with the byte compiler, and will output the same warnings if you fail to follow this paradigm. It's nothing new. If you are lobbying for having more automated discovery and loading of loaddefs files, then you are asking for a new feature, not reporting a bug in Emacs 28 that didn't exist in previous versions. And I don't think I agree that this feature is a good idea, for when a file is compiled. But we could discuss this feature request; my point is that there's no bug here, but a normal behavior we have with any compilation of any Lisp file in any Emacs version. From debbugs-submit-bounces@debbugs.gnu.org Sat May 14 18:40:36 2022 Received: (at 55305) by debbugs.gnu.org; 14 May 2022 22:40:36 +0000 Received: from localhost ([127.0.0.1]:47683 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nq0RA-00020X-Br for submit@debbugs.gnu.org; Sat, 14 May 2022 18:40:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:36660) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nq0R9-00020I-7I for 55305@debbugs.gnu.org; Sat, 14 May 2022 18:40:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:34562) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nq0R3-000299-VE for 55305@debbugs.gnu.org; Sat, 14 May 2022 18:40:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=To:Subject:Date:From:In-Reply-To:References: MIME-Version; bh=fh8tl7QtKbeOHLI1NcD/NQXqzgfCwrHPLMcQGPtca3U=; b=N16vtQhCGz9b 61jdDogEslfrPkZ0O2ekvFJlxlc7Fyz9O9JVoZfsPS7bKALNdsGkuET2ZuGdOu4lLdsiqfVt9KSna wfz0FBx5z6D2Cnngm0YU1vS8o4M086k/zg1NTCpvTTfVAtoU3OA0GVBRLBiz4XF+DItO482fTrGS2 SjGvfBnPruelvtNusiti70t92F69FT9xvMXNhIZmFGaFynRKUQHBkbvtrr3awCS9s2hqBGfMhSiZc bNGAQVcLYfs/tWqvrgyD/Y6/2jZMCbnWgcevhbSItnRftPuitCIamjV4P5NsLznNU6pStQ9+QlktS YVF4nGeDcTmbtnt5y7uz1A==; Received: from mail-vk1-f177.google.com ([209.85.221.177]:35565) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nq0R3-0005rH-OQ for 55305@debbugs.gnu.org; Sat, 14 May 2022 18:40:29 -0400 Received: by mail-vk1-f177.google.com with SMTP id e7so5865116vkh.2 for <55305@debbugs.gnu.org>; Sat, 14 May 2022 15:40:29 -0700 (PDT) X-Gm-Message-State: AOAM530831oCnpwImERdD7FnUYDOsivJcRrUvjWGsnmpUDT0GTYQnHkk 2uJTogsqiAXrv0YIwO1aXZZVBq8m6GhBFRGY7gY= X-Google-Smtp-Source: ABdhPJxnXn5At7X1mdrgsVBtfsqOCrPo5Lr2fXqUVtkxPYHcPjjihkYHVV05mPyZRCL7tYx5M20ePNyRF8wpBycOjZc= X-Received: by 2002:a05:6122:a12:b0:351:c28f:674 with SMTP id 18-20020a0561220a1200b00351c28f0674mr4044645vkn.3.1652568029278; Sat, 14 May 2022 15:40:29 -0700 (PDT) MIME-Version: 1.0 References: <8335hky5iv.fsf@gnu.org> <83zgjnpaby.fsf@gnu.org> <83tu9vp648.fsf@gnu.org> <83ee0wkvcn.fsf@gnu.org> In-Reply-To: <83ee0wkvcn.fsf@gnu.org> From: Robert Weiner Date: Sat, 14 May 2022 18:40:03 -0400 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation To: Eli Zaretskii Content-Type: multipart/alternative; boundary="0000000000005a39b305df007a5a" X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 55305 Cc: 55305@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: rswgnu@gmail.com Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --0000000000005a39b305df007a5a Content-Type: text/plain; charset="UTF-8" On Sat, May 14, 2022 at 11:05 AM Eli Zaretskii wrote: > If you are lobbying for having more automated discovery and loading of > loaddefs files, then you are asking for a new feature, not reporting a > bug in Emacs 28 that didn't exist in previous versions. And I don't > think I agree that this feature is a good idea, for when a file is > compiled. But we could discuss this feature request; my point is that > there's no bug here, but a normal behavior we have with any > compilation of any Lisp file in any Emacs version. > Hi Eli: I see what you are saying. I wish Stefan would say something here too with his perspective of dealing with autoload files in Elpa packages. I will close this as a bug. Thanks, Bob --0000000000005a39b305df007a5a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sat, May 14, 2022 at 11:05 AM El= i Zaretskii <eliz@gnu.org> wrote:=
If you are lobbying for having more automated discovery and loading of
loaddefs files, then you are asking for a new feature, not reporting a
bug in Emacs 28 that didn't exist in previous versions.=C2=A0 And I don= 't
think I agree that this feature is a good idea, for when a file is
compiled.=C2=A0 But we could discuss this feature request; my point is that=
there's no bug here, but a normal behavior we have with any
compilation of any Lisp file in any Emacs version.
Hi Eli:

I see what you are saying.=C2=A0 I wish Stefan would say<= /div>
something here too with his perspective of dealing
with autoload files in= Elpa packages.

I will close this as a bug.

Thanks,

Bo= b


--0000000000005a39b305df007a5a-- From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 01:15:21 2022 Received: (at 55305) by debbugs.gnu.org; 15 May 2022 05:15:22 +0000 Received: from localhost ([127.0.0.1]:48340 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nq6bB-0004Gc-Lq for submit@debbugs.gnu.org; Sun, 15 May 2022 01:15:21 -0400 Received: from eggs.gnu.org ([209.51.188.92]:53390) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nq6bA-0004GQ-Db for 55305@debbugs.gnu.org; Sun, 15 May 2022 01:15:20 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:42872) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nq6b4-0002VK-NT; Sun, 15 May 2022 01:15:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=J/qFKTurQb4f1WMmC3GEmjwoOwsxgWsZ2nhlUblCnZI=; b=eFT17H6NaGIT U1y72XQgFWpCu57zT9ss7iGk//FL6Lew2kUfFI/W3SW5t0W418PHxWwG1pLLSkhnjG52cL8HXCsc/ qcMe9KryI5tnD+jNHCoYaRZnQd+/1eOqpvBJZc7KxTjgofzh4M4qe2YqwahY8FxN403FX0P4bdcQB a70ZtB2AfooI4pEM8MfOxMZ73Kb4v+yKVhVL9AQ+Z7YpomTQ9IlUJVJPb7/UL1E60zLumLcG30wBp npyKAKa7Mkw+gRlsk9rk6zRbjnenbJM1saIP8mX17iIkaEFHFORpWK0LE66dWLx8tFx7fBuVlOgi4 m8yDsQXXWgSiY72PdXvkMw==; Received: from [87.69.77.57] (port=1984 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nq6b4-0007r7-6X; Sun, 15 May 2022 01:15:14 -0400 Date: Sun, 15 May 2022 08:15:00 +0300 Message-Id: <8335hbl6ln.fsf@gnu.org> From: Eli Zaretskii To: rswgnu@gmail.com, Stefan Monnier In-Reply-To: (message from Robert Weiner on Sat, 14 May 2022 18:40:03 -0400) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: <8335hky5iv.fsf@gnu.org> <83zgjnpaby.fsf@gnu.org> <83tu9vp648.fsf@gnu.org> <83ee0wkvcn.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: 55305@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Robert Weiner > Date: Sat, 14 May 2022 18:40:03 -0400 > Cc: 55305@debbugs.gnu.org > > On Sat, May 14, 2022 at 11:05 AM Eli Zaretskii wrote: > > If you are lobbying for having more automated discovery and loading of > loaddefs files, then you are asking for a new feature, not reporting a > bug in Emacs 28 that didn't exist in previous versions. And I don't > think I agree that this feature is a good idea, for when a file is > compiled. But we could discuss this feature request; my point is that > there's no bug here, but a normal behavior we have with any > compilation of any Lisp file in any Emacs version. > > Hi Eli: > > I see what you are saying. I wish Stefan would say > something here too with his perspective of dealing > with autoload files in Elpa packages. Stefan doesn't read this list. I've CC'ed him now, so he could comment. From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 11:59:17 2022 Received: (at 55305) by debbugs.gnu.org; 15 May 2022 15:59:17 +0000 Received: from localhost ([127.0.0.1]:50489 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqGeL-0002bS-0U for submit@debbugs.gnu.org; Sun, 15 May 2022 11:59:17 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:1896) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqGeJ-0002bE-Ad for 55305@debbugs.gnu.org; Sun, 15 May 2022 11:59:15 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 7F5FB806F8; Sun, 15 May 2022 11:59:09 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 49EE480543; Sun, 15 May 2022 11:59:07 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1652630347; bh=PBXxYLblmXjLrssLVXMlIxyE9gGEIDmSWG8BOgkYiYY=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=DdhTaLub4YMggMH3o0P5j16fUwBEk2EJ6pUj66lSxI1a3ovuWqdezHuT7tf/tae4y dklWfZEwFL4h/VhWH/bEypViiNdsMFTj32E7Y6muGF5uosIi2DgH9mc94YXaxKzyK5 5FgAOPAoWjYb14zuI8G4QMT8VD5+tpstLqYNwkad4/wY/aoq3VXYtaNzye36fNWTLG x+yjHvkfE2BGrHM3kXkgj6bYvJlexYzFX8LYC05y3616QKA8yY0O4u1CNnHPvGLmZ0 wPXjjhbhQs4iI/fUpTnXLRfZSuQPzftH7AUVAThllt0cvDM4mJ6bGnPRSobDXu2ejw BGpjp6p/roqlA== Received: from pastel (unknown [45.72.221.51]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id E71F21205FF; Sun, 15 May 2022 11:59:06 -0400 (EDT) From: Stefan Monnier To: Robert Weiner Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation Message-ID: References: Date: Sun, 15 May 2022 11:59:05 -0400 In-Reply-To: (Robert Weiner's message of "Sat, 7 May 2022 16:05:17 -0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.055 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rswgnu@gmail.com, Andrea Corallo , 55305@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) First, sorry for not chiming in earlier (I don't subscribe to the emacs-bugs list, so I only see those bugs that are explicitly forwarded to me). Robert Weiner [2022-05-07 16:05:17] wrote: > Tested under Emacs 28.1 and a recent tip of the Emacs git repo for Emacs > 29 with asynchronous native compilation enabled: > > M-x package-install RET hyperbole RET Hmm... I tried to reproduce it here, with `emacs -Q` this gives me (during the normal compilation), among a bunch of lesser warnings: Compiling file ~/.emacs.d/elpa/hyperbole-8.0.0/test/kexport-tests.el at= Sun May 15 11:01:59 2022 kexport-tests.el:20:2: Error: Cannot open load file: Aucun fichier ou d= ossier de ce type, el-mock I also noticed the following warning in *Messages*: hibtypes:0: Warning: Not registering prefix "pa". Affects: ("parse-lab= el-and-file" "pathname" "pathname-line-and-column" "patch-msg") which points at some namespace uncleanliness in your code. Oh, and: Warning: Eager macro-expansion skipped due to cycle: =E2=80=A6 =3D> (load "hbut.el") =3D> (macroexpand-all =E2=80=A6) =3D>= (macroexpand (eval-and-compile =E2=80=A6)) =3D> (load "hbdata.el") =3D> (l= oad "hgnus.el") =3D> (load "hvar.el") =3D> (load "hsettings.el") =3D> (load= "hui-em-but.el") =3D> (load "hbut.el") Waiting for git... [2 times] You might wan to try and fix this one. > fails to load the hyperbole-autoloads.el file before the > async native compiler and byte compiler produce these errors since > the autoloaded var:append function is not defined: Indeed. > Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-em-but.el: Error: > Symbol's function definition is void var:append Disable showing Disable > logging It took a bit of while to get there (many other things to native-compile before this, apparently), but yes, I'm able to reproduce it. Looking at `comp-run-async-workers` in `comp.el`, I see that the async compilation basically does: emacs -q -l where 's content is basically the `expr` below: do (let* ((expr `((require 'comp) ,(when (boundp 'backtrace-line-length) `(setf backtrace-line-length ,backtrace-line-= length)) (setf comp-file-preloaded-p ,comp-file-preloaded= -p native-compile-target-directory ,native-co= mpile-target-directory native-comp-speed ,native-comp-speed native-comp-debug ,native-comp-debug native-comp-verbose ,native-comp-verbose comp-libgccjit-reproducer ,comp-libgccjit-= reproducer comp-async-compilation t native-comp-eln-load-path ',native-comp-el= n-load-path native-comp-compiler-options ',native-comp-compiler-options native-comp-driver-options ',native-comp-driver-options load-path ',load-path warning-fill-column most-positive-fixnum) ,native-comp-async-env-modifier-form (message "Compiling %s..." ,source-file) (comp--native-compile ,source-file ,(and load t)= ))) so the sync compilation is careful to preserve the current load-path via: load-path ',load-path which is why many of the files can be compiled correctly but it doesn't load the packages's autoloads like a normal session does. I suspect we should add a call to `package-activate-all` somewhere in the above code (and probably preserve `package-directory-list` and `package-user-dir` as well). I just tried to re-trigger the problem after applying the patch below [which also make this part of the code obey our 80-column convention, while at it] and it appears to be fixed (e.g. `hui-em-but.el` was successfully compiled). Andrea, any comment? Stefan diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el index 237de52884b..aa49607462c 100644 --- a/lisp/emacs-lisp/comp.el +++ b/lisp/emacs-lisp/comp.el @@ -3926,22 +3926,27 @@ comp-run-async-workers (file-newer-than-file-p source-file (comp-el-to-eln-filename source-file))) do (let* ((expr `((require 'comp) - ,(when (boundp 'backtrace-line-length) - `(setf backtrace-line-length ,backtrace-line= -length)) - (setf comp-file-preloaded-p ,comp-file-preloade= d-p - native-compile-target-directory ,native-c= ompile-target-directory - native-comp-speed ,native-comp-speed - native-comp-debug ,native-comp-debug - native-comp-verbose ,native-comp-verbose - comp-libgccjit-reproducer ,comp-libgccjit= -reproducer - comp-async-compilation t - native-comp-eln-load-path ',native-comp-e= ln-load-path - native-comp-compiler-options - ',native-comp-compiler-options - native-comp-driver-options - ',native-comp-driver-options - load-path ',load-path - warning-fill-column most-positive-fixnum) + (setq comp-async-compilation t) + (setq warning-fill-column most-positive-fixnum) + ,(let ((set (list 'setq))) + (dolist (var '(comp-file-preloaded-p + native-compile-target-directo= ry + native-comp-speed + native-comp-debug + native-comp-verbose + comp-libgccjit-reproducer + native-comp-eln-load-path + native-comp-compiler-options + native-comp-driver-options + load-path + backtrace-line-length + package-user-dir + package-directory-list)) + (when (boundp var) + (push var set) + (push `',(symbol-value var) set))) + (nreverse set)) + (package-activate-all) ,native-comp-async-env-modifier-form (message "Compiling %s..." ,source-file) (comp--native-compile ,source-file ,(and load t= )))) @@ -3994,7 +3999,7 @@ comp-run-async-workers (run-hooks 'native-comp-async-all-done-hook) (with-current-buffer (get-buffer-create comp-async-buffer-name) (save-excursion - (let ((buffer-read-only nil)) + (let ((inhibit-read-only t)) (goto-char (point-max)) (insert "Compilation finished.\n")))) ;; `comp-deferred-pending-h' should be empty at this stage. From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 12:17:26 2022 Received: (at 55305) by debbugs.gnu.org; 15 May 2022 16:17:26 +0000 Received: from localhost ([127.0.0.1]:50510 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqGvu-00033C-Dx for submit@debbugs.gnu.org; Sun, 15 May 2022 12:17:26 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59146) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqGvs-000330-O6 for 55305@debbugs.gnu.org; Sun, 15 May 2022 12:17:25 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:50078) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqGvn-0004Ya-AP; Sun, 15 May 2022 12:17:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=G1yCYmjreyqVgwAAHbDiYFEcrplv2IjF4DZ5v76Ozzc=; b=cUKkRJsUMo9n GPtny8WKZRfgkp4DNpW79pc3Od/H56lvy5e0Tqet4kbmLOnDuSi72dwa0jP0pb1Mf7v/9q/EaynNG bGERg6xXyaj6kCzCctMG02uPkncDjxYYxAtqHcP9MPlbes6rEdH2w/6tMvqN+jmix5j5CJolslbRs XGc6hWlnJOrCxKWy9TxiXtYUjSHC+d3pt8uhHde+aqN7i/S34JQikJBXolnD4N0RR/EporNlGUFci 0IC33NtgnHybDwJbQQIkne4yruGCTUXdA8zSYmSV5Cm1wRj8mtNJfZ7ltF2ASwtJhmnbqtTFrHdEN +eGUkL9w7WA2Kl4ZdMtX7Q==; Received: from [87.69.77.57] (port=2739 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqGvm-0002Vv-QF; Sun, 15 May 2022 12:17:19 -0400 Date: Sun, 15 May 2022 19:17:05 +0300 Message-Id: <83czgekby6.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (bug-gnu-emacs@gnu.org) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: 55305@debbugs.gnu.org, rswgnu@gmail.com, akrl@sdf.com, rsw@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: rswgnu@gmail.com, Andrea Corallo , 55305@debbugs.gnu.org > Date: Sun, 15 May 2022 11:59:05 -0400 > From: Stefan Monnier via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > I suspect we should add a call to `package-activate-all` somewhere > in the above code (and probably preserve `package-directory-list` and > `package-user-dir` as well). I don't see why comp.el should call package-related functions (or indeed know anything about packages and distinguish between packages and other Lisp files). It makes no sense to me. Compilation should not consider user customizations or be dependent on them. I could support some general infrastructure to detect whether a given file has separate autoloads, and perhaps load them when compiling, but that's all. And even this should be discussed, because I don't think I like the idea of a compilation always loading the autoloads, it's in many/most cases an overkill IMNSHO. > I just tried to re-trigger the problem after applying the patch below > [which also make this part of the code obey our 80-column convention, > while at it] and it appears to be fixed (e.g. `hui-em-but.el` was > successfully compiled). > Andrea, any comment? I'm firmly against this, sorry. Let's look for more elegant ways; this one is too blunt, and most Lisp files don't need it. Moreover, activating the packages will make every compilation dependent on the current user's customizations and installed packages, which is the antithesis of batch-mode compilation: it isn't a coincidence that "-batch" implies "-Q". From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 12:22:24 2022 Received: (at 55305) by debbugs.gnu.org; 15 May 2022 16:22:24 +0000 Received: from localhost ([127.0.0.1]:50515 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqH0i-0003C5-4H for submit@debbugs.gnu.org; Sun, 15 May 2022 12:22:24 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59700) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqH0f-0003Bs-IN for 55305@debbugs.gnu.org; Sun, 15 May 2022 12:22:22 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:50178) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqH0a-00064c-Av; Sun, 15 May 2022 12:22:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=PCDTYrofURhcasdaFBmn0zNxXJbHLRMcUQnR8MFGvBc=; b=HPf7FcD+CgQL aiFe8/zmRq3kQGgxOMbOQnCVLitfR2uLWqqvmGd3cQhqysojPIg4yW/2GpCtPWAI1N6h29OYKKmM0 AJjIu/dU3weAy40l986mZqB+qzAaooFeBO6lJDCjvqVFD3MVROHjwSmZD0acD2smOh1LOP75NJLXp Lbvnks3HvgEVRN88Mo+JZrSrFOBBmoQkFxcU3s8lrC4KFixNvCZRWGXQEEknOy/CLtB8x8o8K/qKL Cg0DNmAj3uD1+cFTyFrCxZB1kMqPSf8D5zG1fPyIiQ/jTWE64yy3yhoHzyC8+JifjcRStyp+KL6ym PZ08ZVEd3buj/088n2KHEw==; Received: from [87.69.77.57] (port=3040 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqH0Z-000313-Rs; Sun, 15 May 2022 12:22:16 -0400 Date: Sun, 15 May 2022 19:22:02 +0300 Message-Id: <83bkvykbpx.fsf@gnu.org> From: Eli Zaretskii To: akrl@sdf.org In-Reply-To: <83czgekby6.fsf@gnu.org> (message from Eli Zaretskii on Sun, 15 May 2022 19:17:05 +0300) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: <83czgekby6.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, rswgnu@gmail.com, monnier@iro.umontreal.ca, 55305@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) [Resending for Andrea, whose address was incorrect.] > Cc: rswgnu@gmail.com, Andrea Corallo , 55305@debbugs.gnu.org > Date: Sun, 15 May 2022 11:59:05 -0400 > From: Stefan Monnier via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > I suspect we should add a call to `package-activate-all` somewhere > in the above code (and probably preserve `package-directory-list` and > `package-user-dir` as well). I don't see why comp.el should call package-related functions (or indeed know anything about packages and distinguish between packages and other Lisp files). It makes no sense to me. Compilation should not consider user customizations or be dependent on them. I could support some general infrastructure to detect whether a given file has separate autoloads, and perhaps load them when compiling, but that's all. And even this should be discussed, because I don't think I like the idea of a compilation always loading the autoloads, it's in many/most cases an overkill IMNSHO. > I just tried to re-trigger the problem after applying the patch below > [which also make this part of the code obey our 80-column convention, > while at it] and it appears to be fixed (e.g. `hui-em-but.el` was > successfully compiled). > Andrea, any comment? I'm firmly against this, sorry. Let's look for more elegant ways; this one is too blunt, and most Lisp files don't need it. Moreover, activating the packages will make every compilation dependent on the current user's customizations and installed packages, which is the antithesis of batch-mode compilation: it isn't a coincidence that "-batch" implies "-Q". From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 12:47:18 2022 Received: (at 55305) by debbugs.gnu.org; 15 May 2022 16:47:18 +0000 Received: from localhost ([127.0.0.1]:50537 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqHOn-0003nV-SJ for submit@debbugs.gnu.org; Sun, 15 May 2022 12:47:18 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:42792) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqHOm-0003nJ-8l for 55305@debbugs.gnu.org; Sun, 15 May 2022 12:47:16 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id BB8F2100328; Sun, 15 May 2022 12:47:10 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 8B70D100139; Sun, 15 May 2022 12:47:01 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1652633221; bh=3zZv3FJB5N93qUHBKnQ7OrGxCtce5AmzQjwdNYqpJao=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=iweXMrrnV67dO/SDfur6ivA/m/StkQxz5yMSRPIYYW2ZRcznfLTauRMJ99uoSIqRr xyTli2kAZoGmnqv21Vba0wb7RXvKwMZZLCS8cX9F5SOQNFzLfh0BChihQmBrEQPSB5 dbH9VCEX4gr+MiHPOkh9iA5Yh6c1hHmIEmS+DC9mkaJzq5gohUl4Ywm2QAKmzDnw5R 5qpR4xyWxShe7HlIDZi30Tv8POTL34K3WBJTmyLugzalLenbcGI9cNYjKlAmpuZ58T xP4APT290Fxs+GVSLQUoNkyINWZA++ih4AWyaQzCKSbsjiFftxDwd8rG/dtQf4DLNr /WZmi6Z/ybWIQ== Received: from pastel (unknown [45.72.221.51]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 41F311206B4; Sun, 15 May 2022 12:47:01 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation Message-ID: References: <83czgekby6.fsf@gnu.org> <83bkvykbpx.fsf@gnu.org> Date: Sun, 15 May 2022 12:47:00 -0400 In-Reply-To: <83bkvykbpx.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 15 May 2022 19:22:02 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.041 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> Cc: rswgnu@gmail.com, Andrea Corallo , 55305@debbugs.gnu.o= rg >> Date: Sun, 15 May 2022 11:59:05 -0400 >> From: Stefan Monnier via "Bug reports for GNU Emacs, >> the Swiss army knife of text editors" >>=20 >> I suspect we should add a call to `package-activate-all` somewhere >> in the above code (and probably preserve `package-directory-list` and >> `package-user-dir` as well). > > I don't see why comp.el should call package-related functions (or > indeed know anything about packages and distinguish between packages > and other Lisp files). It makes no sense to me. Compilation should > not consider user customizations or be dependent on them. Compiling a `.el` file requires loading files, running macros, and calling functions, all of which may not come with Emacs and may depend on the user's specific customizations and set of installed packages (regardless of whether they're installed via package.el or some other way). So in order for the compilation to happen correctly, our async workers need to mimic to some extent the currently running Emacs session. The current code only does that to the extent that it preserves the `load-path`, but this is not always sufficient. My suggested patch adds the call to `package-activate-all` which is usually executed in `startup.el` between loading `early-init.el` and `init.el` and whose intention is to initialize things for the user-installed packages much like things are unconditionally initialized for the bundled packages (i.e. enough so it's easy to use them, but sufficiently little that it doesn't get in the way if the user doesn't want to use the package). Just like the current code, my proposed change can fail to do the right in some circumstances and I don't think there is a way to setup the sync workers such that they'll always do the right thing. My proposed patch just gets a bit closer to reproducing the user's setup in "the usual case" so I think it's less problematic than what we have. AFAIK the only way to make async compilation work reliably is to make it generate the `.eln` file without using the `.el` file (i.e. using the `.elc` file instead, which can be compiled without having to load any user-installed file, expand any macro, or run any user-installed function). That will also save us from mis-compiling file and from (re)emitting compilation warnings. But Someone=E2=84=A2 would have to work on that (I can't think of any reaso= n why it should be difficult, tho it might require a few changes to the .elc files we generate to preserve a bit more info from the source code, but it wouldn't require a new .elc format of anything complicated like that). Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 13:02:15 2022 Received: (at 55305) by debbugs.gnu.org; 15 May 2022 17:02:15 +0000 Received: from localhost ([127.0.0.1]:50555 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqHdH-0004DG-0M for submit@debbugs.gnu.org; Sun, 15 May 2022 13:02:15 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37248) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqHdF-0004D2-92 for 55305@debbugs.gnu.org; Sun, 15 May 2022 13:02:13 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:50968) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqHd9-0005gx-VU; Sun, 15 May 2022 13:02:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=tLuEn8SEYUVl4sIFB9T3mtzABwRFQcAw1ZnAxyI5lek=; b=SXMw8djKOsVZ mdKevG745obDxuV4xrpw5v1nTwb+5a9IFGU+rlQHPq3zzfgOu/YdtMhHSuA+7k+N20YotyvwaPzZ+ +xN5Ww3BbLnyV9pn3VmJ8sq71C0Px1Cr6q6lBkcdTieXX9h3RFGS3ExRPrVTPBqg9bB6Wia/w8dsj y5/XAJLozBUTzne48sNK23vfMVHdeiyX2xkLe2L47WVE9FIlhCiiqzMWEjHZZ2wj1LqufEuW/nOAC 7n0oLLGCEnPz4jV1LcCpDphfn+eTX+a8Uyxf7r8o7AJOMTUt5BOdxhRqDoKmZ32j9u8VTRFikAcPs gIv0aKIH6RpPUeiKKCK8dA==; Received: from [87.69.77.57] (port=1502 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqHd9-0001Nk-Da; Sun, 15 May 2022 13:02:07 -0400 Date: Sun, 15 May 2022 20:01:53 +0300 Message-Id: <83a6bik9vi.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (message from Stefan Monnier on Sun, 15 May 2022 12:47:00 -0400) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: <83czgekby6.fsf@gnu.org> <83bkvykbpx.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: akrl@sdf.org, 55305@debbugs.gnu.org, rswgnu@gmail.com, rsw@gnu.org > Date: Sun, 15 May 2022 12:47:00 -0400 > > > I don't see why comp.el should call package-related functions (or > > indeed know anything about packages and distinguish between packages > > and other Lisp files). It makes no sense to me. Compilation should > > not consider user customizations or be dependent on them. > > Compiling a `.el` file requires loading files, running macros, and > calling functions, all of which may not come with Emacs and may depend > on the user's specific customizations and set of installed packages > (regardless of whether they're installed via package.el or some other > way). > > So in order for the compilation to happen correctly, our async workers > need to mimic to some extent the currently running Emacs session. That was never the way byte-compilation worked in Emacs. We have all those 'require' and 'eval-when-compile' things precisely so a file can tell the compiler what is needed for the compilation. And we _need_ a way to make the compilation be completely independent of any local customizations or installed packages. > My suggested patch adds the call to `package-activate-all` which is > usually executed in `startup.el` between loading `early-init.el` and > `init.el` and whose intention is to initialize things for the > user-installed packages much like things are unconditionally initialized > for the bundled packages (i.e. enough so it's easy to use them, but > sufficiently little that it doesn't get in the way if the user doesn't > want to use the package). We don't call package-activate-all at startup when Emacs is told to ignore user and site customizations. That is NOT an accident, that is the only way to have *.elc and *.eln files that can be copied to another system and still work the same. Changing this makes no sense. I'm firmly against doing this. > AFAIK the only way to make async compilation work reliably is to make it > generate the `.eln` file without using the `.el` file (i.e. using the > `.elc` file instead, which can be compiled without having to load any > user-installed file, expand any macro, or run any user-installed > function). That will also save us from mis-compiling file and from > (re)emitting compilation warnings. This is unrelated, and is an entirely different discussion (which comes up from time to time, and we didn't yet find a way around the obstacles). From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 13:15:46 2022 Received: (at 55305) by debbugs.gnu.org; 15 May 2022 17:15:46 +0000 Received: from localhost ([127.0.0.1]:50573 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqHqM-0004Wi-ES for submit@debbugs.gnu.org; Sun, 15 May 2022 13:15:46 -0400 Received: from eggs.gnu.org ([209.51.188.92]:39228) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqHqK-0004WS-Ht for 55305@debbugs.gnu.org; Sun, 15 May 2022 13:15:45 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51212) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqHqE-0001Kf-Es; Sun, 15 May 2022 13:15:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=6lzDGLUra0MICP+7vHWkTVHejU19daO3ZyXXMrhpW+0=; b=ma1CbUDU9I25 04q8UhbjSiI+NzJR8XnwjXiIwrh0K/LNEJddCN87LdEMBSwfINsOwKMmjaaXfAYq1OLNaclcP1HN6 HtL4bWsiCZXuI+PRX+DEiAxQWbpYIA2P7R54Rf4kKUA3QXdVMuv38Co0zJYRHQOnDgHq3dx9reRrk ene6mnbNCHO4gxd0E6TSs918bYlXZ0Dqlps2tGucgZ9l8lblCpdc8mVlm68rlQYPIFrLY1gP2W4lm Nk9Nj5o5hFqf/oybgEMyrnWulXckLL/jn0mLOERkFxOTgZGozq/LYEUgX8dCK2+1LssxkIKYrvvVH YkZ2X2xT3Bln5JWEt5lurg==; Received: from [87.69.77.57] (port=2328 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqHqD-0005JY-Tc; Sun, 15 May 2022 13:15:38 -0400 Date: Sun, 15 May 2022 20:15:24 +0300 Message-Id: <838rr2k98z.fsf@gnu.org> From: Eli Zaretskii To: monnier@iro.umontreal.ca In-Reply-To: <83a6bik9vi.fsf@gnu.org> (message from Eli Zaretskii on Sun, 15 May 2022 20:01:53 +0300) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: <83czgekby6.fsf@gnu.org> <83bkvykbpx.fsf@gnu.org> <83a6bik9vi.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: akrl@sdf.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, rsw@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: rsw@gnu.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, akrl@sdf.org > Date: Sun, 15 May 2022 20:01:53 +0300 > From: Eli Zaretskii > > > My suggested patch adds the call to `package-activate-all` which is > > usually executed in `startup.el` between loading `early-init.el` and > > `init.el` and whose intention is to initialize things for the > > user-installed packages much like things are unconditionally initialized > > for the bundled packages (i.e. enough so it's easy to use them, but > > sufficiently little that it doesn't get in the way if the user doesn't > > want to use the package). > > We don't call package-activate-all at startup when Emacs is told to > ignore user and site customizations. That is NOT an accident, that is > the only way to have *.elc and *.eln files that can be copied to > another system and still work the same. Changing this makes no sense. Btw, having a package load its loaddefs (perhaps with a non-nil 2nd arg) is a much easier solution that doesn't need any changes at all, and will immediately resolve any problems with compilation. As you know only very well, that's what we do in core. From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 16:12:59 2022 Received: (at 55305) by debbugs.gnu.org; 15 May 2022 20:12:59 +0000 Received: from localhost ([127.0.0.1]:50815 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqKbq-0002io-Nj for submit@debbugs.gnu.org; Sun, 15 May 2022 16:12:59 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:31189) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqKbp-0002ia-Bg for 55305@debbugs.gnu.org; Sun, 15 May 2022 16:12:57 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id F314B100163; Sun, 15 May 2022 16:12:51 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 4DC7F10000A; Sun, 15 May 2022 16:12:50 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1652645570; bh=VOSjhnjVuMZTJedgaalZ7/ijMuZ8/9zsTNHQP2BZL8M=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=bZ/Y6gRH3awQb4/xJRh2HYQyFutXC6At+AAoMvolA33jWjIqdzZjUOgPsJ+pOc+T4 GF/Fe4o+kd1C2yWdHpOlfqjgR0MGEOoi0Loot1Cayx494yXoUaJgMgQ0tMZrt6bi+4 HJmeMEFzwRlrgyAYFz0ntWVeWi2PnpVxR83sJvW1EA5vhIWsJjS4Q9PJMmKJg6gnCR 23tjsZOymT85MHluWNMMq7eCv3OcTpAujAT4z5VMJym7pJcktJJrhxRxQdeFMOcAOK J/UUMp6q9A9Shj+2H9G3jCfYTUo5m/yXD5KBTyuE7OI9TWFk/hSLm4bHGFRtnHnMwL mA25K7R0oB94w== Received: from pastel (unknown [45.72.221.51]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id F0920120298; Sun, 15 May 2022 16:12:49 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation Message-ID: References: <83czgekby6.fsf@gnu.org> <83bkvykbpx.fsf@gnu.org> <83a6bik9vi.fsf@gnu.org> Date: Sun, 15 May 2022 16:12:48 -0400 In-Reply-To: <83a6bik9vi.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 15 May 2022 20:01:53 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.041 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> So in order for the compilation to happen correctly, our async workers >> need to mimic to some extent the currently running Emacs session. > That was never the way byte-compilation worked in Emacs. It has never been officially documented, you're right. But all the files bundled in Emacs are byte-compiled in an Emacs session where `lisp/loaddefs.el` has already been loaded and many packages rely on that to minimize the amount of explicit `require` they use (and to break some cyclic dependencies). In ELPA packages, that translates into an assumption that the package's `-autoloads.el` has already been loaded (which is indeed the case during `package-install` and is also the case when compiled via the code in GNU ELPA's `elpa-admin.el`). > We have all those 'require' and 'eval-when-compile' things precisely > so a file can tell the compiler what is needed for the compilation. > And we _need_ a way to make the compilation be completely independent > of any local customizations or installed packages. When a package uses some other package's macro, it necessarily depends on the locally installed packages to be compiled correctly. Until now `comp.el` limits the support for "local customizations or installed packages" to the act of propagating the current session's `load-path`. In theory, it could be sufficient. But this is not the same as what has been provided for the last ten years when compiling ELPA packages, so it will inevitably bump into packages for which it breaks compilation (such as Hyperbole). I'm not claiming that calling `package-activate-all` is right for reasons of principle. We sadly never clearly defined what it is that a package can count on. In practice ELPA packages have been able to count on the fact that their autoloads (and their dependencies's autoloads) have all been loaded (which also implies that all those packages have been added to the `load-path`). > I'm firmly against doing this. OK. >> AFAIK the only way to make async compilation work reliably is to make it >> generate the `.eln` file without using the `.el` file (i.e. using the >> `.elc` file instead, which can be compiled without having to load any >> user-installed file, expand any macro, or run any user-installed >> function). That will also save us from mis-compiling file and from >> (re)emitting compilation warnings. > This is unrelated, I firmly disagree with this, but I agree that it's a side discussion because we haven't yet found anyone motivated to tackle this problem in our native-comp pipeline. So in the mean time we have to paper over the consequences. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 16:39:23 2022 Received: (at 55305) by debbugs.gnu.org; 15 May 2022 20:39:23 +0000 Received: from localhost ([127.0.0.1]:50870 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqL1P-0003PT-5Z for submit@debbugs.gnu.org; Sun, 15 May 2022 16:39:23 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:39866) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqL1N-0003PG-BZ for 55305@debbugs.gnu.org; Sun, 15 May 2022 16:39:21 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 82C96441664; Sun, 15 May 2022 16:39:15 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 2F8F344165A; Sun, 15 May 2022 16:39:10 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1652647150; bh=j3K2cUI8r8SBkTq8Lhuc8i3K3FXmo75BaUMMN9M8EdM=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=aIgOb8vBq67otiJjlL3epslap1gntoRXMmH8f3PXuyodeGIW2rMQ2cgOiI2RVHgQt gyZyXUVezYnMxEmxkcmTYrJvr7+juSXhoMkGljBnXRERRfTVMQI0+syIZLyOI88G1r 5L0EEK7YTZxbyjuGn7XroGcz2jvqdBOGC7Sj8A/hjYNIv8W+Vi86k4K9OBiXcRuNnV PrJG1YaU0RY0yPP6RkOBKhEvfyFUhx1GyUyQBv93kno9FdW3Z8YTpztOGXIKca7C3I KO+ChJ97KkWeumAvdINl0WA4ZDxEFoFK/4dtfBDu5oKZPy2BIwyKbKE93LzZnsMvF1 15Hmke6evlmSQ== Received: from pastel (unknown [45.72.221.51]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 702F6120202; Sun, 15 May 2022 16:39:09 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation Message-ID: References: <83czgekby6.fsf@gnu.org> <83bkvykbpx.fsf@gnu.org> <83a6bik9vi.fsf@gnu.org> Date: Sun, 15 May 2022 16:39:07 -0400 In-Reply-To: <83a6bik9vi.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 15 May 2022 20:01:53 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.052 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > We don't call package-activate-all at startup when Emacs is told to > ignore user and site customizations. That is NOT an accident, that is > the only way to have *.elc and *.eln files that can be copied to > another system and still work the same. Changing this makes no sense. I think I fully agree with you when it comes to compiling those files which are bundled with Emacs. But when compiling files from ELPA packages, this would be an incompatible change, so I'm not sure it's the right move. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 22:31:23 2022 Received: (at 55305) by debbugs.gnu.org; 16 May 2022 02:31:23 +0000 Received: from localhost ([127.0.0.1]:51157 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqQW2-00061Q-UU for submit@debbugs.gnu.org; Sun, 15 May 2022 22:31:23 -0400 Received: from eggs.gnu.org ([209.51.188.92]:44272) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqQW2-00061C-0q for 55305@debbugs.gnu.org; Sun, 15 May 2022 22:31:22 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:60416) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqQVv-0000hL-Vi; Sun, 15 May 2022 22:31:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=SGZewyze+uTxEqjJHLbs8WQxr3cML3D0foLgHbezKog=; b=EUZw9PrrJ/H3 aXzLqKXr3hAEFEEZTBPtDOEXc82jDimjcwpNKbBKRcdxE/pF7IxrEfm3vYJYDEiaiH1RcLkpsb7zY TodwVcplC+FmV9D/pHKngBlVSVBxSlL/8JMJBRfQXwa7o7eavWrlUUeKVGjkCSQ1qzzFJFT0Ltjon AO7tVQ3Wp9K1EuOKQuLkhOE2zs+rP6qRzXBnWDRWHZO5HLoBzaZyHzyCr/n945jO1YVfauzYXfn8K BR8+y3lASPC358N/Qe8D8uXJPC4xmqKXAKpXZHf5jYBxYloSbrRu7QxwgFRzPtU3avWr33jesR/sK MvHN3f3L//nxRHLB4rTd8A==; Received: from [87.69.77.57] (port=4821 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqQVu-0002JK-0o; Sun, 15 May 2022 22:31:15 -0400 Date: Mon, 16 May 2022 05:31:01 +0300 Message-Id: <8335hajjiy.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (message from Stefan Monnier on Sun, 15 May 2022 16:12:48 -0400) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: <83czgekby6.fsf@gnu.org> <83bkvykbpx.fsf@gnu.org> <83a6bik9vi.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: akrl@sdf.org, 55305@debbugs.gnu.org, rswgnu@gmail.com, rsw@gnu.org > Date: Sun, 15 May 2022 16:12:48 -0400 > > > We have all those 'require' and 'eval-when-compile' things precisely > > so a file can tell the compiler what is needed for the compilation. > > And we _need_ a way to make the compilation be completely independent > > of any local customizations or installed packages. > > When a package uses some other package's macro, it necessarily depends on > the locally installed packages to be compiled correctly. My worry is about packages that do NOT depend on such macros. Calling package-activate-all will activate all the packages on the user's system, and there's no way of knowing what those packages do at activation time. They can change variables, advise functions, redefine commands, etc. We have no idea what will be the state of the session after doing that. > I'm not claiming that calling `package-activate-all` is right for > reasons of principle. We sadly never clearly defined what it is that > a package can count on. Then we should do that _before_ we propose solutions that rely on what's there, knowing that what's there was never intended to solve this particular issue. This cure is worse than the disease. From debbugs-submit-bounces@debbugs.gnu.org Sun May 15 22:33:43 2022 Received: (at 55305) by debbugs.gnu.org; 16 May 2022 02:33:43 +0000 Received: from localhost ([127.0.0.1]:51163 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqQYJ-000658-CG for submit@debbugs.gnu.org; Sun, 15 May 2022 22:33:43 -0400 Received: from eggs.gnu.org ([209.51.188.92]:44534) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqQYH-00064w-V5 for 55305@debbugs.gnu.org; Sun, 15 May 2022 22:33:42 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:60456) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqQYC-0000tv-NW; Sun, 15 May 2022 22:33:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=F43hexmk7CyvdgcpiS/jiVYfjy6/KesPv8H04pFYFgQ=; b=XqFoVm/95IFI 1u7VSPSflxT9S/FSgtT6DOHgk9pBJdrOjUVTftXisv98DvbvScalZZJmCFn042sQldnMkz+zEKWsV 0fVkFK74Rt+2sKJlNvZHi8r5C1JedZjgOidWJHO6GNSZbO8dgJ8ConAFmkPvhmYcX2NaI7/FXkTqM nALbHHZHKB/1MiFJ/7SRINmMHBcZ9Ml0uuYyumqlIp42kATca+PxE+u/pklhW3EHmFNoSKQRAfqHs fAqykkp7GxsDNPlgooD3VB4gUi2G5S3FOwwdOfRnT/veuf4+3ljQGLxpUFtpNswPN58YVjwT3Gyaj uxOTvA49KufA89abx0wzSA==; Received: from [87.69.77.57] (port=4965 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqQYC-00043Y-6r; Sun, 15 May 2022 22:33:36 -0400 Date: Mon, 16 May 2022 05:33:24 +0300 Message-Id: <831qwujjez.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (message from Stefan Monnier on Sun, 15 May 2022 16:39:07 -0400) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: <83czgekby6.fsf@gnu.org> <83bkvykbpx.fsf@gnu.org> <83a6bik9vi.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: akrl@sdf.org, 55305@debbugs.gnu.org, rswgnu@gmail.com, rsw@gnu.org > Date: Sun, 15 May 2022 16:39:07 -0400 > > > We don't call package-activate-all at startup when Emacs is told to > > ignore user and site customizations. That is NOT an accident, that is > > the only way to have *.elc and *.eln files that can be copied to > > another system and still work the same. Changing this makes no sense. > > I think I fully agree with you when it comes to compiling those files > which are bundled with Emacs. But when compiling files from ELPA > packages, this would be an incompatible change, so I'm not sure it's > the right move. Your proposal was to modify comp.el, so it will affect any native-compilation, including the compilation of core Emacs files. And even for files that are not bundled, think about downstream Emacs distros that would want to provide some of such files as part of the distributions. They should be compiled in a clean environment. From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 05:34:07 2022 Received: (at 55305) by debbugs.gnu.org; 16 May 2022 09:34:07 +0000 Received: from localhost ([127.0.0.1]:51767 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqX78-0004rq-Oi for submit@debbugs.gnu.org; Mon, 16 May 2022 05:34:07 -0400 Received: from mx.sdf.org ([205.166.94.24]:61625) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqX76-0004rh-EU for 55305@debbugs.gnu.org; Mon, 16 May 2022 05:34:05 -0400 Received: from ma.sdf.org (ma.sdf.org [205.166.94.33]) by mx.sdf.org (8.15.2/8.14.5) with ESMTPS id 24G9Y2CZ011425 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits) verified NO); Mon, 16 May 2022 09:34:02 GMT From: Andrea Corallo To: Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: Date: Mon, 16 May 2022 09:34:02 +0000 In-Reply-To: (Stefan Monnier via's message of "Sun, 15 May 2022 11:59:05 -0400") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 55305 Cc: 55305@debbugs.gnu.org, rswgnu@gmail.com, Stefan Monnier , Andrea Corallo , Robert Weiner X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" writes: > First, sorry for not chiming in earlier (I don't subscribe to the > emacs-bugs list, so I only see those bugs that are explicitly forwarded > to me). > > Robert Weiner [2022-05-07 16:05:17] wrote: >> Tested under Emacs 28.1 and a recent tip of the Emacs git repo for Emacs >> 29 with asynchronous native compilation enabled: >> >> M-x package-install RET hyperbole RET > > Hmm... I tried to reproduce it here, with `emacs -Q` this gives me > (during the normal compilation), among a bunch of lesser warnings: > > Compiling file ~/.emacs.d/elpa/hyperbole-8.0.0/test/kexport-tests.el = at Sun May 15 11:01:59 2022 > kexport-tests.el:20:2: Error: Cannot open load file: Aucun fichier ou= dossier de ce type, el-mock > > I also noticed the following warning in *Messages*: > > hibtypes:0: Warning: Not registering prefix "pa". Affects: ("parse-l= abel-and-file" "pathname" "pathname-line-and-column" "patch-msg") > > which points at some namespace uncleanliness in your code. > Oh, and: > > Warning: Eager macro-expansion skipped due to cycle: > =E2=80=A6 =3D> (load "hbut.el") =3D> (macroexpand-all =E2=80=A6) = =3D> (macroexpand (eval-and-compile =E2=80=A6)) =3D> (load "hbdata.el") =3D= > (load "hgnus.el") =3D> (load "hvar.el") =3D> (load "hsettings.el") =3D> (= load "hui-em-but.el") =3D> (load "hbut.el") > Waiting for git... [2 times] > > You might wan to try and fix this one. > >> fails to load the hyperbole-autoloads.el file before the >> async native compiler and byte compiler produce these errors since >> the autoloaded var:append function is not defined: > > Indeed. > >> Warning (comp): ~/.emacs.d/elpa/hyperbole-8.0.0/hui-em-but.el: Error: >> Symbol's function definition is void var:append Disable showing Disable >> logging > > It took a bit of while to get there (many other things to > native-compile before this, apparently), but yes, I'm able to > reproduce it. > > Looking at `comp-run-async-workers` in `comp.el`, I see that the async > compilation basically does: > > emacs -q -l > > where 's content is basically the `expr` below: > > do (let* ((expr `((require 'comp) > ,(when (boundp 'backtrace-line-length) > `(setf backtrace-line-length ,backtrace-lin= e-length)) > (setf comp-file-preloaded-p ,comp-file-preload= ed-p > native-compile-target-directory ,native-= compile-target-directory > native-comp-speed ,native-comp-speed > native-comp-debug ,native-comp-debug > native-comp-verbose ,native-comp-verbose > comp-libgccjit-reproducer ,comp-libgccji= t-reproducer > comp-async-compilation t > native-comp-eln-load-path ',native-comp-= eln-load-path > native-comp-compiler-options > ',native-comp-compiler-options > native-comp-driver-options > ',native-comp-driver-options > load-path ',load-path > warning-fill-column most-positive-fixnum) > ,native-comp-async-env-modifier-form > (message "Compiling %s..." ,source-file) > (comp--native-compile ,source-file ,(and load = t)))) > > so the sync compilation is careful to preserve the current load-path > via: > > load-path ',load-path > > which is why many of the files can be compiled correctly but it doesn't > load the packages's autoloads like a normal session does. > > I suspect we should add a call to `package-activate-all` somewhere > in the above code (and probably preserve `package-directory-list` and > `package-user-dir` as well). > > I just tried to re-trigger the problem after applying the patch below > [which also make this part of the code obey our 80-column convention, > while at it] and it appears to be fixed (e.g. `hui-em-but.el` was > successfully compiled). > Andrea, any comment? > > > Stefan Hi Stefan, thanks for having debugged this. I see no harm in propagating `package-user-dir' `package-directory-list' to the async worker if it's useful, but I'm with Eli in not calling `package-activate-all' when compiling. An idea would be that if the code being compiled needs that it could leverage `native-comp-async-env-modifier-form' for that. Best Regards Andrea From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 12:40:53 2022 Received: (at 55305) by debbugs.gnu.org; 16 May 2022 16:40:53 +0000 Received: from localhost ([127.0.0.1]:54908 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqdm8-0007PQ-K5 for submit@debbugs.gnu.org; Mon, 16 May 2022 12:40:52 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:61718) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqdm6-0007PB-9B for 55305@debbugs.gnu.org; Mon, 16 May 2022 12:40:50 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 5263580767; Mon, 16 May 2022 12:40:44 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 8A03F80539; Mon, 16 May 2022 12:40:42 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1652719242; bh=zVFchqdblUOSRVuLbUT9ryiiBYDb9rVGC6+5+LddGoY=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=G/HD/vzvm/9/BDIB8XS6UHAfm9Vmuk2TjQNF9E6QntLo7DMS1tB2DVaiUzogM4PR1 eknfCMTN3iifjJjsRiWj84+dOE8uOdEpKRQDTAChyRoO4SLAOgqvtk0YyyjzqR4VQw cYLQrukmWMkN/tDnaEWRefT96iDWRsuvBlLxqYcHgKim7M65PuWpxMxLlJI0xLizYD fGUIsg7dvth2eDyflHM1yjjQTZ1SqQpYC2x/ud0H+zC/1ntQbiP9TZGn0+R9x97/WD 2meTYlcyTzO4rR28kkU0IpepgR67JoBwHfWJE4yp5p/xOP0yA4qpN7uiYHm7ZO4o4c eq8fsNx0rHkvg== Received: from alfajor (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 6A56F120255; Mon, 16 May 2022 12:40:42 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation Message-ID: References: <83czgekby6.fsf@gnu.org> <83bkvykbpx.fsf@gnu.org> <83a6bik9vi.fsf@gnu.org> <8335hajjiy.fsf@gnu.org> Date: Mon, 16 May 2022 12:40:41 -0400 In-Reply-To: <8335hajjiy.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 16 May 2022 05:31:01 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.207 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Eli Zaretskii [2022-05-16 05:31:01] wrote: > My worry is about packages that do NOT depend on such macros. Calling > package-activate-all will activate all the packages on the user's > system, and there's no way of knowing what those packages do at > activation time. They can change variables, advise functions, > redefine commands, etc. Yup. I agree it can be worrisome, but: - Those packages are presumably already activated in the current session and hence similarly affected the generation of the `.elc` files if the `.elc` files were generated by `package-install`. - Preserving the `load-path` like we already do also exposes similar problems (some files may shadow the ones we need). > We have no idea what will be the state of the session after > doing that. After the files are compiled, the session is killed (we're talking about an Emacs sub-process performing native compilation asynchronously), so I don't think we need to worry too much about that. >> I'm not claiming that calling `package-activate-all` is right for >> reasons of principle. We sadly never clearly defined what it is that >> a package can count on. > Then we should do that _before_ we propose solutions that rely on > what's there, knowing that what's there was never intended to solve > this particular issue. I don't understand what you're saying, here. There is a de-facto definition of what a package can count on (which is that its autoloads file has been loaded), which has been used ever since the inception of `package.el`. We can't go back to a time before that. I don't know enough of what you mean by "this particular issue" to judge whether this design was made to solve this issue or not, but I do know that the autoloads file is loaded on purpose before compiling the `.elc` files so as to make it more convenient to write the code of a package, just like we happily rely on `loaddefs.el` being loaded when working on Emacs's bundled code. > This cure is worse than the disease. My proposed patch is not a cure. The cure would be to compile from the `.elc` file (since the root of the problem is that we need those autoloads and the `load-path` to macroexpand the code, and the only reason why the native-compiler needs to macroexpand the code is because it (re)starts from the `.el` file instead of using the `.elc` file where no macro-expansion need to take place). Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 12:42:58 2022 Received: (at submit) by debbugs.gnu.org; 16 May 2022 16:42:58 +0000 Received: from localhost ([127.0.0.1]:54917 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqdoA-0007T0-FI for submit@debbugs.gnu.org; Mon, 16 May 2022 12:42:58 -0400 Received: from lists.gnu.org ([209.51.188.17]:52448) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqdo7-0007Ss-Gx for submit@debbugs.gnu.org; Mon, 16 May 2022 12:42:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53050) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqdo2-0001kz-RW for bug-gnu-emacs@gnu.org; Mon, 16 May 2022 12:42:53 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:11203) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqdo0-0002xI-9n; Mon, 16 May 2022 12:42:49 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 1BEF94416D5; Mon, 16 May 2022 12:42:47 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id D69B74416D3; Mon, 16 May 2022 12:42:45 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1652719365; bh=2WVwiavIfNKD/4FW7m2GwcFSft+g34iSy+UNLFZjsIQ=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=iJtTCWNg866Tiwf/r+NsL6HZPoe8jeHsLulEdwBNDdLxLVWFtOiCbkhPXki4JqWln TliwSyIh7IMn6Xf7qO2jRqc9+FX9EJyTB6OPctnWoX7gSHUs69hgstlfq4gkiHDckJ nbOQtpArh1RCO7y/X6eqB4/sJae+5FPM66poy5C0VtK4zqisGx4ujGE1/IEXgu3/Nf f+W2O6kq6KYY5cVl82z4aHwjrwRaEaDItRZpujyyYavvOn0U3R58Nf1DAhM+fpRXrW Ilusc/1uVoqzgSgEeskQ44ooP7paYgMe5+aHpFgjOpTEtM8tUCzvRxG/s4RuZFM5oG cTAzgDJL7obSA== Received: from alfajor (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id BCC3E1208EC; Mon, 16 May 2022 12:42:45 -0400 (EDT) From: Stefan Monnier To: Andrea Corallo Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation Message-ID: References: Date: Mon, 16 May 2022 12:42:44 -0400 In-Reply-To: (Andrea Corallo's message of "Mon, 16 May 2022 09:34:02 +0000") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.264 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: Received-SPF: pass client-ip=132.204.25.50; envelope-from=monnier@iro.umontreal.ca; helo=mailscanner.iro.umontreal.ca X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: 55305@debbugs.gnu.org, "Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors" , rswgnu@gmail.com, Andrea Corallo , Robert Weiner X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) > I see no harm in propagating `package-user-dir' `package-directory-list' > to the async worker if it's useful, but I'm with Eli in not calling > `package-activate-all' when compiling. The currently discussed error happens because `hyperbole-autoloads.el` was not loaded before the compilation. Setting `package-user-dir` and `package-directory-list` won't help. > An idea would be that if the code being compiled needs that it could > leverage `native-comp-async-env-modifier-form' for that. Ewww! Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 12:57:36 2022 Received: (at 55305) by debbugs.gnu.org; 16 May 2022 16:57:37 +0000 Received: from localhost ([127.0.0.1]:54924 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqe25-0007ru-QB for submit@debbugs.gnu.org; Mon, 16 May 2022 12:57:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34898) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqe24-0007ri-3C for 55305@debbugs.gnu.org; Mon, 16 May 2022 12:57:20 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40628) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqe1w-0005PB-QO; Mon, 16 May 2022 12:57:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=jqs9E0t1DNeMvhZwWkZEp9c4WCmeebaG03f6tun2KZw=; b=bRKy1GtN8nH+ otH45Eb9LbHTIjQnAxR5KYI3zOyBv79x1PB32cp/6CnS68qxeJNs5tZEzhHlI5sFfAdqFSuZ0qvZJ tY3FOxrftap5BzUSvCwfwCvuAuCkeypgGMenVc7nOy6pMvzDUt4OQcQGH79eLkQH7ANqUKdo3hl6x o/eeIJUV3Xzx1KXCY4vm3OB9Q4+DrrSqxEolIiJeltm+7mfVUtj/BrlTTtf1DNR48+oUzM5QjCzpZ 5zm2ky+fEH6m0R9kGjDy8JSz6DnaT2W2HZVJU6PlVL/lMnnd7ZiZi9hZsY+0GIZlUJEr3HtN4L/An oyxpjiYmqgeZ1xTnZm5DMQ==; Received: from [87.69.77.57] (port=2206 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqe1w-0000HN-8I; Mon, 16 May 2022 12:57:12 -0400 Date: Mon, 16 May 2022 19:57:00 +0300 Message-Id: <83k0aliffn.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (message from Stefan Monnier on Mon, 16 May 2022 12:40:41 -0400) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: <83czgekby6.fsf@gnu.org> <83bkvykbpx.fsf@gnu.org> <83a6bik9vi.fsf@gnu.org> <8335hajjiy.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > From: Stefan Monnier > Cc: akrl@sdf.org, 55305@debbugs.gnu.org, rswgnu@gmail.com, rsw@gnu.org > Date: Mon, 16 May 2022 12:40:41 -0400 > > Eli Zaretskii [2022-05-16 05:31:01] wrote: > > My worry is about packages that do NOT depend on such macros. Calling > > package-activate-all will activate all the packages on the user's > > system, and there's no way of knowing what those packages do at > > activation time. They can change variables, advise functions, > > redefine commands, etc. > > Yup. I agree it can be worrisome, but: > - Those packages are presumably already activated in the current session > and hence similarly affected the generation of the `.elc` files if the > `.elc` files were generated by `package-install`. Two wrongs don't make one right. Besides, as long as the *.elc files are produced by batch-compilation, this will not happen. And I'm talking only about batch compilation, not about interactive compilation that the user does from a live session. > - Preserving the `load-path` like we already do also exposes similar > problems (some files may shadow the ones we need). The effect of load-path is a much smaller one, and not propagating load-path would cause much graver problems. > > We have no idea what will be the state of the session after > > doing that. > > After the files are compiled, the session is killed (we're talking about > an Emacs sub-process performing native compilation asynchronously), so > I don't think we need to worry too much about that. My worry is about the effects on the produced *.elc/*.eln files, so killing the session doesn't help. > >> I'm not claiming that calling `package-activate-all` is right for > >> reasons of principle. We sadly never clearly defined what it is that > >> a package can count on. > > Then we should do that _before_ we propose solutions that rely on > > what's there, knowing that what's there was never intended to solve > > this particular issue. > > I don't understand what you're saying, here. I'm sating we should define what is it the package can count on before suggesting solutions based on lack of such a definition. > There is a de-facto definition of what a package can count on (which is > that its autoloads file has been loaded), which has been used ever since > the inception of `package.el`. That's when the package is loaded, not when it's compiled. > I don't know enough of what you mean by "this particular issue" to judge > whether this design was made to solve this issue or not, but I do know > that the autoloads file is loaded on purpose before compiling the `.elc` > files so as to make it more convenient to write the code of a package, > just like we happily rely on `loaddefs.el` being loaded when working on > Emacs's bundled code. Loading the loaddefs file by some (as yet nonexistent) mechanism is something I'm prepared to discuss with the purpose of maybe finding some reasonable solution (though it isn't yet clear to me how to do that ion general, for the reasons I explained several messages up-thread). What I'm NOT interested in considering is a call to package-activate-all as part of compiling every Lisp file. > > This cure is worse than the disease. > > My proposed patch is not a cure. The cure would be to compile from the > `.elc` file (since the root of the problem is that we need those > autoloads and the `load-path` to macroexpand the code, and the only > reason why the native-compiler needs to macroexpand the code is because > it (re)starts from the `.el` file instead of using the `.elc` file > where no macro-expansion need to take place). I'm also okay with leaving this issue unsolved until such time as we have native compilation based on .elc files, and revisiting this then. From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 12:59:57 2022 Received: (at 55305) by debbugs.gnu.org; 16 May 2022 16:59:57 +0000 Received: from localhost ([127.0.0.1]:54929 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqe4M-0007vF-2b for submit@debbugs.gnu.org; Mon, 16 May 2022 12:59:57 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35370) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqe4A-0007up-6T for 55305@debbugs.gnu.org; Mon, 16 May 2022 12:59:41 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40668) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqe44-0005e0-O2; Mon, 16 May 2022 12:59:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=oyWIOL2boQX8fhJMmtN0xLH0jPv1pXqkl84oBH3UYH0=; b=hNQc4iRpNp02 oyuCJavK7n8aIzW/WEUzhGYv7D2he0SD9X/wL9BOI1erxF70mad3xSpSMMxD/GFPPgBxFmdmF0KWr pp/B67hhEzOSyk9IzyeNCIRIWJTUJTADhM2xk0hTw5XMuVwrITwe7Rgyj1P0Utcvp/IYB89VsAM6o JqUYwHvcdDpTz21+OjvriePtqt+ztaqr+KKx08UwYffDYYkmgDRp+k9ZobUC+4GvRdqITWahOKEsv foBa5FzRBb+csa+stJRlNEt2VhrUElyTS2Ti7tkybQQt98D5ROvfh+nGxucD9lj1EPHILlgSdr09h Z/E4fqrsJRhjqk3eNVmJ8g==; Received: from [87.69.77.57] (port=2346 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqe43-0004vj-Qf; Mon, 16 May 2022 12:59:24 -0400 Date: Mon, 16 May 2022 19:59:13 +0300 Message-Id: <83ilq5ifby.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (bug-gnu-emacs@gnu.org) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, akrl@sdf.com, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Cc: 55305@debbugs.gnu.org, rswgnu@gmail.com, akrl@sdf.com, rsw@gnu.org > Date: Mon, 16 May 2022 12:42:44 -0400 > From: Stefan Monnier via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > > I see no harm in propagating `package-user-dir' `package-directory-list' > > to the async worker if it's useful, but I'm with Eli in not calling > > `package-activate-all' when compiling. > > The currently discussed error happens because `hyperbole-autoloads.el` > was not loaded before the compilation. Setting `package-user-dir` and > `package-directory-list` won't help. But loading hyperbole-autoloads.el from the files that need it will help, and will solve this problem simply and without any complications. Look how much energy we invested in discussing a problem that takes a trivial one-liner to solve. From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 13:17:24 2022 Received: (at 55305) by debbugs.gnu.org; 16 May 2022 17:17:24 +0000 Received: from localhost ([127.0.0.1]:54961 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqeLU-0008Nb-Ho for submit@debbugs.gnu.org; Mon, 16 May 2022 13:17:24 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:2888) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqeLT-0008NN-65 for 55305@debbugs.gnu.org; Mon, 16 May 2022 13:17:23 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 6B75480342; Mon, 16 May 2022 13:17:17 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id F1D7280767; Mon, 16 May 2022 13:17:15 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1652721435; bh=3B+HTWm3dnoiQyOh6pEJgGJc/v8VAb/PyrSSwdEOOJ4=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=d05X3d0IouF0yY5aXb138n9Co7ImMyws8GMSztrQh65ifRVv43qVRF9jDcJWxlv+k kAwXd8Z5aTZO5hqN1J8NSvP2yyglEhvuBEJPmM6pfmJdAD07E1qHisjreq6Cszuyrw ZzyPfcMpjpEWqNqNQxZd7iFsTV0A+tG2BAi7gk/P1/taolazDUN3/tDsHXGQxU31ut KmL6btTF68xyxCjI39HTfpkDjl4TJGnc3TUCaB0eJkMlBVUkO0Ul7ZXQ5P3yaeia6K BIocx9Fc07vsYdoMy6wYK436vCPIGBuu/NO+CJcmOe1krytSANJQIbF7aOPDe+s0i4 kizK1YVNBbx6Q== Received: from alfajor (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 9CC38120263; Mon, 16 May 2022 13:17:15 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation Message-ID: References: <83czgekby6.fsf@gnu.org> <83bkvykbpx.fsf@gnu.org> <83a6bik9vi.fsf@gnu.org> <8335hajjiy.fsf@gnu.org> <83k0aliffn.fsf@gnu.org> Date: Mon, 16 May 2022 13:17:14 -0400 In-Reply-To: <83k0aliffn.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 16 May 2022 19:57:00 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.207 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, rswgnu@gmail.com, 55305@debbugs.gnu.org, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Besides, as long as the *.elc files are produced by batch-compilation, > this will not happen. AFAIK `.elc` files generated by batch-compilation is the rule for Emacs's bundled files but the exception for ELPA-installed packages. [ Many packages fail to compile properly when compiled in a batch session ;-) This has improved somewhat over the years, tho. ] >> - Preserving the `load-path` like we already do also exposes similar >> problems (some files may shadow the ones we need). > The effect of load-path is a much smaller one, > and not propagating load-path would cause much graver problems. Agreed. >> I don't understand what you're saying, here. > I'm sating we should define what is it the package can count on before > suggesting solutions based on lack of such a definition. My solution is not based on lack of definition. It's based on the de-facto definition that's been in use for the last 10 years. >> There is a de-facto definition of what a package can count on (which is >> that its autoloads file has been loaded), which has been used ever since >> the inception of `package.el`. > That's when the package is loaded, not when it's compiled. No, I'm talking about the compilation of the package's `.el` files into `.elc` files. > I'm also okay with leaving this issue unsolved until such time as we > have native compilation based on .elc files, and revisiting this then. Might be a good motivation to get this `.elc` compilation sooner rather than later. Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 18:27:23 2022 Received: (at 55305) by debbugs.gnu.org; 16 May 2022 22:27:23 +0000 Received: from localhost ([127.0.0.1]:55262 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqjBS-0007aa-QY for submit@debbugs.gnu.org; Mon, 16 May 2022 18:27:23 -0400 Received: from mail-qt1-f181.google.com ([209.85.160.181]:41766) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqjBQ-0007aM-Mq for 55305@debbugs.gnu.org; Mon, 16 May 2022 18:27:21 -0400 Received: by mail-qt1-f181.google.com with SMTP id u3so2924162qta.8 for <55305@debbugs.gnu.org>; Mon, 16 May 2022 15:27:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=Eh5Nb3NUj8D5RVXBQIDpRvqestpRobgEwe5+R/8h4Bs=; b=eQdxYKDmAlQqPBnxys60MRknOwmBzKo1XFjovWNEetcXOhDfBAvQlG0+XjAR4bcv4r Oi9Rt1a7eWL6EXpg34MmbFN/TvaRDxasgBVTt5d6T9uTyzqwEKkuj3Ul92C+ctNEficE V+uNviNiFpfsnGaiCLN6tFwJ1p8rvveCximVNimGIZ0zT9gx1Cq75jlEL/WiRpQXzgFN Q3iNmwXpc1nvWpDwueXJVRs087//SrufCAMY6UifYDVc9KLjNkD7KIQFydvWdcT55Dlv 3R2ZrDqfq6OTnsu5cIbs01LX6uYVJNlYrml46rFpLqqFqqWmaMDrw8RHTwH2e2XB262u U32w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=Eh5Nb3NUj8D5RVXBQIDpRvqestpRobgEwe5+R/8h4Bs=; b=MdF4MjdotBou2T09Cc5mRmLn//MQwQXdrlOeBlOpenaR7KJbWdOawIp6HGi2F0kO7o IwEZdMYYj9/YPgsUzBtF/1mTNH9FjwtlJMvUk3StX0Z6gQKNKJ0S0ZOHr++mNDJ302Ka 6pcFfg9NshHSqDMg4/vdhTSCKoQPPBdj5U8C7JRJnymuzFUKljEFIocC8fagZmj2PGsJ PbJ+BWpImFRXZuKCyPFQGAlnD7gKtFfNlK1feTiao4Dk/ZQCQEGKD07pNLkmbqctc62F SGwEhtX109umFh8XsOX/4E1xwkjcqAF9OPpSVofQSWG6Y1EytvwcW8pjMa55Lwmofqu2 x/ug== X-Gm-Message-State: AOAM533H/yBs7lv4gi7VxE4yCuZuPnf1esHfM0rPKvQVyXwaCSW8+vcB 0H/FO/yyGkWuzcGgYC7MCno= X-Google-Smtp-Source: ABdhPJx4a2zkqxSajFVElh6V8qujmhZlTXLM9R1VmtdEbczl1SvLE0QFAh9NYiCrYveI8udLQvQY0A== X-Received: by 2002:a05:622a:250:b0:2f3:cfd5:45aa with SMTP id c16-20020a05622a025000b002f3cfd545aamr17212748qtx.676.1652740035079; Mon, 16 May 2022 15:27:15 -0700 (PDT) Received: from smtpclient.apple (ool-2f148454.dyn.optonline.net. [47.20.132.84]) by smtp.gmail.com with ESMTPSA id h18-20020a379e12000000b0069fc13ce213sm6506440qke.68.2022.05.16.15.27.14 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 16 May 2022 15:27:14 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Robert Weiner Mime-Version: 1.0 (1.0) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation Date: Mon, 16 May 2022 18:27:13 -0400 Message-Id: <2C575270-EBA0-4234-A22E-74D40BFE82F7@gmail.com> References: <83ilq5ifby.fsf@gnu.org> In-Reply-To: <83ilq5ifby.fsf@gnu.org> To: Eli Zaretskii X-Mailer: iPhone Mail (19D52) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, 55305@debbugs.gnu.org, Stefan Monnier , akrl@sdf.com, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Just FYI, Hyperbole used to generate the hyperbole-autoloads.el file itself a= nd include it in the packaged release but this broke the Elpa build process.= -- Bob > On May 16, 2022, at 12:59 PM, Eli Zaretskii wrote: >=20 > =EF=BB=BF >>=20 >> Cc: 55305@debbugs.gnu.org, rswgnu@gmail.com, akrl@sdf.com, rsw@gnu.org >> Date: Mon, 16 May 2022 12:42:44 -0400 >> From: Stefan Monnier via "Bug reports for GNU Emacs, >> the Swiss army knife of text editors" >>=20 >>> I see no harm in propagating `package-user-dir' `package-directory-list'= >>> to the async worker if it's useful, but I'm with Eli in not calling >>> `package-activate-all' when compiling. >>=20 >> The currently discussed error happens because `hyperbole-autoloads.el` >> was not loaded before the compilation. Setting `package-user-dir` and >> `package-directory-list` won't help. >=20 > But loading hyperbole-autoloads.el from the files that need it will > help, and will solve this problem simply and without any > complications. Look how much energy we invested in discussing a > problem that takes a trivial one-liner to solve. From debbugs-submit-bounces@debbugs.gnu.org Mon May 16 22:28:03 2022 Received: (at 55305) by debbugs.gnu.org; 17 May 2022 02:28:03 +0000 Received: from localhost ([127.0.0.1]:55478 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqmwN-0001DU-7W for submit@debbugs.gnu.org; Mon, 16 May 2022 22:28:03 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59900) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqmwK-0001Cx-0U for 55305@debbugs.gnu.org; Mon, 16 May 2022 22:28:01 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49780) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqmwD-0002GA-QX; Mon, 16 May 2022 22:27:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=6+fBcRbYtz10rk2GdpLa/6XGK6s8dA4YyJ4pXDYPdug=; b=Yfqmg/84ZeGi 4qMB65VG1jC7CVdwQgeU0hzTQjZRoWW/nszyL5Ov6s6DsRHKl/mYoy6wo7gYKLgPF25jySUS/MJix mby4zalM4SFH8vf9Ejvw7XZbwlOLwR5ZToeao2zDJsRwYi6RgUafwRUNzhmzFJ/NMx8F824xfEiwr 9NsBnhramTlKp8SQugApfMS+R25nuFPAqPHevqRZ8qhEf/H9MzT4gFusSJBESEVniCGBcA2W4jrZ4 xR0Qx2EyDu+rep2yd+1nEoDMJ08fKQkRSeH53IUFoIx4ky5NKceL/QbPJT+wwfsM1Klv1pxKLuqxY yUx1rifm2AufEJ3x55Ou8g==; Received: from [87.69.77.57] (port=1253 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqmwD-0000ut-99; Mon, 16 May 2022 22:27:53 -0400 Date: Tue, 17 May 2022 05:27:42 +0300 Message-Id: <83czgcj3kx.fsf@gnu.org> From: Eli Zaretskii To: Robert Weiner In-Reply-To: <2C575270-EBA0-4234-A22E-74D40BFE82F7@gmail.com> (message from Robert Weiner on Mon, 16 May 2022 18:27:13 -0400) Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation References: <83ilq5ifby.fsf@gnu.org> <2C575270-EBA0-4234-A22E-74D40BFE82F7@gmail.com> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305 Cc: rsw@gnu.org, 55305@debbugs.gnu.org, monnier@iro.umontreal.ca, akrl@sdf.com, akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Robert Weiner > Date: Mon, 16 May 2022 18:27:13 -0400 > Cc: Stefan Monnier , akrl@sdf.org, > 55305@debbugs.gnu.org, akrl@sdf.com, rsw@gnu.org > > Just FYI, Hyperbole used to generate the hyperbole-autoloads.el file itself and include it in the packaged release but this broke the Elpa build process. Then perhaps the ELPA build process needs to be augmented not to break in those cases. From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 14 08:28:18 2022 Received: (at control) by debbugs.gnu.org; 14 Jun 2022 12:28:18 +0000 Received: from localhost ([127.0.0.1]:33376 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o15ec-0000no-5f for submit@debbugs.gnu.org; Tue, 14 Jun 2022 08:28:18 -0400 Received: from quimby.gnus.org ([95.216.78.240]:49154) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o15ea-0000na-Vs for control@debbugs.gnu.org; Tue, 14 Jun 2022 08:28:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Subject:From:To:Message-Id:Date:Sender:Reply-To:Cc: MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=WpNV9IK84nrR/5E73MHRlo1SyGoSkohVeayS72SeScQ=; b=NTfLnOMIac58PpU177knIjYzGt ReDBB8QtA1g6EuM/Oe8XTtnW4RqgYSIGH8DvlTzYSZyVaA/10R7uls/wF+BW/8ybeeeDkEmv3Inn2 NGlpAQYqUykYGMEzDMqvoaO5QmwEDkGxrgfvDb2is/BOfggCPsHXRmzOjPG5rh2fo1SI=; Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o15eT-0006OC-8F for control@debbugs.gnu.org; Tue, 14 Jun 2022 14:28:11 +0200 Date: Tue, 14 Jun 2022 14:28:08 +0200 Message-Id: <87tu8nmntz.fsf@gnus.org> To: control@debbugs.gnu.org From: Lars Ingebrigtsen Subject: control message for bug #55305 X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: tags 55305 - moreinfo quit Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) tags 55305 - moreinfo quit From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 07 17:36:20 2023 Received: (at 55305-done) by debbugs.gnu.org; 7 Jun 2023 21:36:20 +0000 Received: from localhost ([127.0.0.1]:54963 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q70pG-0006Lu-89 for submit@debbugs.gnu.org; Wed, 07 Jun 2023 17:36:20 -0400 Received: from eggs.gnu.org ([209.51.188.92]:60570) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q70pB-0006Lc-5t for 55305-done@debbugs.gnu.org; Wed, 07 Jun 2023 17:36:17 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q70p4-0000Tc-W3; Wed, 07 Jun 2023 17:36:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=aAotpf3n/4vdEUh1tviQj4v3jSHk9WQ5WW1vUTKjAY4=; b=ryxJ1xzKW/PfTJrfLYWX 5Z2VhJy3ICJUxpYY+N1oaZlJ6c1C+hwKRzf4rH8r8RTNMO5CRw/HEzmBo4axoV4WdySdTsOGEDBdq 9u6PxKRM8NAmNm9wGO1qt4TPXtIIfzGxb4FO3TLKB7PcN4Cj9HVJ3WHU+1sbI9K1Aq2CUibtEHrHr pRmxkWppgmix9X9APDBl6DQefrLRLUxiV5NhWbNlo3GlU2TuxBXUXgeeHspfeMu9ncdLvPcigpowC rCIUXHkzugDybtBU5dgoMcmjuohfIlocJqQ+pK3zFKmvak1R11Peo7yRtbHGtSXJUFPzSgLfkgy/t FHvUXp2/6nLBVA==; Received: from acorallo by fencepost.gnu.org with local (Exim 4.90_1) (envelope-from ) id 1q70p2-0007Rw-Ty; Wed, 07 Jun 2023 17:36:06 -0400 From: Andrea Corallo To: Eli Zaretskii Subject: Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation In-Reply-To: <83czgcj3kx.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 17 May 2022 05:27:42 +0300") References: <83ilq5ifby.fsf@gnu.org> <2C575270-EBA0-4234-A22E-74D40BFE82F7@gmail.com> <83czgcj3kx.fsf@gnu.org> Date: Wed, 07 Jun 2023 17:36:04 -0400 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 55305-done Cc: 55305-done@debbugs.gnu.org, akrl@sdf.com, rsw@gnu.org, monnier@iro.umontreal.ca, Robert Weiner , akrl@sdf.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Eli Zaretskii writes: >> From: Robert Weiner >> Date: Mon, 16 May 2022 18:27:13 -0400 >> Cc: Stefan Monnier , akrl@sdf.org, >> 55305@debbugs.gnu.org, akrl@sdf.com, rsw@gnu.org >> >> Just FYI, Hyperbole used to generate the hyperbole-autoloads.el file itself and include it in the packaged release but this broke the Elpa build process. > > Then perhaps the ELPA build process needs to be augmented not to break > in those cases. I'm closing this old bug as the outcome was that the issue is not a compiler bug. If the ELPA infrastructure needs some improvment we probably need a dedicated bug for that. Happy to reopen anyway if necessary. Bests Andrea From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 07 22:04:18 2023 Received: (at 55305) by debbugs.gnu.org; 8 Jun 2023 02:04:18 +0000 Received: from localhost ([127.0.0.1]:55243 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q750c-0007Rb-8J for submit@debbugs.gnu.org; Wed, 07 Jun 2023 22:04:18 -0400 Received: from mail-qv1-f52.google.com ([209.85.219.52]:41228) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q750Y-0007RJ-G8 for 55305@debbugs.gnu.org; Wed, 07 Jun 2023 22:04:16 -0400 Received: by mail-qv1-f52.google.com with SMTP id 6a1803df08f44-625b3d820ceso98576d6.1 for <55305@debbugs.gnu.org>; Wed, 07 Jun 2023 19:04:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686189848; x=1688781848; h=to:in-reply-to:references:message-id:date:subject:mime-version:from :content-transfer-encoding:from:to:cc:subject:date:message-id :reply-to; bh=O+3jhg0HlGhd3fpIYjCp9z2/M/WOw576LMy6dXrslFQ=; b=c+NBuPGYZYua41DBcdBmsvFLnHNkdv0EaOb6FxJ7EAOqMtPEC7aL+rdd5yzZhrGRMA pFogDehGu81ARMdCytOGP6+2gxUVXBIMitraGbD+p9XSUBSXV7h6F1iz0YN47SIMc6i1 sN6SFuWlhEfRJNjc2ibWa8SyMNa1o1D05HOjCsxdUZcmx3a9GIpSkAMuCWjj5tmRnM1D qIBSZDm0/QDmiFI91Q/1RYIogYnDDYg0VgKcWM889Rkbcf2cR6R6n24AT0e4MUVafFzd N7AGPN0pbe00KFZk6KNtRmmvy1EArsCyyTkgfPDGl5bdMhnLD6TiNRLttKo5MqsCarr3 FOWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686189848; x=1688781848; h=to:in-reply-to:references:message-id:date:subject:mime-version:from :content-transfer-encoding:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=O+3jhg0HlGhd3fpIYjCp9z2/M/WOw576LMy6dXrslFQ=; b=dO1h1jJPWp6Mn5ivKagi8rHkmLQIju50C6X2NUn+Suj/fRk0eINPQ7Sbv0FIL+M6Tw HdGJYiLzHsLCFxQyWSG+CGBL7iCwaOf/qnlKDLRE3HW4iqCqqYEJxjUCs4pp2MP5DWts UEkFtC9dIuVHVJmyACT4QO9Q/HFr2KrIBhyixdvdGQ6UdW50mSP0p2sd0Sf+0IikFe7b gRitu7xOwCxVeRzr58io0VrLzq1Ior9Wqs9PAgNZhJTSIbIFtML18Q6qAFm7BwKc5tWo 82lpEDpxaNtYSnE64+kObTlJH8o3kU7K7PiH1gcWOoj0VLtu9GfdQv+b4FeaRQVCWhvc FbxA== X-Gm-Message-State: AC+VfDw9bTXPUAQ6F/SShJpEvV4dn+iD0JGf0Pi4G6LoBSzAFIqe6pJk i5tWUOyzPL7M/2VIB8rnmYPBxjP/CpI= X-Google-Smtp-Source: ACHHUZ4bRpfZb1POZkiTgfZO5tyar7i3rujwB/i39jkztzWr5ePs5pPhjb6zqpssE266a14ysbOtFQ== X-Received: by 2002:a05:620a:2993:b0:75b:23a1:69e9 with SMTP id r19-20020a05620a299300b0075b23a169e9mr4860430qkp.0.1686189848213; Wed, 07 Jun 2023 19:04:08 -0700 (PDT) Received: from smtpclient.apple ([2600:1017:b426:1e8e:7993:985b:7f44:343d]) by smtp.gmail.com with ESMTPSA id c2-20020a05620a164200b00759322a6089sm32996qko.14.2023.06.07.19.04.07 for <55305@debbugs.gnu.org> (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 07 Jun 2023 19:04:07 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Robert Weiner Mime-Version: 1.0 (1.0) Subject: Re: bug#55305: closed (Re: bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation) Date: Wed, 7 Jun 2023 22:03:56 -0400 Message-Id: References: In-Reply-To: To: 55305@debbugs.gnu.org X-Mailer: iPhone Mail (20F66) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 55305 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Fyi, this was fixed long ago in the Hyerbole elpa-devel release, we just did= not realize there was a bug still open on it. -- Bob > On Jun 7, 2023, at 5:37 PM, help-debbugs@gnu.org wrote: >=20 > =EF=BB=BFYour bug report >=20 > #55305: 28.0.50: With async nativecomp, package manager fails to load hype= rbole-autoloads.el before compilation >=20 > which was filed against the emacs package, has been closed. >=20 > The explanation is attached below, along with your original report. > If you require more details, please reply to 55305@debbugs.gnu.org. >=20 > --=20 > 55305: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D55305 > GNU Bug Tracking System > Contact help-debbugs@gnu.org with problems > > From unknown Sun Jun 22 00:40:27 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 06 Jul 2023 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator