From unknown Mon Jun 16 23:44:13 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#75812 <75812@debbugs.gnu.org> To: bug#75812 <75812@debbugs.gnu.org> Subject: Status: 29.2.50; Support native-compiling package-install from batch Emacs Reply-To: bug#75812 <75812@debbugs.gnu.org> Date: Tue, 17 Jun 2025 06:44:13 +0000 retitle 75812 29.2.50; Support native-compiling package-install from batch = Emacs reassign 75812 emacs submitter 75812 Spencer Baugh severity 75812 wishlist tag 75812 confirmed thanks From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 24 12:54:21 2025 Received: (at submit) by debbugs.gnu.org; 24 Jan 2025 17:54:22 +0000 Received: from localhost ([127.0.0.1]:46962 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbNsr-0004ok-Hk for submit@debbugs.gnu.org; Fri, 24 Jan 2025 12:54:21 -0500 Received: from lists.gnu.org ([2001:470:142::17]:54198) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tbNso-0004oR-0P for submit@debbugs.gnu.org; Fri, 24 Jan 2025 12:54:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tbNsi-0006EH-7r for bug-gnu-emacs@gnu.org; Fri, 24 Jan 2025 12:54:12 -0500 Received: from mxout5.mail.janestreet.com ([64.215.233.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tbNsg-0004uk-KQ for bug-gnu-emacs@gnu.org; Fri, 24 Jan 2025 12:54:11 -0500 From: Spencer Baugh To: bug-gnu-emacs@gnu.org Subject: 29.2.50; Support native-compiling package-install from batch Emacs Date: Fri, 24 Jan 2025 12:54:08 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=waixah; t=1737741248; bh=jZPahb10PgY3a29dJgal1w/zeavAZFBzejuE61tpVec=; h=From:To:Cc:Subject:Date; b=LbvCNA14d1V6ptQRS80bkn2wiFB3EIwOz+wWjBjcuc+SSHEqsQyS5colnVg7uCAXR evLqFblAerb2g2+M4Zt/xuq7gtGFLqfHKi5bXIZT9CGSMrXBfoFm3tWvttD2h+Hbyi lUHoNM4YaeU/NqG8AWbuvpVKbL7jJv3SmGVQYgboVg4nQkaUf52SDqwx2vpYucb9EX 3wkcUeMy5dxlANY6I6YCQ+Upq/ccwdztQNCrR52KK+xz6u2oZ6ZXfEOSfYe+aFfwYs MA839LSNpCldz19HnN3i7pLy5pnTQv7KNyB8R9CDf+OuAem/1CoOFw34zi5rpOwlZd 8zhEtlBIMWnIA== Received-SPF: pass client-ip=64.215.233.18; envelope-from=sbaugh@janestreet.com; helo=mxout5.mail.janestreet.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit Cc: app-emacs-dev@janestreet.com, Stefan Kangas X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) Currently, setting package-native-compile to t will make package-install asynchronously native-compile the package. This works great for interactive usage. However, this is unsuitable for a batch Emacs running package-install. "emacs -batch --eval (package-install something)" will exit once the package is installed, but before the asynchronous native compilation completes; the native compilation will just be abandoned. So, currently, there's no straightforward way to native compile a package from a batch Emacs. Running package-install from a batch Emacs is useful as a way to update or pre-install packages in scripts, without requiring user interaction. This is widely used by package managers like Nix or dpkg to install Emacs packages as OS packages. In my specific case, it's used to install and byte-compile common packages into site-lisp. In all these cases, it would be nice to be able to native-compile the package at installation time and include the .eln files in the package directory. A straightforward solution would be for package-install-file to support synchronously native-compiling the installed package, such that after package-install finishes running, the native compilation is complete. Perhaps with a new value of package-native-compile? From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 24 17:03:46 2025 Received: (at 75812) by debbugs.gnu.org; 24 Jan 2025 22:03:46 +0000 Received: from localhost ([127.0.0.1]:47310 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbRmD-00034W-LY for submit@debbugs.gnu.org; Fri, 24 Jan 2025 17:03:45 -0500 Received: from mail-ej1-x636.google.com ([2a00:1450:4864:20::636]:48194) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1tbRmA-000347-JD for 75812@debbugs.gnu.org; Fri, 24 Jan 2025 17:03:43 -0500 Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-ab2c9b8aecaso458974966b.0 for <75812@debbugs.gnu.org>; Fri, 24 Jan 2025 14:03:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1737756216; x=1738361016; darn=debbugs.gnu.org; h=cc:to:subject:message-id:date:mime-version:references:in-reply-to :from:from:to:cc:subject:date:message-id:reply-to; bh=tHObQOqAkZ25TE+/EPavdl9hT2/QGgCzqwNj+l1wtOM=; b=m1cBfJwu/HenhXZwyvwf4pQfrg7g70aMkmJzxbejWap4NMe3l8IMR73yPkw9qbqRiE RbeEucIYCd59G8Rcjm0bcli2efXQy2ZKrOUhUS727G178HWGh4in9YT9zZ4+lHvuuKTP 2k6uxfNq97j1keBtAxaHOo2deEOmHL9tArQxVKn+N5jSM45VdleOflHjtkQeL1/Hl7Tk MyF17BN4K3Xy5RZGOUnKsxAHrUngFsjMMPW5H6VKm2m0w8hoEcugx8mJgYAvYYbMxlHU oaSnTn0D9jJihmkyugxx+BfGyP93Tgv8jEE98gZHeJNfYrnoFUdlBwTQoODDlbiBfSC7 tllw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737756216; x=1738361016; h=cc:to:subject:message-id:date:mime-version:references:in-reply-to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tHObQOqAkZ25TE+/EPavdl9hT2/QGgCzqwNj+l1wtOM=; b=aBODcqZglMAh8Vekm0OBSp7PhQcMlYWZhAvSptoMo0lyneKE1nTO+Zn8lRRs0cvVIe a2pfkxfN9/2ly8isyTkEE1f+rXHKuhPaP0qTDYVnzZAJxpE3/M69F80sFr6Rj/U/zvsT budc05L7iJQaf/0snSONDi6tTWCCHDt76ZUYIeRQSOz57yUvO8kzEhLEpFrU4CatIWiA NjGPbbdjnqUoaDDJu773UA2MOMG1oPWH5NsO0qBZ9BB0+e1B8J1XpZ/sbRdPvFWEw4W0 QBT0bbU4T1T6WauoUGDvO61I2vTP571VXXIQ/DPL1VXYvxCmgsBV/zc7k2IpmQu12nna h37w== X-Gm-Message-State: AOJu0YzhfopM8eyklJfRIZV10jUjAFU4dHwbYi3Gop+Th929I6pGmmgz FMJQayqVCr75FVZG52qUml/ooLpCXJSMvbUf0uPqSiypDFWEXDOtAPSPFIGc5VGuoVAfH0baUv5 MSPllbxbqdBDOHnN6bMWQNzPTyZGiXvUg X-Gm-Gg: ASbGncuLHrwghYqswexwEovxXB+/3jHXQqmgRm32XI+SrMnea0KsaDgALW1ETkmk6ZQ Z3uA2Gfm9/NZH/Xn4I7q2igUzYiMfdyMZgnmMT3Ye6RgpIAL4dbDlHjyirsBA3GU= X-Google-Smtp-Source: AGHT+IFns+rZ3WptjMwIWTJLnStw+w5r7MSavF3FVjUV6D+NwTWGqCE0MTLXfOaJMk88/R5zkBEMoQSk5ntUKNousWk= X-Received: by 2002:a17:906:dc93:b0:aa6:7737:1991 with SMTP id a640c23a62f3a-ab38b0b9158mr3365078966b.2.1737756216102; Fri, 24 Jan 2025 14:03:36 -0800 (PST) Received: from 753933720722 named unknown by gmailapi.google.com with HTTPREST; Fri, 24 Jan 2025 16:03:35 -0600 From: Stefan Kangas In-Reply-To: (Spencer Baugh's message of "Fri, 24 Jan 2025 12:54:08 -0500") References: MIME-Version: 1.0 Date: Fri, 24 Jan 2025 16:03:35 -0600 X-Gm-Features: AWEUYZlSt9Sj_ITpsdZn4nYH0V4PQqPTtpV4k4jEqa513zMtTqN1mVY_02IiM_4 Message-ID: Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs To: Spencer Baugh Content-Type: text/plain; charset="UTF-8" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 75812 Cc: app-emacs-dev@janestreet.com, 75812@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) severity 75812 wishlist tags 75812 + confirmed thanks Spencer Baugh writes: > Currently, setting package-native-compile to t will make package-install > asynchronously native-compile the package. This works great for > interactive usage. > > However, this is unsuitable for a batch Emacs running package-install. > "emacs -batch --eval (package-install something)" will exit once the > package is installed, but before the asynchronous native compilation > completes; the native compilation will just be abandoned. So, > currently, there's no straightforward way to native compile a package > from a batch Emacs. > > Running package-install from a batch Emacs is useful as a way to update > or pre-install packages in scripts, without requiring user interaction. > This is widely used by package managers like Nix or dpkg to install > Emacs packages as OS packages. In my specific case, it's used to > install and byte-compile common packages into site-lisp. In all these > cases, it would be nice to be able to native-compile the package at > installation time and include the .eln files in the package directory. > > A straightforward solution would be for package-install-file to support > synchronously native-compiling the installed package, such that after > package-install finishes running, the native compilation is complete. > Perhaps with a new value of package-native-compile? Sounds good to me, thanks. We'll take patches implementing this. From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 24 17:35:59 2025 Received: (at 75812) by debbugs.gnu.org; 24 Jan 2025 22:35:59 +0000 Received: from localhost ([127.0.0.1]:47383 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbSHO-0007hF-VX for submit@debbugs.gnu.org; Fri, 24 Jan 2025 17:35:59 -0500 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:36417) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tbSHN-0007gw-Ae for 75812@debbugs.gnu.org; Fri, 24 Jan 2025 17:35:57 -0500 From: Spencer Baugh To: Stefan Kangas Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs In-Reply-To: (Stefan Kangas's message of "Fri, 24 Jan 2025 16:03:35 -0600") References: Date: Fri, 24 Jan 2025 17:35:50 -0500 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=waixah; t=1737758150; bh=p5yN0wlervDv7O9SmsYg+wxjLoA2KGBoWc6pFvui2A4=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=YZMPCXdvA7HHu0n7NS1RZykS1QcU0Z2ksfGjzP9EAAyp+g5DuK7Yj/yry+fEgqrQl gwuVjwdaCJG3/EjRV72MNk6fPVsluK9gLGjoFGZJFwI4wBSFJZSKj9qCbLm6pnxGc+ y+tXDF0DAqzN7wJul388F3FzkFrRp+fdzdu9g8jfgrUzFtpxOGSEzFE86T4F0QSQum Zci1n+pWHToHkM6Lt6DLUnPTOmrIgWjdz0I7uXSZ2/Xt1kj31SRYi1kGbiWywxi8m6 7Zx5HdR2A4b67yR08+pfJoPExouJw0NbVLXOVwHeE29nJPzsp5cVsCeUJhFJRLXve2 g9+4gPWuN4bTQ== X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75812 Cc: app-emacs-dev@janestreet.com, 75812@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Stefan Kangas writes: > Spencer Baugh writes: > >> Currently, setting package-native-compile to t will make package-install >> asynchronously native-compile the package. This works great for >> interactive usage. >> >> However, this is unsuitable for a batch Emacs running package-install. >> "emacs -batch --eval (package-install something)" will exit once the >> package is installed, but before the asynchronous native compilation >> completes; the native compilation will just be abandoned. So, >> currently, there's no straightforward way to native compile a package >> from a batch Emacs. >> >> Running package-install from a batch Emacs is useful as a way to update >> or pre-install packages in scripts, without requiring user interaction. >> This is widely used by package managers like Nix or dpkg to install >> Emacs packages as OS packages. In my specific case, it's used to >> install and byte-compile common packages into site-lisp. In all these >> cases, it would be nice to be able to native-compile the package at >> installation time and include the .eln files in the package directory. >> >> A straightforward solution would be for package-install-file to support >> synchronously native-compiling the installed package, such that after >> package-install finishes running, the native compilation is complete. >> Perhaps with a new value of package-native-compile? > > Sounds good to me, thanks. We'll take patches implementing this. I'm happy to implement this, I'm just a bit lost about where to start. I tried the obvious "iterate over *.el files in the package directory, calling native-compile", but that failed on various files in various packages. There's a lot of code in package--compile and byte-recompile-directory, which does a bunch of stuff like ignoring certain files. I guess I need that code. Ultimately, package--compile calls byte-compile-file, so if I could get byte-compile-file to also emit native-compiled files, that would work. Is there a way to do that? byte+native-compile looks like a relevant variable, but is that the right direction to head in? From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 25 02:57:34 2025 Received: (at 75812) by debbugs.gnu.org; 25 Jan 2025 07:57:34 +0000 Received: from localhost ([127.0.0.1]:48305 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbb2s-0007P7-Br for submit@debbugs.gnu.org; Sat, 25 Jan 2025 02:57:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:54216) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tbb2q-0007Om-4X for 75812@debbugs.gnu.org; Sat, 25 Jan 2025 02:57:32 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tbb2j-0003wq-Na; Sat, 25 Jan 2025 02:57:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=99OjFhnSRSPzYUpErRuREnrBy6aPmBKqwfkfME52CMM=; b=l7htKcrHbc/G c0+LqL6tS4tjJxK1AvSWMARwnDfPyTR8tHvLoSiCZ/7oymkO4wlfVbBItgXgjpEgjJnd+V6K8fcH7 3LryqTbfZm+3ehRONmf2MC0ukO7IPdpJesK6lMBdnz92NeHWWbZjgHTaZdb5S5y6q9uGACBX74biK deJ8TZ1j22U3aHEOpAX12oopE09rZAayIl+oDIZ2F/73BpJM6+corC5BKtLVc0y/YHmnNWX7T3ZS6 RtpyQyqKiavlck4OF5xhV81bFpODo2OBDsN21QrBamMxqaYKZV3UK3Bw7glgt57bmP5DMefxhMpv3 qCNkbnKc1VYx89L8iYuYpw==; Date: Sat, 25 Jan 2025 09:57:22 +0200 Message-Id: <86plkbwckd.fsf@gnu.org> From: Eli Zaretskii To: Spencer Baugh , Andrea Corallo In-Reply-To: (bug-gnu-emacs@gnu.org) Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs References: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75812 Cc: app-emacs-dev@janestreet.com, stefankangas@gmail.com, 75812@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: app-emacs-dev@janestreet.com, 75812@debbugs.gnu.org > Date: Fri, 24 Jan 2025 17:35:50 -0500 > From: Spencer Baugh via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > > Sounds good to me, thanks. We'll take patches implementing this. > > I'm happy to implement this, I'm just a bit lost about where to start. > I tried the obvious "iterate over *.el files in the package directory, > calling native-compile", but that failed on various files in various > packages. Why not simply wait till all the async subprocesses compiling to native code finish and exit? Bonus points for displaying some kind of status or progress messages while waiting, so the user is informed why the wait. Andrea, is there a better way you can suggest? From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 30 10:29:41 2025 Received: (at 75812) by debbugs.gnu.org; 30 Jan 2025 15:29:41 +0000 Received: from localhost ([127.0.0.1]:47254 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tdWU8-0004x6-Qd for submit@debbugs.gnu.org; Thu, 30 Jan 2025 10:29:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:58212) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tdWU6-0004wm-17 for 75812@debbugs.gnu.org; Thu, 30 Jan 2025 10:29:38 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tdWU0-0000wp-92; Thu, 30 Jan 2025 10:29:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=U4m/DM0TK+6gEseJ4BcV1eOZO1QTM7ug6smEw7XE/LM=; b=nxTwc8R1pzLT4rbOeQPH SAm0Uju5xnAJC0mklVPYuv5QuWDq+Rh2mzT/KIdvAb/YT83RzQAWtyyT/DkOpk5lauJ0AFPTZGQyt Sr8RThPxSaNCdMYl4c85oOgtr7D8yRVtyaOH2J5VRrryhGcLxA8O1bYPXELam9DiPpuD8Wy+9RJ4V LJVITDCJrxRTSaQvby+vUeX1uN9JsPvbPNcqBarF/CIk+gWQfivetpOHTwdaMgwGGDflOY6jwk40w mXC2LhkFukz3cCswSKd0a290Wf3OZyOztR+Qa8qYuW91RaLnG8MmAo9AbisFCrLKpi29gQvyBnxlu vBrGofzeI8rWnw==; Received: from acorallo by fencepost.gnu.org with local (Exim 4.90_1) (envelope-from ) id 1tdWTu-00048G-40; Thu, 30 Jan 2025 10:29:31 -0500 From: Andrea Corallo To: Eli Zaretskii Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs In-Reply-To: <86plkbwckd.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 25 Jan 2025 09:57:22 +0200") References: <86plkbwckd.fsf@gnu.org> Date: Thu, 30 Jan 2025 10:29:23 -0500 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75812 Cc: Spencer Baugh , app-emacs-dev@janestreet.com, stefankangas@gmail.com, 75812@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Eli Zaretskii writes: >> Cc: app-emacs-dev@janestreet.com, 75812@debbugs.gnu.org >> Date: Fri, 24 Jan 2025 17:35:50 -0500 >> From: Spencer Baugh via "Bug reports for GNU Emacs, >> the Swiss army knife of text editors" >> >> > Sounds good to me, thanks. We'll take patches implementing this. >> >> I'm happy to implement this, I'm just a bit lost about where to start. >> I tried the obvious "iterate over *.el files in the package directory, >> calling native-compile", but that failed on various files in various >> packages. > > Why not simply wait till all the async subprocesses compiling to > native code finish and exit? Bonus points for displaying some kind of > status or progress messages while waiting, so the user is informed why > the wait. > > Andrea, is there a better way you can suggest? Not ATM, I agree having some way to visualize/report the status of the compilation queue would be nice. From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 30 10:39:39 2025 Received: (at 75812) by debbugs.gnu.org; 30 Jan 2025 15:39:39 +0000 Received: from localhost ([127.0.0.1]:47275 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tdWdm-0005TM-Hm for submit@debbugs.gnu.org; Thu, 30 Jan 2025 10:39:39 -0500 Received: from mail-vk1-xa32.google.com ([2607:f8b0:4864:20::a32]:55414) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1tdWdk-0005T4-47 for 75812@debbugs.gnu.org; Thu, 30 Jan 2025 10:39:37 -0500 Received: by mail-vk1-xa32.google.com with SMTP id 71dfb90a1353d-5162571e761so301998e0c.2 for <75812@debbugs.gnu.org>; Thu, 30 Jan 2025 07:39:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1738251570; x=1738856370; darn=debbugs.gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=1CdbdwqVCoiR5RSkK25f+ObZrX69Qn3BL7bwtEAWe5E=; b=VPZfP+fSp43/tRzyH8M5MwtEg3DL2KfZZnW5iihBmlJAnhP7tyXGXz6KdN5M+rzcht MVi3QrA87H1Kzy8xBXqRfv724voOW+qzcp0VGxEuFuVxm0W6j841X8L4g7C6pitxIYo/ eqJoMMRY1pRG+z9kV3N+eyO7jGuepMdabz+jR+sQ3b6+xNhwaMO8o3YGGdyq8Cw9UF8W BIl3eSShBWh/oTXXdaLl36HoAx+BGRdg2zp8U9vUBvq6UqTY62dh32gCbK2prqzv7VzV XZT6G+fZ4fZ9nFmNuH2m21/Y9hSk4RUzsb1s564nZXl/AW8H0nXKakejLTQmx3xtfwYL ZwMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738251570; x=1738856370; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1CdbdwqVCoiR5RSkK25f+ObZrX69Qn3BL7bwtEAWe5E=; b=gZyxEFBfvEaOcrv5IqTr+JGZ2tmlJMEmaIDBdh4XXWy7nbxm/uR1iFgaRSRRyN+LD7 yOuRH5iAeELcec1CRXLmSqwOMOXeqgLilyBHOSRyicMJX79dDFgCqATuhyCOgda82cl5 aWqKD5qxqLVEH3oPsclpqeJlwFoPIBqAJ9cGKzQC3a5kq2KKfP9yQQX6ArqghypypGPI PxiT2oUeqImoMmHdg5KZzcSsLVNpZfdKjh3gOVdtNxJSFSUmhsKl9o3jwN9UstbNsg7p SLiI5J0XuvpAEx5SkfNzc4QoxR+bkRMM63FrzdcmtdfX+4DqWojVBTC038HDIqkJ2keg 7cHA== X-Forwarded-Encrypted: i=1; AJvYcCVVqHo4IIdT/0M2UBp28uRTOK/yLTij6PujXq4xx6NqX0cNEoZ3VIR8RyUZFabHczDQPFMVkg==@debbugs.gnu.org X-Gm-Message-State: AOJu0YyXjArvlqsKHPopuJ7PGccUQomlOaYyGlAbsoJ6sEIu2/N2loHh B+zR+lvfXfxZfFMTI/1h3uhvjrSCAkfkgSI+lXdJJo6OY0+3vyUXsFhVIX4+FC1KZwnH4IcMVmf rk89ox3jH/K2qen2STiHSwVjicH4= X-Gm-Gg: ASbGncvpYKEhisIG1tPqANiIzzWxRODoghiOWhYnOXJXAiP+GQMD8m23pMSmV5l2GAc zxrhcIszF95TWmkK+vyCX8FOSALLgWn+cUwiniPD7rKcjiiBscsGQ8D/ErrlZGA1hL/JganqS X-Google-Smtp-Source: AGHT+IGd39A4tBQdqY11GQH1iV7cu+GIbupPL3wYsZp9Hd/8bVfRp2kVEbQB/46/UYVKBdmbXXI7ruyXJw0trVdzUDM= X-Received: by 2002:a05:6122:2491:b0:518:859e:87ae with SMTP id 71dfb90a1353d-51e9e4e30bdmr7687791e0c.7.1738251570355; Thu, 30 Jan 2025 07:39:30 -0800 (PST) MIME-Version: 1.0 References: <86plkbwckd.fsf@gnu.org> In-Reply-To: From: Ship Mints Date: Thu, 30 Jan 2025 10:37:39 -0500 X-Gm-Features: AWEUYZlGAf9wv2CRWF_9mU7lPjOGugkqWezUKJ1KyxqJxXE4LFsoJtfk-KQQdhU Message-ID: Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs To: Andrea Corallo Content-Type: multipart/alternative; boundary="00000000000061a862062cee3b65" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 75812 Cc: Spencer Baugh , Eli Zaretskii , app-emacs-dev@janestreet.com, stefankangas@gmail.com, 75812@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --00000000000061a862062cee3b65 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable If it is of interest, I wrote the below to visualize the compilation queue. It appears when something is enqueued for compilation, and disappears when the queue is empty. (when (native-comp-available-p) (defvar my:comp--async-running-func) (if (version< emacs-version "30") (progn (declare-function comp--async-runnings nil) (require 'comp) (setq my:comp--async-running-func #'comp-async-runnings)) (require 'comp-run) (setq my:comp--async-running-func #'comp--async-runnings)) (defun my/native-comp-mode-line-lighter () "mode-line lighter for `my/native-comp-mode-line-mode'." (when comp-files-queue (propertize (format " [=E2=97=B4%s:%d/%d]" comp-native-version-dir (length comp-files-queue) (funcall my:comp--async-running-func)) 'face (list :height 0.75)))) (define-minor-mode my/native-comp-mode-line-mode "Display the native compilation queue depth during compiles." :global t :group 'mode-line :lighter (:eval (my/native-comp-mode-line-lighter))) (when my:mode-line-native-comp (my/native-comp-mode-line-mode))) On Thu, Jan 30, 2025 at 10:30=E2=80=AFAM Andrea Corallo = wrote: > Eli Zaretskii writes: > > >> Cc: app-emacs-dev@janestreet.com, 75812@debbugs.gnu.org > >> Date: Fri, 24 Jan 2025 17:35:50 -0500 > >> From: Spencer Baugh via "Bug reports for GNU Emacs, > >> the Swiss army knife of text editors" > >> > >> > Sounds good to me, thanks. We'll take patches implementing this. > >> > >> I'm happy to implement this, I'm just a bit lost about where to start. > >> I tried the obvious "iterate over *.el files in the package directory, > >> calling native-compile", but that failed on various files in various > >> packages. > > > > Why not simply wait till all the async subprocesses compiling to > > native code finish and exit? Bonus points for displaying some kind of > > status or progress messages while waiting, so the user is informed why > > the wait. > > > > Andrea, is there a better way you can suggest? > > Not ATM, I agree having some way to visualize/report the status of the > compilation queue would be nice. > > > > --00000000000061a862062cee3b65 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
If it is of interest, I wrote the below to visualize the compilation que= ue. It appears when something is enqueued for compilation, and disappears w= hen the queue is empty.

(when (native-comp-available-p)
=C2=A0 (defvar my:comp--async= -running-func)
=C2=A0 (if (version< emacs-version "30")
= =C2=A0 =C2=A0 =C2=A0 (progn
=C2=A0 =C2=A0 =C2=A0 =C2=A0 (declare-functio= n comp--async-runnings nil)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 (require 'co= mp)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 (setq my:comp--async-running-func #'= comp-async-runnings))
=C2=A0 =C2=A0 (require 'comp-run)
=C2=A0 = =C2=A0 (setq my:comp--async-running-func #'comp--async-runnings))
= =C2=A0 (defun my/native-comp-mode-line-lighter ()
=C2=A0 =C2=A0 "mo= de-line lighter for `my/native-comp-mode-line-mode'."
=C2=A0 = =C2=A0 (when comp-files-queue
=C2=A0 =C2=A0 =C2=A0 (propertize
=C2=A0= =C2=A0 =C2=A0 =C2=A0(format " [=E2=97=B4%s:%d/%d]" comp-native-v= ersion-dir (length comp-files-queue) (funcall my:comp--async-running-func))=
=C2=A0 =C2=A0 =C2=A0 =C2=A0'face (list :height 0.75))))

=C2= =A0 (define-minor-mode my/native-comp-mode-line-mode
=C2=A0 =C2=A0 "= ;Display the native compilation queue depth during compiles."
=C2= =A0 =C2=A0 :global t
=C2=A0 =C2=A0 :group 'mode-line
=C2=A0 =C2= =A0 :lighter (:eval (my/native-comp-mode-line-lighter)))
=C2=A0 (when my= :mode-line-native-comp
=C2=A0 =C2=A0 (my/native-comp-mode-line-mode)))

On Thu, Jan 30, 2025 at 10:30=E2=80=AFAM Andr= ea Corallo <acorallo@gnu.org>= wrote:
Eli Zare= tskii <eliz@gnu.org> writes:

>> Cc: app-emacs-dev@janestreet.com, 75812@debbugs.gnu.org
>> Date: Fri, 24 Jan 2025 17:35:50 -0500
>> From:=C2=A0 Spencer Baugh via "Bug reports for GNU Emacs,
>>=C2=A0 the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>=
>>
>> > Sounds good to me, thanks.=C2=A0 We'll take patches imple= menting this.
>>
>> I'm happy to implement this, I'm just a bit lost about whe= re to start.
>> I tried the obvious "iterate over *.el files in the package d= irectory,
>> calling native-compile", but that failed on various files in = various
>> packages.
>
> Why not simply wait till all the async subprocesses compiling to
> native code finish and exit?=C2=A0 Bonus points for displaying some ki= nd of
> status or progress messages while waiting, so the user is informed why=
> the wait.
>
> Andrea, is there a better way you can suggest?

Not ATM, I agree having some way to visualize/report the status of the
compilation queue would be nice.



--00000000000061a862062cee3b65-- From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 23 12:26:52 2025 Received: (at 75812) by debbugs.gnu.org; 23 Apr 2025 16:26:53 +0000 Received: from localhost ([127.0.0.1]:58536 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u7cw0-00067X-1T for submit@debbugs.gnu.org; Wed, 23 Apr 2025 12:26:52 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:53491) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u7cvx-00066A-Pj for 75812@debbugs.gnu.org; Wed, 23 Apr 2025 12:26:50 -0400 From: Spencer Baugh To: Ship Mints Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs In-Reply-To: (Ship Mints's message of "Thu, 30 Jan 2025 10:37:39 -0500") References: <86plkbwckd.fsf@gnu.org> Date: Wed, 23 Apr 2025 12:26:44 -0400 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=waixah; t=1745425604; bh=qncjrBgmxEow8H5e5nmfAtPMJhmnUoX/ODOdLnyTqgE=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=sv98RpdG/sVYcami9DwthGTM/6zqL4tax2jljDJQZvAA8fpzBt/hDDEAcR8EJDYbV 9lfv2KYpv8ejSPgBpDQZvZcQrC7r7uPJA+sESnl54aFlH/qGSsLEmwn60i2hp15YKL Uux7gCz3MQ5PUq987tzbTXuPirMAKYXyDAebqyjcUfG/Ia9OZ+gAhbXFJ190y8OqWA Lj8LQ9bvMGU+QtaSJvbbAu6e3DPp8U9aCbf/7oK+39v6jpzekLT1Z7+MrCQUKy0b7F YMDKadfiWmdq7ftKrlwNB0eGFMANji5HAdUDZLpe6Nbu/h0bzm3ifUAPaIlcJ4yEJY oBkvmxgDt3L5Q== X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75812 Cc: Eli Zaretskii , Andrea Corallo , stefankangas@gmail.com, 75812@debbugs.gnu.org, app-emacs-dev@janestreet.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Conveniently there's already a hook, native-comp-async-all-done-hook, which gets called when the async native compilation is done. I just need to synchronously wait for that hook to be called. So what about something like this? (defun native-comp-async-wait-for-all-done () (unless (zerop (comp--async-runnings)) (let* ((native-comp-finished-pipe (make-pipe-process :name "native-comp-finished")) (native-comp-async-all-done-hook (list (lambda () (delete-process native-comp-finished-pipe))))) (accept-process-output native-comp-finished-pipe)))) The use of a pipe process is weird, but I don't know of a better way to wait for a hook to be called. --------------------- There's another related issue for the packaging use case I mentioned in my original email: The .eln files are placed in ~/.emacs.d/eln-cache/ by default when doing this native compilation. That complicates building and installing these packages independently. Is there a way to put the .eln files for a package in that individual package directory instead? The same directory as the .el and .elc files. The packages are recompiled for each new Emacs version anyway, so I think this should be safe from a versioning perspective. From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 24 01:28:59 2025 Received: (at 75812) by debbugs.gnu.org; 24 Apr 2025 05:28:59 +0000 Received: from localhost ([127.0.0.1]:33718 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u7p8t-0001H9-9u for submit@debbugs.gnu.org; Thu, 24 Apr 2025 01:28:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49582) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u7p8q-0001Gq-Sg for 75812@debbugs.gnu.org; Thu, 24 Apr 2025 01:28:57 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u7p8k-00071r-KX; Thu, 24 Apr 2025 01:28:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=sj69zHzg/L01W9/ZhfeD2lCt/AxM8VWo88Qr+bzgmOk=; b=Gm2zmlICnJHJ 4R31pScolx0u+iw6hiZlFm60dd5CEOi2viwsMIcU/0+SGl6lYhUPKu/v+XjnxEU4CTlZ4rFqoDXtu dP3pXA31yPW7tK+09mmPQ/c3LIZuZ7wM3W4SGddT70GILQRBgsyGVfXPc7ZhoJ3jjSrRwH5Q7HuxL 0LgY+OgnejqkpYHa0zegsIo5g0BsUYbnU3LyxUZxINP/trqxfeqcRSPS5IYtMmM2VUC3ReRJbrYKT cIHE+XkseoaVOH0Pd3I/FPd96dgbJlvMqjKJVLuV0BnyBNhCWoomakLQWEQBkf99qTLotYJuIta7V YkqsOowDXmbKwGOvFyBx/w==; Date: Thu, 24 Apr 2025 08:28:43 +0300 Message-Id: <86ikmuyvp0.fsf@gnu.org> From: Eli Zaretskii To: Spencer Baugh In-Reply-To: (message from Spencer Baugh on Wed, 23 Apr 2025 12:26:44 -0400) Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs References: <86plkbwckd.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75812 Cc: acorallo@gnu.org, stefankangas@gmail.com, shipmints@gmail.com, 75812@debbugs.gnu.org, app-emacs-dev@janestreet.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Spencer Baugh > Cc: Andrea Corallo , Eli Zaretskii , > app-emacs-dev@janestreet.com, stefankangas@gmail.com, > 75812@debbugs.gnu.org > Date: Wed, 23 Apr 2025 12:26:44 -0400 > > > Conveniently there's already a hook, native-comp-async-all-done-hook, > which gets called when the async native compilation is done. I just > need to synchronously wait for that hook to be called. > > So what about something like this? > > (defun native-comp-async-wait-for-all-done () > (unless (zerop (comp--async-runnings)) > (let* ((native-comp-finished-pipe (make-pipe-process :name "native-comp-finished")) > (native-comp-async-all-done-hook (list (lambda () (delete-process native-comp-finished-pipe))))) > (accept-process-output native-comp-finished-pipe)))) > > The use of a pipe process is weird, but I don't know of a better way to > wait for a hook to be called. That will not work on Windows, so let's not. I don't understand why you need all that complexity. Why not loop calling comp--async-runnings as long as it returns non-nil, each time sleeping for, say, 0.5 sec? Or why not install a native-comp-async-all-done-hook that will notice when compilations are done and allow the batch job to exit? All you need is a means to wait until all the async compilations are finished, no? Or what am I missing? > There's another related issue for the packaging use case I mentioned in > my original email: The .eln files are placed in ~/.emacs.d/eln-cache/ by > default when doing this native compilation. That complicates building > and installing these packages independently. Is there a way to put the > .eln files for a package in that individual package directory instead? You need to tweak native-comp-eln-load-path, e.g. prepend to the default-value list the directory where you want the *.eln files to be written. But you are well advised to leave the other directories in the list alone. From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 24 17:25:57 2025 Received: (at 75812) by debbugs.gnu.org; 24 Apr 2025 21:25:57 +0000 Received: from localhost ([127.0.0.1]:42048 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u844y-0005yH-LB for submit@debbugs.gnu.org; Thu, 24 Apr 2025 17:25:57 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:52223) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u844v-0005y3-As for 75812@debbugs.gnu.org; Thu, 24 Apr 2025 17:25:54 -0400 From: Spencer Baugh To: Eli Zaretskii Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs In-Reply-To: <86ikmuyvp0.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 24 Apr 2025 08:28:43 +0300") References: <86plkbwckd.fsf@gnu.org> <86ikmuyvp0.fsf@gnu.org> Date: Thu, 24 Apr 2025 17:25:47 -0400 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=waixah; t=1745529947; bh=z7KcFX6IDPxT0rRMSLLrFQZk7MDByX9T0Tz1gVZbUss=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=SMBOXxXDmwmSY4SkFhaN3MQ5OckegahYW3pBIK9bo5Q73tj+C82T+xxDYGenNUAuL ATxM3kHuemupjGGjssUlE4h0VzInSgkFxHyYOgTOMi9JFPZaz/gC92a2NiratTEEXQ 4oOf8Ucf2b0QHxQtXlqI39vZRL3rEL6Govh9N2JizNuyKVLleLlurjtyxof9xNj/i2 4e8h9PxwbNYnVdCqWjhB1TC0lZ32na1g2SLvhSJY/JupaKctkXMzMHJA5SsAmQaAdV iD2JlN10Zn/+dLnZr0DJj8lIsAHqub28pzdi0FtfxvTFP0H0s6u/8tghF8ovqW3R5y ymujqg3Wojx+w== X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75812 Cc: app-emacs-dev@janestreet.com, acorallo@gnu.org, stefankangas@gmail.com, 75812@debbugs.gnu.org, shipmints@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Eli Zaretskii writes: >> From: Spencer Baugh >> Cc: Andrea Corallo , Eli Zaretskii , >> app-emacs-dev@janestreet.com, stefankangas@gmail.com, >> 75812@debbugs.gnu.org >> Date: Wed, 23 Apr 2025 12:26:44 -0400 >> >> >> Conveniently there's already a hook, native-comp-async-all-done-hook, >> which gets called when the async native compilation is done. I just >> need to synchronously wait for that hook to be called. >> >> So what about something like this? >> >> (defun native-comp-async-wait-for-all-done () >> (unless (zerop (comp--async-runnings)) >> (let* ((native-comp-finished-pipe (make-pipe-process :name "native-comp-finished")) >> (native-comp-async-all-done-hook (list (lambda () (delete-process native-comp-finished-pipe))))) >> (accept-process-output native-comp-finished-pipe)))) >> >> The use of a pipe process is weird, but I don't know of a better way to >> wait for a hook to be called. > > That will not work on Windows, so let's not. Fair. > I don't understand why you need all that complexity. Why not loop > calling comp--async-runnings as long as it returns non-nil, each time > sleeping for, say, 0.5 sec? Sleeping for 0.5 sec means that building a package will take an extra .25 seconds on average, which adds up over hundreds of packages. Sleeping for 0.0001 sec wastes CPU, which also adds up over hundreds of packages. > Or why not install a native-comp-async-all-done-hook that will notice > when compilations are done and allow the batch job to exit? All you > need is a means to wait until all the async compilations are finished, > no? How do you propose to do that? I guess I could call kill-emacs from the hook, but I have more code I want to run after the native compilation is finished, and I can't move it all into the hook. >> There's another related issue for the packaging use case I mentioned in >> my original email: The .eln files are placed in ~/.emacs.d/eln-cache/ by >> default when doing this native compilation. That complicates building >> and installing these packages independently. Is there a way to put the >> .eln files for a package in that individual package directory instead? > > You need to tweak native-comp-eln-load-path, e.g. prepend to the > default-value list the directory where you want the *.eln files to be > written. But you are well advised to leave the other directories in > the list alone. Makes sense. Does that mean that, if I want the .eln files to be loaded from the individual package directory, I have to put every single package directory on native-comp-eln-load-path at runtime? That seems annoying, and probably slow. Is there a way to get an .eln file to be loaded from the directory the .elc file was found in? Maybe that just happens by default before searching native-comp-eln-load-path? From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 25 03:12:50 2025 Received: (at 75812) by debbugs.gnu.org; 25 Apr 2025 07:12:50 +0000 Received: from localhost ([127.0.0.1]:46413 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8DEv-0006zU-Lq for submit@debbugs.gnu.org; Fri, 25 Apr 2025 03:12:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48054) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8DEt-0006zA-6A for 75812@debbugs.gnu.org; Fri, 25 Apr 2025 03:12:47 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u8DEm-0000XJ-GS; Fri, 25 Apr 2025 03:12:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=kwYSLa5b5vwNGOEVrMWh55TkhjC4Sqr+kztxOZ5/4R4=; b=YpeeW+8XNlWx Vdxq5KRBK00evcJbElB2WuoEbeHyA54G3B4xNUqcmY5p4NqdiQ02CPytPROKGoaTYPs61tvCIb8HX iISK+wvEq+NwpRsivuxxgMkIHlPrhCbQcAebl1gT0GyGszl9dmNmRKO8G36v47MDSCm+wZfJO/WYJ 2wd3DBJjjJL1pIuJ3YQQdza0SDT/jH/N8gwyyNMtURCYmxOjuQfmOsdcXpZ7JtcMI+scnkzfGqo5q mBE72DrfycrFR7ZAvhv8LjKkwV3IqnNNPEg+9qEmvAVphkuPQjbeupi+YJCMNCl6WnuEdSsg5WToB tnAHodmeEBFPIHQeihFGAg==; Date: Fri, 25 Apr 2025 10:12:27 +0300 Message-Id: <86bjskyask.fsf@gnu.org> From: Eli Zaretskii To: Spencer Baugh In-Reply-To: (message from Spencer Baugh on Thu, 24 Apr 2025 17:25:47 -0400) Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs References: <86plkbwckd.fsf@gnu.org> <86ikmuyvp0.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75812 Cc: app-emacs-dev@janestreet.com, acorallo@gnu.org, stefankangas@gmail.com, 75812@debbugs.gnu.org, shipmints@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Spencer Baugh > Cc: acorallo@gnu.org, stefankangas@gmail.com, shipmints@gmail.com, > 75812@debbugs.gnu.org, app-emacs-dev@janestreet.com > Date: Thu, 24 Apr 2025 17:25:47 -0400 > > Eli Zaretskii writes: > > > I don't understand why you need all that complexity. Why not loop > > calling comp--async-runnings as long as it returns non-nil, each time > > sleeping for, say, 0.5 sec? > > Sleeping for 0.5 sec means that building a package will take an extra > .25 seconds on average, which adds up over hundreds of packages. > Sleeping for 0.0001 sec wastes CPU, which also adds up over hundreds of > packages. So make it 0.02 sec instead. 0.5 sec was just an example; no need to take a suggestion "ad absurdum", you know. > > Or why not install a native-comp-async-all-done-hook that will notice > > when compilations are done and allow the batch job to exit? All you > > need is a means to wait until all the async compilations are finished, > > no? > > How do you propose to do that? I guess I could call kill-emacs from the > hook, but I have more code I want to run after the native compilation is > finished, and I can't move it all into the hook. Then don't call kill-emacs from the hook; instead, use 'throw', or run the rest of the code from the hook directly. > >> There's another related issue for the packaging use case I mentioned in > >> my original email: The .eln files are placed in ~/.emacs.d/eln-cache/ by > >> default when doing this native compilation. That complicates building > >> and installing these packages independently. Is there a way to put the > >> .eln files for a package in that individual package directory instead? > > > > You need to tweak native-comp-eln-load-path, e.g. prepend to the > > default-value list the directory where you want the *.eln files to be > > written. But you are well advised to leave the other directories in > > the list alone. > > Makes sense. > > Does that mean that, if I want the .eln files to be loaded from the > individual package directory, I have to put every single package > directory on native-comp-eln-load-path at runtime? That seems annoying, > and probably slow. I'm confused. It was you who asked how to avoid placing the *.eln files in ~/.emacs.d/eln-cache/, but now you say it causes problems? "If it hurts, don't do that." I don't understand why the default location is problematic in the first place, and you haven't provided any reasons. And why are you talking about a separate directory for each package? One could envision using a single directory for the *.eln files of all the packages, for example. Whether that is reasonable again depends on your goals and specific cases, which you haven't described. > Is there a way to get an .eln file to be loaded from the directory the > .elc file was found in? No, and it doesn't make sense to do that, because the *.eln files are not in the directories mentioned in native-comp-eln-load-path, they are in versioned subdirectories of those directories, and those subdirectories must be verified by computing the hashes of the corresponding *.el files, to make sure we are not by chance loading incompatible machine code. From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 29 14:05:47 2025 Received: (at 75812) by debbugs.gnu.org; 29 Apr 2025 18:05:47 +0000 Received: from localhost ([127.0.0.1]:60830 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u9pL0-0006EY-7M for submit@debbugs.gnu.org; Tue, 29 Apr 2025 14:05:46 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:59765) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u9pKx-0006DN-Mb for 75812@debbugs.gnu.org; Tue, 29 Apr 2025 14:05:44 -0400 From: Spencer Baugh To: Eli Zaretskii Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs In-Reply-To: <86bjskyask.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 25 Apr 2025 10:12:27 +0300") References: <86plkbwckd.fsf@gnu.org> <86ikmuyvp0.fsf@gnu.org> <86bjskyask.fsf@gnu.org> Date: Tue, 29 Apr 2025 14:05:38 -0400 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=waixah; t=1745949938; bh=vQem/mv6TlYPqtKQvd1Mcqec58LrApVgJZHZTFLy8io=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=n/ILcvw9YVEkDlnL8Y1Q/n4Y7nufCQs0rAf9C54jAZ9lKH/HQ48hEe9Q0ZPWLGXmQ qpDcEKxzXybWWxnsXT9fX+GVdUBAgVZD8W4Tglnvj1ZfwKizA2kZYlUxH6zCu0v9p8 k0iqfEKljezPtr2Xl5FrU27YY2/+WNW43XnoThj5wu8PmHZ21u0KbOUzkcavPtGZ1C psDwjD4SPeV3OjnePhFxpwXJzu6OnZMS5wS4txdUnrtXOybevZ6ddFozEU0qb+L3i0 iPhKMdLkAPwJrw2RzINO4GWWUDwugYg+qsALeb4nRt0mJTNim4bI6AwnQusTfx9AlP ia4gtdLiml4mQ== X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75812 Cc: acorallo@gnu.org, app-emacs-dev@janestreet.com, stefankangas@gmail.com, 75812@debbugs.gnu.org, shipmints@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Eli Zaretskii writes: >> > Or why not install a native-comp-async-all-done-hook that will notice >> > when compilations are done and allow the batch job to exit? All you >> > need is a means to wait until all the async compilations are finished, >> > no? >> >> How do you propose to do that? I guess I could call kill-emacs from the >> hook, but I have more code I want to run after the native compilation is >> finished, and I can't move it all into the hook. > > Then don't call kill-emacs from the hook; instead, use 'throw', or run > the rest of the code from the hook directly. So a function like this? (defun native-comp-async-wait-for-all-done () (unless (zerop (comp--async-runnings)) (catch 'native-comp-done (let ((native-comp-async-all-done-hook (list (lambda () (throw 'native-comp-done))))) (while (sit-for 999)))))) Could we add that? >> >> There's another related issue for the packaging use case I mentioned in >> >> my original email: The .eln files are placed in ~/.emacs.d/eln-cache/ by >> >> default when doing this native compilation. That complicates building >> >> and installing these packages independently. Is there a way to put the >> >> .eln files for a package in that individual package directory instead? >> > >> > You need to tweak native-comp-eln-load-path, e.g. prepend to the >> > default-value list the directory where you want the *.eln files to be >> > written. But you are well advised to leave the other directories in >> > the list alone. >> >> Makes sense. >> >> Does that mean that, if I want the .eln files to be loaded from the >> individual package directory, I have to put every single package >> directory on native-comp-eln-load-path at runtime? That seems annoying, >> and probably slow. > > I'm confused. It was you who asked how to avoid placing the *.eln > files in ~/.emacs.d/eln-cache/, but now you say it causes problems? > "If it hurts, don't do that." I don't understand why the default > location is problematic in the first place, and you haven't provided > any reasons. > > And why are you talking about a separate directory for each package? > One could envision using a single directory for the *.eln files of all > the packages, for example. Whether that is reasonable again depends > on your goals and specific cases, which you haven't described. Yes, it's a matter of compiling each package separately in parallel, which is annoying to do if they all write files to a single directory as a side-effect. But, I think what I could do is compile each package with a different directory prepended to native-comp-eln-load-path, and then once all the packages are compiled, merge those directories into one omnibus directory, which is bundled with my installed Emacs. Would it be reasonable to merge those directories into the last entry of native-comp-eln-load-path? (e.g. /usr/lib/emacs/30.1.50/native-lisp) Or will it break something for there to be more than just bundled elisp in that directory? >> Is there a way to get an .eln file to be loaded from the directory the >> .elc file was found in? > > No, and it doesn't make sense to do that, because the *.eln files are > not in the directories mentioned in native-comp-eln-load-path, they > are in versioned subdirectories of those directories, and those > subdirectories must be verified by computing the hashes of the > corresponding *.el files, to make sure we are not by chance loading > incompatible machine code. That's fair, makes sense. From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 29 14:46:50 2025 Received: (at 75812) by debbugs.gnu.org; 29 Apr 2025 18:46:50 +0000 Received: from localhost ([127.0.0.1]:33273 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u9pyj-00035F-KR for submit@debbugs.gnu.org; Tue, 29 Apr 2025 14:46:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45106) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u9pyc-00033a-LK for 75812@debbugs.gnu.org; Tue, 29 Apr 2025 14:46:46 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9pyW-00050z-Ki; Tue, 29 Apr 2025 14:46:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=fCaxRZ2I+naR2ACAzc0Ib3DOtecd+fmQKr1uOD6MlbE=; b=SljCwqY9nAwj RHiVRvOOUtecceRJXtAYovlc3HvltIC2oVAoPTqeblgbV80v/nCDNomOvLTvkTwXs0u7PKgRgrE9Z aB58TxjxGfh4xz4aH/FToKI4ffWJdeDQYBTXFV9vdoQA+hGuQq3vtnutU+W6CCbv9I0lNVfDpidts fAjRHfCrXymV8UDyCEXnDCQzg4Q0DvZ70gPUvNy9XyvpUk8TtX6A4PZYirOK2TX3re94TMguMZHpM WpMZWp/6b49DDyG8D0/exID/kv1jran0mTkd7kokkDres6ea8rYTXyPVoRlQWIHbVR22tG6x0NQNx HIyvBHLaUlIhAds0qQ3csA==; Date: Tue, 29 Apr 2025 21:45:44 +0300 Message-Id: <865ximq013.fsf@gnu.org> From: Eli Zaretskii To: Spencer Baugh In-Reply-To: (message from Spencer Baugh on Tue, 29 Apr 2025 14:05:38 -0400) Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs References: <86plkbwckd.fsf@gnu.org> <86ikmuyvp0.fsf@gnu.org> <86bjskyask.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75812 Cc: acorallo@gnu.org, app-emacs-dev@janestreet.com, stefankangas@gmail.com, 75812@debbugs.gnu.org, shipmints@gmail.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Spencer Baugh > Cc: app-emacs-dev@janestreet.com, acorallo@gnu.org, > stefankangas@gmail.com, 75812@debbugs.gnu.org, shipmints@gmail.com > Date: Tue, 29 Apr 2025 14:05:38 -0400 > > Eli Zaretskii writes: > >> > Or why not install a native-comp-async-all-done-hook that will notice > >> > when compilations are done and allow the batch job to exit? All you > >> > need is a means to wait until all the async compilations are finished, > >> > no? > >> > >> How do you propose to do that? I guess I could call kill-emacs from the > >> hook, but I have more code I want to run after the native compilation is > >> finished, and I can't move it all into the hook. > > > > Then don't call kill-emacs from the hook; instead, use 'throw', or run > > the rest of the code from the hook directly. > > So a function like this? > > (defun native-comp-async-wait-for-all-done () > (unless (zerop (comp--async-runnings)) > (catch 'native-comp-done > (let ((native-comp-async-all-done-hook (list (lambda () (throw 'native-comp-done))))) > (while (sit-for 999)))))) > > Could we add that? Why do we need to add it? Can't you just use it in your code? > Yes, it's a matter of compiling each package separately in parallel, > which is annoying to do if they all write files to a single directory as > a side-effect. > > But, I think what I could do is compile each package with a different > directory prepended to native-comp-eln-load-path, and then once all the > packages are compiled, merge those directories into one omnibus > directory, which is bundled with my installed Emacs. I still don't understand what you are trying to achieve, but the above should work. > Would it be reasonable to merge those directories into the last entry of > native-comp-eln-load-path? (e.g. /usr/lib/emacs/30.1.50/native-lisp) No! The last directory is in many cases not writable by users, and it includes preloaded files, see the doc string of the variable. The last directory should be left alone. Instead, add another directory to the default value, between the user's eln-cache and the last member, and have your local packages' *.eln files there. From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 29 15:28:19 2025 Received: (at 75812) by debbugs.gnu.org; 29 Apr 2025 19:28:20 +0000 Received: from localhost ([127.0.0.1]:33963 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u9qct-0008Lf-Bu for submit@debbugs.gnu.org; Tue, 29 Apr 2025 15:28:19 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:37905) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u9qcr-0008LB-76 for 75812@debbugs.gnu.org; Tue, 29 Apr 2025 15:28:17 -0400 From: Spencer Baugh To: Eli Zaretskii Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs In-Reply-To: <865ximq013.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 29 Apr 2025 21:45:44 +0300") References: <86plkbwckd.fsf@gnu.org> <86ikmuyvp0.fsf@gnu.org> <86bjskyask.fsf@gnu.org> <865ximq013.fsf@gnu.org> Date: Tue, 29 Apr 2025 15:28:11 -0400 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=waixah; t=1745954892; bh=P+l1neQzoCuGyYmbM5Pq5NY+JBhbVGxl5JPCeNiFDt0=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=dKDvm9swYQU1wviT3IWDSfg70mS/DRhGYfYbejD55Pn1KbDn9LWoMLKiGCnU6aEFv 2f5CuBHCpnDt52aFKn81/H5MAedmDM4Cl0/te3+sHw/xvbb5Pac+nka2am9X7w/zDu VjoGaWwDMKlmvMP/yQCZyKK26pme65aaGhzZX9nx6dV+vUy+PG7eUDcKlShUzB17/m M976ZcrXeXAO/ivdoKIuK+dEtZ7wfIky9/8fKSbKMMWWZAeRvd4bKvonFt/MwH7BuF l4aKg/cjwYBmDqpjIntPV31NGTyZIU/9Ynx52rLT1pGp8b3bmqP8ICmZn8HdiQ4fRn 5DTj9hvNpVmAQ== X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75812 Cc: shipmints@gmail.com, acorallo@gnu.org, stefankangas@gmail.com, 75812@debbugs.gnu.org, app-emacs-dev@janestreet.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Eli Zaretskii writes: >> From: Spencer Baugh >> Cc: app-emacs-dev@janestreet.com, acorallo@gnu.org, >> stefankangas@gmail.com, 75812@debbugs.gnu.org, shipmints@gmail.com >> Date: Tue, 29 Apr 2025 14:05:38 -0400 >> >> Eli Zaretskii writes: >> >> > Or why not install a native-comp-async-all-done-hook that will notice >> >> > when compilations are done and allow the batch job to exit? All you >> >> > need is a means to wait until all the async compilations are finished, >> >> > no? >> >> >> >> How do you propose to do that? I guess I could call kill-emacs from the >> >> hook, but I have more code I want to run after the native compilation is >> >> finished, and I can't move it all into the hook. >> > >> > Then don't call kill-emacs from the hook; instead, use 'throw', or run >> > the rest of the code from the hook directly. >> >> So a function like this? >> >> (defun native-comp-async-wait-for-all-done () >> (unless (zerop (comp--async-runnings)) >> (catch 'native-comp-done >> (let ((native-comp-async-all-done-hook (list (lambda () (throw 'native-comp-done))))) >> (while (sit-for 999)))))) >> >> Could we add that? > > Why do we need to add it? Can't you just use it in your code? Since others in this thread have said they would like to be able to wait monitor the completion of native compilation. It seems like it would be good to have a high quality implementation be part of Emacs. (and frankly, I don't think throwing out of a hook as a means of control flow is a good idea, and I'd like to do something better than that) >> Yes, it's a matter of compiling each package separately in parallel, >> which is annoying to do if they all write files to a single directory as >> a side-effect. >> >> But, I think what I could do is compile each package with a different >> directory prepended to native-comp-eln-load-path, and then once all the >> packages are compiled, merge those directories into one omnibus >> directory, which is bundled with my installed Emacs. > > I still don't understand what you are trying to achieve, but the above > should work. Great, thank you for confirming! >> Would it be reasonable to merge those directories into the last entry of >> native-comp-eln-load-path? (e.g. /usr/lib/emacs/30.1.50/native-lisp) > > No! The last directory is in many cases not writable by users, and it > includes preloaded files, see the doc string of the variable. The > last directory should be left alone. Instead, add another directory > to the default value, between the user's eln-cache and the last > member, and have your local packages' *.eln files there. I'm doing this package precompilation at Emacs build time, and the last directory is writable at that time. After installation, of course, it is not writable, but that doesn't matter since only these precompiled package .elns are going to be written there. In general, I have site-lisp that is byte-compiled and native-compiled, and it seems like the right place to write .eln files for site-lisp is the last directory on native-comp-eln-load-path. After all, site-lisp is effectively bundled with Emacs, set up at install time just like core Elisp. Is there some issue with doing that? From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 30 01:49:37 2025 Received: (at 75812) by debbugs.gnu.org; 30 Apr 2025 05:49:37 +0000 Received: from localhost ([127.0.0.1]:38597 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uA0K9-0005nJ-9Q for submit@debbugs.gnu.org; Wed, 30 Apr 2025 01:49:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39570) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uA0K6-0005mx-Hd for 75812@debbugs.gnu.org; Wed, 30 Apr 2025 01:49:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uA0K0-00007V-Hj; Wed, 30 Apr 2025 01:49:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=uS0Fae5Mcty/Qqu/p4M6s7x9hJbvRcgEd13dl6a6AoM=; b=kvC+KuaQ6XRC y/1itT8BSJEpviZcu6vVvyx54cvbB67C+grV035UBGN37JhGa2VA7GAOZMYB0v+/60Ho0f4HTE5j7 FxgcL5zYQreYmmdPM74EZ3pF6zkTcz0UdCE4447z0AQ2WmZ55gjiYyRQQyH6FT3zjD8Va8A4jduvM Xix8u9vFkLImRbjaGKOJF2BJ8wANkAH8PyrYH1LYz24ock9Gc6yssSAqNw7R6GRO7Ht0NBsFSN2UO l7mfjxOJcKDaDFxsGJgGcWJ72Q+4Q0q1BLevhnHJeDpyjZ0zoD6iD7mVBObXkMmn97d+8t5ezoISY ZDZ+9PVhzQobR1aESiaxHw==; Date: Wed, 30 Apr 2025 08:49:21 +0300 Message-Id: <8634dqp5b2.fsf@gnu.org> From: Eli Zaretskii To: Spencer Baugh In-Reply-To: (message from Spencer Baugh on Tue, 29 Apr 2025 15:28:11 -0400) Subject: Re: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs References: <86plkbwckd.fsf@gnu.org> <86ikmuyvp0.fsf@gnu.org> <86bjskyask.fsf@gnu.org> <865ximq013.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75812 Cc: shipmints@gmail.com, acorallo@gnu.org, stefankangas@gmail.com, 75812@debbugs.gnu.org, app-emacs-dev@janestreet.com X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Spencer Baugh > Cc: acorallo@gnu.org, app-emacs-dev@janestreet.com, > stefankangas@gmail.com, 75812@debbugs.gnu.org, shipmints@gmail.com > Date: Tue, 29 Apr 2025 15:28:11 -0400 > > Eli Zaretskii writes: > > >> (defun native-comp-async-wait-for-all-done () > >> (unless (zerop (comp--async-runnings)) > >> (catch 'native-comp-done > >> (let ((native-comp-async-all-done-hook (list (lambda () (throw 'native-comp-done))))) > >> (while (sit-for 999)))))) > >> > >> Could we add that? > > > > Why do we need to add it? Can't you just use it in your code? > > Since others in this thread have said they would like to be able to wait > monitor the completion of native compilation. It seems like it would be > good to have a high quality implementation be part of Emacs. (and > frankly, I don't think throwing out of a hook as a means of control flow > is a good idea, and I'd like to do something better than that) I'm not sure we can come up with a general enough function for this purpose, because what the caller wants to do while waiting will differ from application to application. In the above case the caller just waits, but that's not necessarily what everyone will want to do. So if you want to propose a general-purpose function, we need more thorough design and a better abstraction, IMO. Without that, the above is just a thin wrapper around an existing hook. > >> Would it be reasonable to merge those directories into the last entry of > >> native-comp-eln-load-path? (e.g. /usr/lib/emacs/30.1.50/native-lisp) > > > > No! The last directory is in many cases not writable by users, and it > > includes preloaded files, see the doc string of the variable. The > > last directory should be left alone. Instead, add another directory > > to the default value, between the user's eln-cache and the last > > member, and have your local packages' *.eln files there. > > I'm doing this package precompilation at Emacs build time, and the last > directory is writable at that time. After installation, of course, it > is not writable, but that doesn't matter since only these precompiled > package .elns are going to be written there. > > In general, I have site-lisp that is byte-compiled and native-compiled, > and it seems like the right place to write .eln files for site-lisp is > the last directory on native-comp-eln-load-path. After all, site-lisp > is effectively bundled with Emacs, set up at install time just like core > Elisp. Is there some issue with doing that? You are invalidating the assumption that the last directory is special in that it holds the results of compiling the Lisp files that are bundled with Emacs. You asked me whether this can be done, and my warm recommendation is not to do that. You can do whatever you want regardless, of course, but then please don't come back reporting what you think is a bug which was caused by that, because placing other *.eln files in that directory is not supported.