From unknown Sun Jun 22 08:06:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#45102] [PATCH 0/4] Making fewer 'stat' calls during startup Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 07 Dec 2020 15:23:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 45102 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45102@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.160735453414242 (code B ref -1); Mon, 07 Dec 2020 15:23:01 +0000 Received: (at submit) by debbugs.gnu.org; 7 Dec 2020 15:22:14 +0000 Received: from localhost ([127.0.0.1]:54898 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIL8-0003hd-Gv for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:22:14 -0500 Received: from lists.gnu.org ([209.51.188.17]:54886) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIL4-0003hT-Vh for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:22:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39324) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kmIL4-0004o7-Ok for guix-patches@gnu.org; Mon, 07 Dec 2020 10:22:10 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:56893) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmIL3-00030z-LW; Mon, 07 Dec 2020 10:22:10 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48406 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kmIL1-0001wk-Nv; Mon, 07 Dec 2020 10:22:08 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Mon, 7 Dec 2020 16:21:59 +0100 Message-Id: <20201207152159.28551-1-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Guix! On Guix System, with 6 entries in $GUILE_LOAD_PATH (which should just be 2 entries, one for /run/current-system and one for ~/.guix-profile), I see this: --8<---------------cut here---------------start------------->8--- $ strace -c -e stat,openat guix guix: mankas komanda nomo Provu 'guix --help' por pli da informo. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 90.94 0.002207 0 2283 2057 stat 9.06 0.000220 0 334 177 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.002427 0 2617 2234 total --8<---------------cut here---------------end--------------->8--- This is a lot of ‘stat’ calls for nothing, 4 times more than when GUILE_LOAD_PATH is unset: --8<---------------cut here---------------start------------->8--- $ env -i $(type -P strace) -c -e stat,openat $(type -P guix) guix: missing command name Try `guix --help' for more information. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 57.89 0.000187 0 495 272 stat 42.11 0.000136 0 321 175 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000323 0 816 447 total --8<---------------cut here---------------end--------------->8--- This patch series takes a sledgehammer approach to always have as few ‘stat’ calls as possible during startup. After the change, I get: --8<---------------cut here---------------start------------->8--- $ strace -e stat,openat -c /gnu/store/g9gylj723si8i2cp8ia57a7kr4i8b1m9-guix-20201207.14/bin/guix guix: mankas komanda nomo Provu 'guix --help' por pli da informo. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 60.53 0.000322 0 454 231 stat 39.47 0.000210 0 334 177 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000532 0 788 408 total $ env -i $(type -P strace) -e stat,openat -c /gnu/store/g9gylj723si8i2cp8ia57a7kr4i8b1m9-guix-20201207.14/bin/guix guix: missing command name Try `guix --help' for more information. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 69.92 0.000272 0 442 219 stat 30.08 0.000117 0 321 175 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000389 0 763 394 total --8<---------------cut here---------------end--------------->8--- What impact does it have on startup time? Hard to tell. On my 4-year old x86_64 laptop with an SSD, it seems to be negligible, even on a cold cache. I suspect things are different on spinning disks and on NFS. The semantic difference should be invisible to users: their modules are still visible and usable in manifests, in ‘guix repl’, etc.; they just cannot take precedence over modules from Guile and from the channels. For instance, one cannot define a (gnu packages base) module or (ice-9 rdelim), drop it in $GUILE_LOAD_PATH, and have that module picked up in lieu of the original one. I think that’s a reasonable tradeoff. Thoughts? Ludo’. Ludovic Courtès (4): build: 'script/guix' uses our own 'guile' executable. self: Move Guile early in the module search path. guix: 'guile' executable ignores GUILE_LOAD_PATH during startup. self: Remove the empty string from '%load-extensions'. Makefile.am | 1 + gnu/packages/aux-files/guile-launcher.c | 46 +++++++++++++++++++++++-- guix/self.scm | 26 +++++++++----- scripts/guix.in | 2 +- 4 files changed, 63 insertions(+), 12 deletions(-) -- 2.29.2 From unknown Sun Jun 22 08:06:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#45102] [PATCH 1/4] build: 'script/guix' uses our own 'guile' executable. References: <20201207152159.28551-1-ludo@gnu.org> In-Reply-To: <20201207152159.28551-1-ludo@gnu.org> Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 07 Dec 2020 15:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45102 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45102@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 45102-submit@debbugs.gnu.org id=B45102.160735501815173 (code B ref 45102); Mon, 07 Dec 2020 15:31:02 +0000 Received: (at 45102) by debbugs.gnu.org; 7 Dec 2020 15:30:18 +0000 Received: from localhost ([127.0.0.1]:54936 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmISv-0003wa-Mt for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:30:18 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51990) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmISs-0003w5-RP for 45102@debbugs.gnu.org; Mon, 07 Dec 2020 10:30:15 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57163) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmISn-0005vL-Km; Mon, 07 Dec 2020 10:30:09 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48438 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kmISj-0000cR-HU; Mon, 07 Dec 2020 10:30:07 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Mon, 7 Dec 2020 16:29:56 +0100 Message-Id: <20201207152959.28864-1-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * Makefile.am (do_subst): Substitute @abs_top_builddir@. * scripts/guix.in: Use it. --- Makefile.am | 1 + scripts/guix.in | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/Makefile.am b/Makefile.am index 1a3ca227a4..9803ba5dc7 100644 --- a/Makefile.am +++ b/Makefile.am @@ -39,6 +39,7 @@ do_subst = $(SED) \ -e 's,[@]GUILE[@],$(GUILE),g' \ -e 's,[@]guilemoduledir[@],$(guilemoduledir),g' \ -e 's,[@]guileobjectdir[@],$(guileobjectdir),g' \ + -e 's,[@]abs_top_builddir[@],$(abs_top_builddir),g' \ -e 's,[@]localedir[@],$(localedir),g' scripts/guix: scripts/guix.in Makefile diff --git a/scripts/guix.in b/scripts/guix.in index 0a3ab1f64d..e0194d6ea2 100644 --- a/scripts/guix.in +++ b/scripts/guix.in @@ -1,4 +1,4 @@ -#!@GUILE@ \ +#!@abs_top_builddir@/guile \ --no-auto-compile -e main -s !# ;;; GNU Guix --- Functional package management for GNU -- 2.29.2 From unknown Sun Jun 22 08:06:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#45102] [PATCH 2/4] self: Move Guile early in the module search path. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 07 Dec 2020 15:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45102 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45102@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 45102-submit@debbugs.gnu.org id=B45102.160735501915183 (code B ref 45102); Mon, 07 Dec 2020 15:31:02 +0000 Received: (at 45102) by debbugs.gnu.org; 7 Dec 2020 15:30:19 +0000 Received: from localhost ([127.0.0.1]:54939 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmISx-0003wp-5c for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:30:19 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51994) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmISv-0003wL-RO for 45102@debbugs.gnu.org; Mon, 07 Dec 2020 10:30:18 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57174) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmISq-0005wB-9C; Mon, 07 Dec 2020 10:30:12 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48438 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kmISo-0000cR-82; Mon, 07 Dec 2020 10:30:11 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Mon, 7 Dec 2020 16:29:57 +0100 Message-Id: <20201207152959.28864-2-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201207152959.28864-1-ludo@gnu.org> References: <20201207152959.28864-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Until now, Guile modules would first be searched for in MODULE-DIRECTORY, then in each $GUILE_LOAD_PATH entry, and finally in Guile itself. * guix/self.scm (guix-command): Make GUILE the second entry in the %LOAD-PATH and %LOAD-COMPILED-PATH. --- guix/self.scm | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/guix/self.scm b/guix/self.scm index c0de14b79a..ca67f653fa 100644 --- a/guix/self.scm +++ b/guix/self.scm @@ -650,17 +650,22 @@ load path." (program-file "guix-command" #~(begin (set! %load-path - (cons (string-append #$module-directory - "/share/guile/site/" - (effective-version)) - %load-path)) + (append (list (string-append #$module-directory + "/share/guile/site/" + (effective-version)) + (string-append #$guile "/share/guile/" + (effective-version))) + %load-path)) (set! %load-compiled-path - (cons (string-append #$module-directory - "/lib/guile/" - (effective-version) - "/site-ccache") - %load-compiled-path)) + (append (list (string-append #$module-directory + "/lib/guile/" + (effective-version) + "/site-ccache") + (string-append #$guile "/lib/guile/" + (effective-version) + "/ccache")) + %load-compiled-path)) ;; To maximize the chances that locales are set up right ;; out-of-the-box, bundle "common" UTF-8 locales. -- 2.29.2 From unknown Sun Jun 22 08:06:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#45102] [PATCH 3/4] guix: 'guile' executable ignores GUILE_LOAD_PATH during startup. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 07 Dec 2020 15:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45102 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45102@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 45102-submit@debbugs.gnu.org id=B45102.160735502215199 (code B ref 45102); Mon, 07 Dec 2020 15:31:02 +0000 Received: (at 45102) by debbugs.gnu.org; 7 Dec 2020 15:30:22 +0000 Received: from localhost ([127.0.0.1]:54942 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIT0-0003x5-EF for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:30:22 -0500 Received: from eggs.gnu.org ([209.51.188.92]:52000) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmISy-0003wR-KM for 45102@debbugs.gnu.org; Mon, 07 Dec 2020 10:30:20 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57179) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmISt-0005xC-EF; Mon, 07 Dec 2020 10:30:15 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48438 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kmISq-0000cR-J7; Mon, 07 Dec 2020 10:30:13 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Mon, 7 Dec 2020 16:29:58 +0100 Message-Id: <20201207152959.28864-3-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201207152959.28864-1-ludo@gnu.org> References: <20201207152959.28864-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) When starting the 'guix' command, this ensures Guile modules are immediately found instead of being search for in other directories. This reduces the number of 'stat' calls during startup when GUILE_LOAD_PATH is set to (almost) that of "env -i $(type -P guix)". * gnu/packages/aux-files/guile-launcher.c (load_path) (load_compiled_path): New variables. (inner_main): Restore GUILE_LOAD_PATH and GUILE_LOAD_COMPILED_PATH and set %load-path and %load-compiled-path accordingly. (main): Save GUILE_LOAD_PATH and GUILE_LOAD_COMPILED_PATH and unset them. --- gnu/packages/aux-files/guile-launcher.c | 46 +++++++++++++++++++++++-- 1 file changed, 44 insertions(+), 2 deletions(-) diff --git a/gnu/packages/aux-files/guile-launcher.c b/gnu/packages/aux-files/guile-launcher.c index 886ede2846..1dd5d77e66 100644 --- a/gnu/packages/aux-files/guile-launcher.c +++ b/gnu/packages/aux-files/guile-launcher.c @@ -1,5 +1,5 @@ /* GNU Guix --- Functional package management for GNU - Copyright 1996-1997,2000-2001,2006,2008,2011,2013,2018 + Copyright 1996-1997,2000-2001,2006,2008,2011,2013,2018,2020 Free Software Foundation, Inc. Copyright (C) 2020 Ludovic Courtès @@ -19,14 +19,47 @@ along with GNU Guix. If not, see . */ /* This file implements a variant of the 'guile' executable that does not - complain about locale issues. */ + complain about locale issues and arranges to reduce startup time by + ignoring GUILE_LOAD_PATH and GUILE_LOAD_COMPILED_PATH until it has + booted. */ +#include +#include #include #include +/* Saved values of GUILE_LOAD_PATH and GUILE_LOAD_COMPILED_PATH. */ +static const char *load_path, *load_compiled_path; + static void inner_main (void *unused, int argc, char **argv) { + if (load_path != NULL) + { + setenv ("GUILE_LOAD_PATH", load_path, 1); + SCM load_path_var = + scm_c_public_lookup ("guile", "%load-path"); + SCM addition = + scm_parse_path (scm_from_locale_string (load_path), SCM_EOL); + scm_variable_set_x (load_path_var, + scm_append + (scm_list_2 (scm_variable_ref (load_path_var), + addition))); + } + + if (load_compiled_path != NULL) + { + setenv ("GUILE_LOAD_COMPILED_PATH", load_compiled_path, 1); + SCM load_compiled_path_var = + scm_c_public_lookup ("guile", "%load-compiled-path"); + SCM addition = + scm_parse_path (scm_from_locale_string (load_compiled_path), SCM_EOL); + scm_variable_set_x (load_compiled_path_var, + scm_append + (scm_list_2 (scm_variable_ref (load_compiled_path_var), + addition))); + } + scm_shell (argc, argv); } @@ -40,6 +73,15 @@ main (int argc, char **argv) which is always preferable over the C locale. */ setlocale (LC_ALL, "en_US.utf8"); + const char *str; + str = getenv ("GUILE_LOAD_PATH"); + load_path = str != NULL ? strdup (str) : NULL; + str = getenv ("GUILE_LOAD_COMPILED_PATH"); + load_compiled_path = str ? strdup (str) : NULL; + + unsetenv ("GUILE_LOAD_PATH"); + unsetenv ("GUILE_LOAD_COMPILED_PATH"); + scm_install_gmp_memory_functions = 1; scm_boot_guile (argc, argv, inner_main, 0); return 0; /* never reached */ -- 2.29.2 From unknown Sun Jun 22 08:06:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#45102] [PATCH 4/4] self: Remove the empty string from '%load-extensions'. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 07 Dec 2020 15:31:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45102 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45102@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 45102-submit@debbugs.gnu.org id=B45102.160735503215223 (code B ref 45102); Mon, 07 Dec 2020 15:31:03 +0000 Received: (at 45102) by debbugs.gnu.org; 7 Dec 2020 15:30:32 +0000 Received: from localhost ([127.0.0.1]:54945 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIT9-0003xS-Pb for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:30:32 -0500 Received: from eggs.gnu.org ([209.51.188.92]:52040) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIT8-0003xE-2Y for 45102@debbugs.gnu.org; Mon, 07 Dec 2020 10:30:30 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57189) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmIT2-0005zC-6i; Mon, 07 Dec 2020 10:30:24 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48438 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kmISs-0000cR-C5; Mon, 07 Dec 2020 10:30:21 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Mon, 7 Dec 2020 16:29:59 +0100 Message-Id: <20201207152959.28864-4-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201207152959.28864-1-ludo@gnu.org> References: <20201207152959.28864-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/self.scm (guix-command): Set '%load-extensions'. --- guix/self.scm | 3 +++ 1 file changed, 3 insertions(+) diff --git a/guix/self.scm b/guix/self.scm index ca67f653fa..7cda6656c9 100644 --- a/guix/self.scm +++ b/guix/self.scm @@ -649,6 +649,9 @@ load path." (program-file "guix-command" #~(begin + ;; Remove the empty extension from the search path. + (set! %load-extensions '(".scm")) + (set! %load-path (append (list (string-append #$module-directory "/share/guile/site/" -- 2.29.2 From unknown Sun Jun 22 08:06:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#45102] [PATCH 0/4] Making fewer 'stat' calls during startup Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 08 Dec 2020 13:51:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45102 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 45102@debbugs.gnu.org Received: via spool by 45102-submit@debbugs.gnu.org id=B45102.160743542321981 (code B ref 45102); Tue, 08 Dec 2020 13:51:02 +0000 Received: (at 45102) by debbugs.gnu.org; 8 Dec 2020 13:50:23 +0000 Received: from localhost ([127.0.0.1]:56920 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmdNm-0005iS-T6 for submit@debbugs.gnu.org; Tue, 08 Dec 2020 08:50:23 -0500 Received: from eggs.gnu.org ([209.51.188.92]:52796) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmdNi-0005i3-Un for 45102@debbugs.gnu.org; Tue, 08 Dec 2020 08:50:22 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49976) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmdNc-0002da-3Q; Tue, 08 Dec 2020 08:50:12 -0500 Received: from [2a01:e0a:19b:d9a0:1037:ae1:62bd:ffe9] (port=53314 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kmdNZ-0005FA-Ov; Tue, 08 Dec 2020 08:50:11 -0500 From: Mathieu Othacehe References: <20201207152159.28551-1-ludo@gnu.org> Date: Tue, 08 Dec 2020 14:50:08 +0100 In-Reply-To: <20201207152159.28551-1-ludo@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Mon, 7 Dec 2020 16:21:59 +0100") Message-ID: <871rg0tnjj.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hey Ludo, > The semantic difference should be invisible to users: their modules > are still visible and usable in manifests, in =E2=80=98guix repl=E2=80=99= , etc.; they > just cannot take precedence over modules from Guile and from the > channels. For instance, one cannot define a (gnu packages base) > module or (ice-9 rdelim), drop it in $GUILE_LOAD_PATH, and have that > module picked up in lieu of the original one. I think that=E2=80=99s a > reasonable tradeoff. I think that's reasonable too. I tested it locally and have the following command: --8<---------------cut here---------------start------------->8--- strace -c -e stat,openat ./pre-inst-env guix --8<---------------cut here---------------end--------------->8--- drop from 1671 calls to 1017 calls, which is nice. The patchset looks good to me. Thanks, Mathieu From unknown Sun Jun 22 08:06:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#45102] [PATCH 0/4] Making fewer 'stat' calls during startup Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 08 Dec 2020 20:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45102 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Mathieu Othacehe Cc: 45102@debbugs.gnu.org Received: via spool by 45102-submit@debbugs.gnu.org id=B45102.160745907824292 (code B ref 45102); Tue, 08 Dec 2020 20:25:02 +0000 Received: (at 45102) by debbugs.gnu.org; 8 Dec 2020 20:24:38 +0000 Received: from localhost ([127.0.0.1]:59930 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmjXK-0006Jk-Fy for submit@debbugs.gnu.org; Tue, 08 Dec 2020 15:24:38 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38932) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmjXI-0006JV-O2 for 45102@debbugs.gnu.org; Tue, 08 Dec 2020 15:24:37 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58602) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmjXD-000463-GE for 45102@debbugs.gnu.org; Tue, 08 Dec 2020 15:24:31 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=53432 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kmjXC-00034l-PX; Tue, 08 Dec 2020 15:24:30 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20201207152159.28551-1-ludo@gnu.org> <871rg0tnjj.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 18 Frimaire an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 08 Dec 2020 21:24:29 +0100 In-Reply-To: <871rg0tnjj.fsf@gnu.org> (Mathieu Othacehe's message of "Tue, 08 Dec 2020 14:50:08 +0100") Message-ID: <875z5cujuq.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Mathieu Othacehe skribis: >> The semantic difference should be invisible to users: their modules >> are still visible and usable in manifests, in =E2=80=98guix repl=E2=80= =99, etc.; they >> just cannot take precedence over modules from Guile and from the >> channels. For instance, one cannot define a (gnu packages base) >> module or (ice-9 rdelim), drop it in $GUILE_LOAD_PATH, and have that >> module picked up in lieu of the original one. I think that=E2=80=99s a >> reasonable tradeoff. > > I think that's reasonable too. I tested it locally and have the > following command: > > strace -c -e stat,openat ./pre-inst-env guix > > drop from 1671 calls to 1017 calls, which is nice. You should try: ./pre-inst-env strace -c stat guix but even then you=E2=80=99ll get more =E2=80=98stat=E2=80=99 calls than the= =E2=80=98guix=E2=80=99 command provided by =E2=80=98guix pull=E2=80=99. You can run, say: strace -c $(make as-derivation)/bin/guix > The patchset looks good to me. Cool, thanks! Ludo=E2=80=99. From unknown Sun Jun 22 08:06:26 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Subject: bug#45102: closed (Re: [bug#45102] [PATCH 0/4] Making fewer 'stat' calls during startup) Message-ID: References: <875z5843ga.fsf@gnu.org> <20201207152159.28551-1-ludo@gnu.org> X-Gnu-PR-Message: they-closed 45102 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 45102@debbugs.gnu.org Date: Fri, 11 Dec 2020 18:13:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1607710382-31713-1" This is a multi-part message in MIME format... ------------=_1607710382-31713-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #45102: [PATCH 0/4] Making fewer 'stat' calls during startup which was filed against the guix-patches package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 45102@debbugs.gnu.org. --=20 45102: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D45102 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1607710382-31713-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 45102-done) by debbugs.gnu.org; 11 Dec 2020 18:12:34 +0000 Received: from localhost ([127.0.0.1]:42895 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1knmu9-0008Ep-TM for submit@debbugs.gnu.org; Fri, 11 Dec 2020 13:12:34 -0500 Received: from eggs.gnu.org ([209.51.188.92]:44262) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1knmu8-0008Ea-VF for 45102-done@debbugs.gnu.org; Fri, 11 Dec 2020 13:12:33 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51360) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmu3-0007SN-Qo for 45102-done@debbugs.gnu.org; Fri, 11 Dec 2020 13:12:27 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=32936 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1knmu1-0007Fb-3J; Fri, 11 Dec 2020 13:12:25 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: [bug#45102] [PATCH 0/4] Making fewer 'stat' calls during startup References: <20201207152159.28551-1-ludo@gnu.org> <871rg0tnjj.fsf@gnu.org> <875z5cujuq.fsf@gnu.org> Date: Fri, 11 Dec 2020 19:12:21 +0100 In-Reply-To: <875z5cujuq.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Tue, 08 Dec 2020 21:24:29 +0100") Message-ID: <875z5843ga.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45102-done Cc: 45102-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Pushed as 41d01b4e2e74a3e655bac03c241f0de7cb34b75f! Ludo=E2=80=99. ------------=_1607710382-31713-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 7 Dec 2020 15:22:14 +0000 Received: from localhost ([127.0.0.1]:54898 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIL8-0003hd-Gv for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:22:14 -0500 Received: from lists.gnu.org ([209.51.188.17]:54886) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIL4-0003hT-Vh for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:22:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39324) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kmIL4-0004o7-Ok for guix-patches@gnu.org; Mon, 07 Dec 2020 10:22:10 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:56893) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmIL3-00030z-LW; Mon, 07 Dec 2020 10:22:10 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48406 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kmIL1-0001wk-Nv; Mon, 07 Dec 2020 10:22:08 -0500 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: guix-patches@gnu.org Subject: [PATCH 0/4] Making fewer 'stat' calls during startup Date: Mon, 7 Dec 2020 16:21:59 +0100 Message-Id: <20201207152159.28551-1-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Guix! On Guix System, with 6 entries in $GUILE_LOAD_PATH (which should just be 2 entries, one for /run/current-system and one for ~/.guix-profile), I see this: --8<---------------cut here---------------start------------->8--- $ strace -c -e stat,openat guix guix: mankas komanda nomo Provu 'guix --help' por pli da informo. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 90.94 0.002207 0 2283 2057 stat 9.06 0.000220 0 334 177 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.002427 0 2617 2234 total --8<---------------cut here---------------end--------------->8--- This is a lot of ‘stat’ calls for nothing, 4 times more than when GUILE_LOAD_PATH is unset: --8<---------------cut here---------------start------------->8--- $ env -i $(type -P strace) -c -e stat,openat $(type -P guix) guix: missing command name Try `guix --help' for more information. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 57.89 0.000187 0 495 272 stat 42.11 0.000136 0 321 175 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000323 0 816 447 total --8<---------------cut here---------------end--------------->8--- This patch series takes a sledgehammer approach to always have as few ‘stat’ calls as possible during startup. After the change, I get: --8<---------------cut here---------------start------------->8--- $ strace -e stat,openat -c /gnu/store/g9gylj723si8i2cp8ia57a7kr4i8b1m9-guix-20201207.14/bin/guix guix: mankas komanda nomo Provu 'guix --help' por pli da informo. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 60.53 0.000322 0 454 231 stat 39.47 0.000210 0 334 177 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000532 0 788 408 total $ env -i $(type -P strace) -e stat,openat -c /gnu/store/g9gylj723si8i2cp8ia57a7kr4i8b1m9-guix-20201207.14/bin/guix guix: missing command name Try `guix --help' for more information. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 69.92 0.000272 0 442 219 stat 30.08 0.000117 0 321 175 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000389 0 763 394 total --8<---------------cut here---------------end--------------->8--- What impact does it have on startup time? Hard to tell. On my 4-year old x86_64 laptop with an SSD, it seems to be negligible, even on a cold cache. I suspect things are different on spinning disks and on NFS. The semantic difference should be invisible to users: their modules are still visible and usable in manifests, in ‘guix repl’, etc.; they just cannot take precedence over modules from Guile and from the channels. For instance, one cannot define a (gnu packages base) module or (ice-9 rdelim), drop it in $GUILE_LOAD_PATH, and have that module picked up in lieu of the original one. I think that’s a reasonable tradeoff. Thoughts? Ludo’. Ludovic Courtès (4): build: 'script/guix' uses our own 'guile' executable. self: Move Guile early in the module search path. guix: 'guile' executable ignores GUILE_LOAD_PATH during startup. self: Remove the empty string from '%load-extensions'. Makefile.am | 1 + gnu/packages/aux-files/guile-launcher.c | 46 +++++++++++++++++++++++-- guix/self.scm | 26 +++++++++----- scripts/guix.in | 2 +- 4 files changed, 63 insertions(+), 12 deletions(-) -- 2.29.2 ------------=_1607710382-31713-1--