From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 02 14:52:12 2020 Received: (at submit) by debbugs.gnu.org; 2 Jun 2020 18:52:12 +0000 Received: from localhost ([127.0.0.1]:40959 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jgC1D-0007A7-UP for submit@debbugs.gnu.org; Tue, 02 Jun 2020 14:52:12 -0400 Received: from lists.gnu.org ([209.51.188.17]:42260) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jgC1C-00079z-HJ for submit@debbugs.gnu.org; Tue, 02 Jun 2020 14:52:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56238) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgC1C-00077B-90 for bug-guix@gnu.org; Tue, 02 Jun 2020 14:52:10 -0400 Received: from relay2-d.mail.gandi.net ([217.70.183.194]:50479) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgC1A-0005w9-H9; Tue, 02 Jun 2020 14:52:09 -0400 Received: from webmail.gandi.net (webmail18.sd4.0x35.net [10.200.201.18]) (Authenticated sender: brice@waegenei.re) by relay2-d.mail.gandi.net (Postfix) with ESMTPA id 6560440002; Tue, 2 Jun 2020 18:52:04 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 02 Jun 2020 18:52:04 +0000 From: Brice Waegeneire To: bug-guix@gnu.org Subject: Failing test: gui-installed-desktop-os-encrypted Message-ID: <3f66774219bb18c47b305b8760f58abe@waegenei.re> X-Sender: brice@waegenei.re User-Agent: Roundcube Webmail/1.3.8 Received-SPF: pass client-ip=217.70.183.194; envelope-from=brice@waegenei.re; helo=relay2-d.mail.gandi.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/02 14:52:04 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-Spam-Score: -1.6 (-) X-Debbugs-Envelope-To: submit Cc: ludo@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.6 (--) Hello Guix, For some time now the system test 'gui-installed-desktop-os-encrypted' doesn't succeed, the last time it passed was 2020-04-11[0]. This is problematic since it exercise a lot of packages and thus can be used to test those packages. Ludovic attempted to fixed it[1] but it didn't fixed the test on my side. [0]: http://ci.guix.gnu.org/search?query=gui-installed-desktop-os-encrypted+system%3Ax86_64-linux+spec%3Aguix-master+status%3Asuccess [1]: https://issues.guix.info/40790#7 - Brice From debbugs-submit-bounces@debbugs.gnu.org Sat Jun 06 06:16:55 2020 Received: (at 41668) by debbugs.gnu.org; 6 Jun 2020 10:16:55 +0000 Received: from localhost ([127.0.0.1]:50779 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhVsi-0002x3-1o for submit@debbugs.gnu.org; Sat, 06 Jun 2020 06:16:55 -0400 Received: from eggs.gnu.org ([209.51.188.92]:39532) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhVsY-0002wl-TE for 41668@debbugs.gnu.org; Sat, 06 Jun 2020 06:16:50 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58731) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jhVsR-0002vh-UU; Sat, 06 Jun 2020 06:16:36 -0400 Received: from [2a01:e0a:fa:a50:809b:e323:9e90:279b] (port=39358 helo=meru) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jhVsR-0007Rq-Cy; Sat, 06 Jun 2020 06:16:35 -0400 From: Mathieu Othacehe To: Brice Waegeneire Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> Date: Sat, 06 Jun 2020 12:16:33 +0200 In-Reply-To: <3f66774219bb18c47b305b8760f58abe@waegenei.re> (Brice Waegeneire's message of "Tue, 02 Jun 2020 18:52:04 +0000") Message-ID: <87mu5glcse.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, ludo@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello Brice, > For some time now the system test 'gui-installed-desktop-os-encrypted' > doesn't succeed, the last time it passed was 2020-04-11[0]. This is > problematic since it exercise a lot of packages and thus can be used to > test those packages. Ludovic attempted to fixed it[1] but it didn't fixed > the test on my side. The "gui-installed-*" tests are in a bad shape right now. The good news is that the can reveal many issues, the bad news is that I can't keep the distance fixing them. They are affected by: * Shepherd race condition: https://issues.guix.gnu.org/40981. But this should be fixed with the recent 0.8.1 release. * The cow-store issue here: https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00043.html. * A new issue I discovered, whereby 'guix system init' try to build/download stuff, when it shouldn't: --8<---------------cut here---------------start------------->8--- Jun 6 11:50:06 localhost shepherd[1]: Service cow-store has been started. Jun 6 11:50:06 localhost installer[233]: running command ("guix" "system" "init" "--fallback" "--no-grafts" "--no-substitutes" "/mnt/etc/config.scm" "/mnt") Jun 6 11:50:20 localhost installer[233]: 'guix system init --fallback --no-grafts --no-substitutes /mnt/etc/config.scm /mnt' exited with status 1; output follows: The following derivations will be bu) Jun 6 11:50:20 localhost 7-gzip-mesboot-1.2.4.drv /gnu/store/zrqjn3p0x5gkdnj6qb4b9xxl2dld4fvl-gzip-1.2.4.tar.drv /gnu/store/98g8d622pqhzl0z92y4bjcqi9yl0327x-linux-libre-headers-bootstrap-0.drv2 Jun 6 11:50:20 localhost g76sla-bash-mesboot-4.4.drv /gnu/store/8yad2zhh91pnf27xkwh0kyx4ggnsn5li-bash-4.4.tar.gz.drv /gnu/store/5vp07s0gdvrybbhcz43i58y5p03rrdsk-coreutils-mesboot-8.32.drv k Jun 6 11:50:20 localhost h6mfp7hisj118ndcp8d0wy3ca63-gmp-6.0.0a.tar.xz.drv /gnu/store/iy7vn17sqbrgp2142px897ij0njsr5f2-gmp-6.0.0a.tar.xz.drv /gnu/store/rxr1wc67y74bsfasln4jw8viawpa79g4-mpfr-4.0 Jun 6 11:50:20 localhost ilding /gnu/store/3pz8g76l2kkv36w1n7k25zlswgdyj554-Python-3.5.9.tar.xz.drv... builder for `/gnu/store/3pz8g76l2kkv36w1n7k25zlswgdyj554-Python-3.5.9.tar.xz.drv' failed to produg Jun 6 11:50:20 localhost --8<---------------cut here---------------end--------------->8--- Thanks, Mathieu From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 07 11:28:32 2020 Received: (at 41668) by debbugs.gnu.org; 7 Jun 2020 15:28:32 +0000 Received: from localhost ([127.0.0.1]:54640 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhxDs-000760-Cv for submit@debbugs.gnu.org; Sun, 07 Jun 2020 11:28:32 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58398) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhxDp-00075n-G7 for 41668@debbugs.gnu.org; Sun, 07 Jun 2020 11:28:30 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:53437) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jhxDj-0007qx-6U; Sun, 07 Jun 2020 11:28:23 -0400 Received: from [2a01:e0a:fa:a50:d939:1174:2a3f:75c2] (port=55272 helo=meru) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jhxDi-0000Hf-0L; Sun, 07 Jun 2020 11:28:22 -0400 From: Mathieu Othacehe To: Brice Waegeneire Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> Date: Sun, 07 Jun 2020 17:28:19 +0200 In-Reply-To: <87mu5glcse.fsf@gnu.org> (Mathieu Othacehe's message of "Sat, 06 Jun 2020 12:16:33 +0200") Message-ID: <877dwidhf0.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, ludo@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) --=-=-= Content-Type: text/plain Hey, > * A new issue I discovered, whereby 'guix system init' try to > build/download stuff, when it shouldn't: > > Jun 6 11:50:06 localhost shepherd[1]: Service cow-store has been started. > Jun 6 11:50:06 localhost installer[233]: running command ("guix" "system" "init" "--fallback" "--no-grafts" "--no-substitutes" "/mnt/etc/config.scm" "/mnt") > Jun 6 11:50:20 localhost installer[233]: 'guix system init --fallback --no-grafts --no-substitutes /mnt/etc/config.scm /mnt' exited with status 1; output follows: The following derivations will be bu) > > Jun 6 11:50:20 localhost 7-gzip-mesboot-1.2.4.drv /gnu/store/zrqjn3p0x5gkdnj6qb4b9xxl2dld4fvl-gzip-1.2.4.tar.drv /gnu/store/98g8d622pqhzl0z92y4bjcqi9yl0327x-linux-libre-headers-bootstrap-0.drv2 > Jun 6 11:50:20 localhost g76sla-bash-mesboot-4.4.drv /gnu/store/8yad2zhh91pnf27xkwh0kyx4ggnsn5li-bash-4.4.tar.gz.drv /gnu/store/5vp07s0gdvrybbhcz43i58y5p03rrdsk-coreutils-mesboot-8.32.drv k > Jun 6 11:50:20 localhost h6mfp7hisj118ndcp8d0wy3ca63-gmp-6.0.0a.tar.xz.drv /gnu/store/iy7vn17sqbrgp2142px897ij0njsr5f2-gmp-6.0.0a.tar.xz.drv /gnu/store/rxr1wc67y74bsfasln4jw8viawpa79g4-mpfr-4.0 > Jun 6 11:50:20 localhost ilding /gnu/store/3pz8g76l2kkv36w1n7k25zlswgdyj554-Python-3.5.9.tar.xz.drv... builder for `/gnu/store/3pz8g76l2kkv36w1n7k25zlswgdyj554-Python-3.5.9.tar.xz.drv' failed to produg > Jun 6 11:50:20 localhost Ok, I think I found what's going on here. As we are now using multiple glibc to build locales, we need to make sure that they are all available in the installation closure. The attached patch fixes the issue. I will also propose a patch, probably on core-update, patching "invoke/quiet" so that we can have the output of "guix system init" in the syslog. Debugging (gnu tests install) module always takes me a lot of effort, because of all the code layers involved. I don't have much to propose to improve it, maybe when the CI will be more robust, we will be able to detect those test failures as soon as they happen. Now, the cow-store issue! Thanks, Mathieu --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-system-install-Add-default-libcs-to-gc-roots.patch >From 25950adfa4988c31beaeb334367337ee409868c4 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe Date: Sun, 7 Jun 2020 15:39:39 +0200 Subject: [PATCH] system: install: Add default libcs to gc-roots. During the installation, it may be needed to build locales that are not covered by glibc-utf8-locales. Make sure that the libraries required to build locales are available. This is a follow-up of 0eed77127592323d89f56c215a15374a1aaae110. * gnu/system/install.scm (installation-os)[services]: Add %default-locale-libcs to gc-root-service-type. --- gnu/system/install.scm | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/gnu/system/install.scm b/gnu/system/install.scm index fe49ffdb94..d0ff2e7c52 100644 --- a/gnu/system/install.scm +++ b/gnu/system/install.scm @@ -32,6 +32,7 @@ #:use-module ((guix packages) #:select (package-version)) #:use-module ((guix store) #:select (%store-prefix)) #:use-module (gnu installer) + #:use-module (gnu system locale) #:use-module (gnu services dbus) #:use-module (gnu services networking) #:use-module (gnu services shepherd) @@ -439,10 +440,12 @@ Access documentation at any time by pressing Alt-F2.\x1b[0m ;; things needed by 'profile-derivation' to minimize the amount of ;; download. (service gc-root-service-type - (list bare-bones-os - glibc-utf8-locales - texinfo - guile-3.0)) + (append + (list bare-bones-os + glibc-utf8-locales + texinfo + guile-3.0) + %default-locale-libcs)) ;; Machines without Kernel Mode Setting (those with many old and ;; current AMD GPUs, SiS GPUs, ...) need uvesafb to show the GUI -- 2.26.2 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 07 15:54:42 2020 Received: (at 41668) by debbugs.gnu.org; 7 Jun 2020 19:54:42 +0000 Received: from localhost ([127.0.0.1]:54987 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ji1NS-0003Pj-0E for submit@debbugs.gnu.org; Sun, 07 Jun 2020 15:54:42 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52350) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ji1NQ-0003PV-1W for 41668@debbugs.gnu.org; Sun, 07 Jun 2020 15:54:40 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57234) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ji1NK-0005wB-02; Sun, 07 Jun 2020 15:54:34 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40092 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ji1NI-0005mZ-Ik; Sun, 07 Jun 2020 15:54:32 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <877dwidhf0.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 20 Prairial an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 07 Jun 2020 21:54:31 +0200 In-Reply-To: <877dwidhf0.fsf@gnu.org> (Mathieu Othacehe's message of "Sun, 07 Jun 2020 17:28:19 +0200") Message-ID: <87blluslc8.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Hi, Mathieu Othacehe skribis: >> * A new issue I discovered, whereby 'guix system init' try to >> build/download stuff, when it shouldn't: >> >> Jun 6 11:50:06 localhost shepherd[1]: Service cow-store has been starte= d.=20 >> Jun 6 11:50:06 localhost installer[233]: running command ("guix" "syste= m" "init" "--fallback" "--no-grafts" "--no-substitutes" "/mnt/etc/config.sc= m" "/mnt")=20 >> Jun 6 11:50:20 localhost installer[233]: 'guix system init --fallback -= -no-grafts --no-substitutes /mnt/etc/config.scm /mnt' exited with status 1;= output follows: The following derivations will be bu) >> >> Jun 6 11:50:20 localhost 7-gzip-mesboot-1.2.4.drv /gnu/store/zrqjn= 3p0x5gkdnj6qb4b9xxl2dld4fvl-gzip-1.2.4.tar.drv /gnu/store/98g8d622pqhz= l0z92y4bjcqi9yl0327x-linux-libre-headers-bootstrap-0.drv2 >> Jun 6 11:50:20 localhost g76sla-bash-mesboot-4.4.drv /gnu/store/8y= ad2zhh91pnf27xkwh0kyx4ggnsn5li-bash-4.4.tar.gz.drv /gnu/store/5vp07s0g= dvrybbhcz43i58y5p03rrdsk-coreutils-mesboot-8.32.drv k >> Jun 6 11:50:20 localhost h6mfp7hisj118ndcp8d0wy3ca63-gmp-6.0.0a.tar.xz.= drv /gnu/store/iy7vn17sqbrgp2142px897ij0njsr5f2-gmp-6.0.0a.tar.xz.drv = /gnu/store/rxr1wc67y74bsfasln4jw8viawpa79g4-mpfr-4.0 >> Jun 6 11:50:20 localhost ilding /gnu/store/3pz8g76l2kkv36w1n7k25zlswgdy= j554-Python-3.5.9.tar.xz.drv... builder for `/gnu/store/3pz8g76l2kkv36w1n= 7k25zlswgdyj554-Python-3.5.9.tar.xz.drv' failed to produg >> Jun 6 11:50:20 localhost=20=20 > > Ok, I think I found what's going on here. As we are now using multiple > glibc to build locales, we need to make sure that they are all available > in the installation closure. Good catch! > The attached patch fixes the issue. I will also propose a patch, > probably on core-update, patching "invoke/quiet" so that we can have the > output of "guix system init" in the syslog. How did you do it in the example above? Perhaps we can have a local hack specifically in the installer (otherwise we=E2=80=99d find ourselves duplicating syslog-related logic there.) > Debugging (gnu tests install) module always takes me a lot of effort, > because of all the code layers involved. I don't have much to propose to > improve it, maybe when the CI will be more robust, we will be able to > detect those test failures as soon as they happen. Yeah there are two issues: (1) the installer tests are tricky to debug, and (2) we basically don=E2=80=99t notice when something break. I don=E2=80=99t have good ideas for (1), although I think the speedups indu= ced by the new image API might help. As for (2), I think it=E2=80=99s kind of a problem of CI in general, connec= ted with the fact that we=E2=80=99re all somewhat busy building stuff rather th= an waiting for CI notifications to act on. :-) That=E2=80=99s my impression, but I=E2=80=99d really like to hear from peop= le with experience in CI and/or the system tests. > From 25950adfa4988c31beaeb334367337ee409868c4 Mon Sep 17 00:00:00 2001 > From: Mathieu Othacehe > Date: Sun, 7 Jun 2020 15:39:39 +0200 > Subject: [PATCH] system: install: Add default libcs to gc-roots. > > During the installation, it may be needed to build locales that are not > covered by glibc-utf8-locales. Make sure that the libraries required to b= uild > locales are available. > > This is a follow-up of 0eed77127592323d89f56c215a15374a1aaae110. > > * gnu/system/install.scm (installation-os)[services]: Add > %default-locale-libcs to gc-root-service-type. Perfect, thanks! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 08 04:47:59 2020 Received: (at 41668) by debbugs.gnu.org; 8 Jun 2020 08:48:00 +0000 Received: from localhost ([127.0.0.1]:55897 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jiDRn-0001Wg-Jh for submit@debbugs.gnu.org; Mon, 08 Jun 2020 04:47:59 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35820) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jiDRl-0001WR-Vp for 41668@debbugs.gnu.org; Mon, 08 Jun 2020 04:47:58 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44265) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jiDRf-00020c-8o; Mon, 08 Jun 2020 04:47:51 -0400 Received: from lfbn-ann-1-136-86.w86-200.abo.wanadoo.fr ([86.200.104.86]:42334 helo=meru) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jiDRe-0001cv-Fr; Mon, 08 Jun 2020 04:47:51 -0400 From: Mathieu Othacehe To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <877dwidhf0.fsf@gnu.org> <87blluslc8.fsf@gnu.org> Date: Mon, 08 Jun 2020 10:47:42 +0200 In-Reply-To: <87blluslc8.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Sun, 07 Jun 2020 21:54:31 +0200") Message-ID: <87zh9eaqq9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Hey Ludo, > How did you do it in the example above? Perhaps we can have a local > hack specifically in the installer (otherwise we=E2=80=99d find ourselves > duplicating syslog-related logic there.) I used a custom re-implementation of "invoke/quiet" printing every line on the syslog port. Maybe, we could enable such a mechanism only during the tests. > I don=E2=80=99t have good ideas for (1), although I think the speedups in= duced > by the new image API might help. Yes, that's for sure making the debug more comfortable! > As for (2), I think it=E2=80=99s kind of a problem of CI in general, conn= ected > with the fact that we=E2=80=99re all somewhat busy building stuff rather = than > waiting for CI notifications to act on. :-) > > That=E2=80=99s my impression, but I=E2=80=99d really like to hear from pe= ople with > experience in CI and/or the system tests. Totally agree. When submitting patches to Linux for instance, the CI is able to reply to the patch mail, and warn about any failures. An other difference is that patches are often kept in the maintainer 'staging' branches until the CI is over. I don't know if Guix is a big enough project to appoint some maintainers that would periodically merge their branches to 'master' or if it would just be a huge burden. Something we need to discuss in the future maybe. > Perfect, thanks! Anyway, thanks for your fast review :) Mathieu From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 08 09:24:30 2020 Received: (at 41668) by debbugs.gnu.org; 8 Jun 2020 13:24:31 +0000 Received: from localhost ([127.0.0.1]:56198 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jiHlO-00026B-Bm for submit@debbugs.gnu.org; Mon, 08 Jun 2020 09:24:30 -0400 Received: from eggs.gnu.org ([209.51.188.92]:39884) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jiHlM-00025z-OE for 41668@debbugs.gnu.org; Mon, 08 Jun 2020 09:24:29 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48223) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jiHlG-0004OB-RE; Mon, 08 Jun 2020 09:24:22 -0400 Received: from lfbn-ann-1-136-86.w86-200.abo.wanadoo.fr ([86.200.104.86]:45334 helo=meru) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jiHlD-0003sr-IH; Mon, 08 Jun 2020 09:24:20 -0400 From: Mathieu Othacehe To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <877dwidhf0.fsf@gnu.org> <87blluslc8.fsf@gnu.org> <87zh9eaqq9.fsf@gnu.org> Date: Mon, 08 Jun 2020 15:24:15 +0200 In-Reply-To: <87zh9eaqq9.fsf@gnu.org> (Mathieu Othacehe's message of "Mon, 08 Jun 2020 10:47:42 +0200") Message-ID: <878sgxbshs.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain Hey, Turns out using "invoke/quiet" is not really convenient because it stores all the command output, which can be pretty big, in memory. Instead I did reimplement the command in (gnu installer utils) in the attached patch :). Thanks, Mathieu --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-installer-utils-Dump-command-output-to-syslog-when-t.patch >From 7c1a03b03d5e62876a88d87aff36776c8ce4ddf2 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe Date: Mon, 8 Jun 2020 15:14:49 +0200 Subject: [PATCH] installer: utils: Dump command output to syslog when testing. When debugging the installation tests, it can be very handy to be able to read "run-command" output, for instance when executing "guix system init". Introduce a new "invoke-with-log" procedure that is able to log a command standard and error outputs to the syslog. Use it, only when running the installation tests, to dump "run-command" output. * gnu/installer/utils.scm (open-pipe-with-stderr, invoke-with-log): New procedures, (invoke-log-port): new variable, (run-command): move to the end of the file and use invoke-with-log when running the installation tests. --- gnu/installer/utils.scm | 166 +++++++++++++++++++++++++++++----------- 1 file changed, 122 insertions(+), 44 deletions(-) diff --git a/gnu/installer/utils.scm b/gnu/installer/utils.scm index 5f8fe8ca01..b9d88e46d9 100644 --- a/gnu/installer/utils.scm +++ b/gnu/installer/utils.scm @@ -22,8 +22,13 @@ #:use-module (guix build utils) #:use-module (guix i18n) #:use-module (srfi srfi-1) + #:use-module (srfi srfi-11) + #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) + #:use-module (srfi srfi-34) + #:use-module (srfi srfi-35) #:use-module (ice-9 match) + #:use-module (ice-9 popen) #:use-module (ice-9 rdelim) #:use-module (ice-9 regex) #:use-module (ice-9 format) @@ -68,50 +73,6 @@ number. If no percentage is found, return #f" (and result (string->number (match:substring result 1))))) -(define* (run-command command #:key locale) - "Run COMMAND, a list of strings, in the given LOCALE. Return true if -COMMAND exited successfully, #f otherwise." - (define env (environ)) - - (define (pause) - (format #t (G_ "Press Enter to continue.~%")) - (send-to-clients '(pause)) - (environ env) ;restore environment variables - (match (select (cons (current-input-port) (current-clients)) - '() '()) - (((port _ ...) _ _) - (read-line port)))) - - (setenv "PATH" "/run/current-system/profile/bin") - - (when locale - (let ((supported? (false-if-exception - (setlocale LC_ALL locale)))) - ;; If LOCALE is not supported, then set LANGUAGE, which might at - ;; least give us translated messages. - (if supported? - (setenv "LC_ALL" locale) - (setenv "LANGUAGE" - (string-take locale - (or (string-index locale #\_) - (string-length locale))))))) - - (guard (c ((invoke-error? c) - (newline) - (format (current-error-port) - (G_ "Command failed with exit code ~a.~%") - (invoke-error-exit-status c)) - (syslog "command ~s failed with exit code ~a" - command (invoke-error-exit-status c)) - (pause) - #f)) - (syslog "running command ~s~%" command) - (apply invoke command) - (syslog "command ~s succeeded~%" command) - (newline) - (pause) - #t)) - ;;; ;;; Logging. @@ -219,3 +180,120 @@ accepting socket." (current-clients (reverse remainder)) exp) + + +;;; +;;; Run commands. +;;; + +;; XXX: This is taken from (guix build utils) and could be factorized. +(define (open-pipe-with-stderr program . args) + "Run PROGRAM with ARGS in an input pipe, but, unlike 'open-pipe*', redirect +both its standard output and standard error to the pipe. Return two value: +the pipe to read PROGRAM's data from, and the PID of the child process running +PROGRAM." + ;; 'open-pipe*' doesn't attempt to capture stderr in any way, which is why + ;; we need to roll our own. + (match (pipe) + ((input . output) + (match (primitive-fork) + (0 + (dynamic-wind + (const #t) + (lambda () + (close-port input) + (close-port (syslog-port)) + (dup2 (fileno output) 1) + (dup2 (fileno output) 2) + (apply execlp program program args)) + (lambda () + (primitive-exit 127)))) + (pid + (close-port output) + (values input pid)))))) + +(define invoke-log-port + ;; Port used by INVOKE-WITH-LOG for logging. + (make-parameter #f)) + +(define* (invoke-with-log program . args) + "Invoke PROGRAM with ARGS and log PROGRAM's standard output and standard +error to INVOKE-LOG-PORT. If PROGRAM succeeds, print nothing and return the +unspecified value; otherwise, raise a '&message' error condition with the +status code. This procedure is very similar to INVOKE/QUIET with the +noticeable difference that the program output, that can be quite heavy, is not +stored but directly sent to INVOKE-LOG-PORT if defined." + (let-values (((pipe pid) + (apply open-pipe-with-stderr program args))) + (let loop () + (match (read-line pipe) + ((? eof-object?) + (close-port pipe) + (match (waitpid pid) + ((_ . status) + (unless (zero? status) + (let-syntax ((G_ (syntax-rules () ;for xgettext + ((_ str) str)))) + (raise (condition + (&message + (message (format #f (G_ "'~a~{ ~a~}' exited \ +with status ~a.") + program args + (or (status:exit-val status) + status))))))))))) + (line + (and=> (invoke-log-port) (cut format <> "~a~%" line)) + (loop)))))) + +(define* (run-command command #:key locale) + "Run COMMAND, a list of strings, in the given LOCALE. Return true if +COMMAND exited successfully, #f otherwise." + (define env (environ)) + + (define (pause) + (format #t (G_ "Press Enter to continue.~%")) + (send-to-clients '(pause)) + (environ env) ;restore environment variables + (match (select (cons (current-input-port) (current-clients)) + '() '()) + (((port _ ...) _ _) + (read-line port)))) + + (setenv "PATH" "/run/current-system/profile/bin") + + (when locale + (let ((supported? (false-if-exception + (setlocale LC_ALL locale)))) + ;; If LOCALE is not supported, then set LANGUAGE, which might at + ;; least give us translated messages. + (if supported? + (setenv "LC_ALL" locale) + (setenv "LANGUAGE" + (string-take locale + (or (string-index locale #\_) + (string-length locale))))))) + + (guard (c ((invoke-error? c) + (newline) + (format (current-error-port) + (G_ "Command failed with exit code ~a.~%") + (invoke-error-exit-status c)) + (syslog "command ~s failed with exit code ~a" + command (invoke-error-exit-status c)) + (pause) + #f)) + (syslog "running command ~s~%" command) + ;; If there are any connected clients, assume that we are running + ;; installation tests. In that case, dump the standard and error outputs + ;; to syslog. + (let ((testing? (not (null? (current-clients))))) + (if testing? + (parameterize ((invoke-log-port (syslog-port))) + (apply invoke-with-log command)) + (apply invoke command))) + (syslog "command ~s succeeded~%" command) + (newline) + (pause) + #t)) + +;;; utils.scm ends here -- 2.26.2 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 08 10:26:09 2020 Received: (at 41668) by debbugs.gnu.org; 8 Jun 2020 14:26:09 +0000 Received: from localhost ([127.0.0.1]:57596 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jiIj2-0003yt-Tf for submit@debbugs.gnu.org; Mon, 08 Jun 2020 10:26:09 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:48604) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jiIj1-0003yl-Hy for 41668@debbugs.gnu.org; Mon, 08 Jun 2020 10:26:08 -0400 Received: from localhost (80-110-127-207.cgn.dynamic.surfer.at [80.110.127.207]) by dd26836.kasserver.com (Postfix) with ESMTPSA id E334D336023C; Mon, 8 Jun 2020 16:26:05 +0200 (CEST) Date: Mon, 8 Jun 2020 16:26:02 +0200 From: Danny Milosavljevic To: Ludovic =?ISO-8859-1?Q?Court=E8s?= Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted Message-ID: <20200608162602.5bff8512@scratchpost.org> In-Reply-To: <87blluslc8.fsf@gnu.org> References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <877dwidhf0.fsf@gnu.org> <87blluslc8.fsf@gnu.org> X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/BI0ukv.v6ANM8RPK21_XhEW"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, Mathieu Othacehe , Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --Sig_/BI0ukv.v6ANM8RPK21_XhEW Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi, my CI setup for work automatically mails me failures. It would be really nice to have that for guix master eventually, too. Maybe just mail failures to guix-devel as they happen. I don't check https://ci.guix.gnu.org/ so often, and even when I do the job= set names are kinda weird there, and there's too few stuff on one page--and jus= t in general there's no good overview on there. I mean I can search, but a serv= er can just automate that and just send me the results as they happen. It could also automatically mail failures to the last commiters of the sour= ce files that are relevant--but that's probably difficult to implement. Or even use guix data service in order to find out where it failed exactly and whether it does so for all archs--but maybe later. --Sig_/BI0ukv.v6ANM8RPK21_XhEW Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl7eSnoACgkQ5xo1VCww uqWEAAgAlF03g3wh2NZJn4ZX59n1prZ/R2ESFOL6cOGkJKQ6e4GfQwJgThynjR3k z/cH+bDJ9gARsTMBvBDK1SPg0sr1AgykfuCXd568zCShqN4gE8Hwxo8AAsaKkzZk pnU+myQGzcCX9d8hJlibkO9L1IaHe0Q/3XWHJoa8MrRRtVo4CNDmOAb8ITDb9oyw IJ8iIXsoPd0WyYKvP2uCcm1z9jo8o1jAYzONOWHiiEPJDVtOkLfg3nKUCiAZtlcm 1PG/PwlyBmQXaeYjYvQB3EQbtOQ1/WUi4z2YahpnQYF9zJqIsnlQlBUdnh6ufI/Z U0h5n/CdeeqTr/kUkU76ZrtWEYfMYQ== =3Dyx -----END PGP SIGNATURE----- --Sig_/BI0ukv.v6ANM8RPK21_XhEW-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 09 07:18:37 2020 Received: (at 41668) by debbugs.gnu.org; 9 Jun 2020 11:18:37 +0000 Received: from localhost ([127.0.0.1]:59080 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jicH7-0003P1-8q for submit@debbugs.gnu.org; Tue, 09 Jun 2020 07:18:37 -0400 Received: from eggs.gnu.org ([209.51.188.92]:55298) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jicH4-0003Om-9T for 41668@debbugs.gnu.org; Tue, 09 Jun 2020 07:18:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:39429) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jicGx-000350-J6; Tue, 09 Jun 2020 07:18:27 -0400 Received: from [2a01:e0a:fa:a50:d939:1174:2a3f:75c2] (port=51158 helo=meru) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jicGw-0004Np-Px; Tue, 09 Jun 2020 07:18:27 -0400 From: Mathieu Othacehe To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <877dwidhf0.fsf@gnu.org> <87blluslc8.fsf@gnu.org> <87zh9eaqq9.fsf@gnu.org> <878sgxbshs.fsf@gnu.org> Date: Tue, 09 Jun 2020 13:18:25 +0200 In-Reply-To: <878sgxbshs.fsf@gnu.org> (Mathieu Othacehe's message of "Mon, 08 Jun 2020 15:24:15 +0200") Message-ID: <87v9k0o5by.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain Hey, > Instead I did reimplement the command in (gnu installer utils) in the > attached patch :). There were an issue with exception handling, here's a v2. Note that it uses the &invoke-error constructor that should be made public I guess. Thanks, Mathieu --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-installer-utils-Dump-command-output-to-syslog-when-t.patch >From 18754c8c62eabb341e0f710d83ff435ef950ca8e Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe Date: Mon, 8 Jun 2020 15:14:49 +0200 Subject: [PATCH] installer: utils: Dump command output to syslog when testing. When debugging the installation tests, it can be very handy to be able to read "run-command" output, for instance when executing "guix system init". Introduce a new "invoke-with-log" procedure that is able to log a command standard and error outputs to the syslog. Use it, only when running the installation tests, to dump "run-command" output. * gnu/installer/utils.scm (open-pipe-with-stderr, invoke-with-log): New procedures, (invoke-log-port): new variable, (run-command): move to the end of the file and use invoke-with-log when running the installation tests. --- gnu/installer/utils.scm | 164 +++++++++++++++++++++++++++++----------- 1 file changed, 120 insertions(+), 44 deletions(-) diff --git a/gnu/installer/utils.scm b/gnu/installer/utils.scm index 5f8fe8ca01..68b3dd5009 100644 --- a/gnu/installer/utils.scm +++ b/gnu/installer/utils.scm @@ -22,8 +22,13 @@ #:use-module (guix build utils) #:use-module (guix i18n) #:use-module (srfi srfi-1) + #:use-module (srfi srfi-11) + #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) + #:use-module (srfi srfi-34) + #:use-module (srfi srfi-35) #:use-module (ice-9 match) + #:use-module (ice-9 popen) #:use-module (ice-9 rdelim) #:use-module (ice-9 regex) #:use-module (ice-9 format) @@ -68,50 +73,6 @@ number. If no percentage is found, return #f" (and result (string->number (match:substring result 1))))) -(define* (run-command command #:key locale) - "Run COMMAND, a list of strings, in the given LOCALE. Return true if -COMMAND exited successfully, #f otherwise." - (define env (environ)) - - (define (pause) - (format #t (G_ "Press Enter to continue.~%")) - (send-to-clients '(pause)) - (environ env) ;restore environment variables - (match (select (cons (current-input-port) (current-clients)) - '() '()) - (((port _ ...) _ _) - (read-line port)))) - - (setenv "PATH" "/run/current-system/profile/bin") - - (when locale - (let ((supported? (false-if-exception - (setlocale LC_ALL locale)))) - ;; If LOCALE is not supported, then set LANGUAGE, which might at - ;; least give us translated messages. - (if supported? - (setenv "LC_ALL" locale) - (setenv "LANGUAGE" - (string-take locale - (or (string-index locale #\_) - (string-length locale))))))) - - (guard (c ((invoke-error? c) - (newline) - (format (current-error-port) - (G_ "Command failed with exit code ~a.~%") - (invoke-error-exit-status c)) - (syslog "command ~s failed with exit code ~a" - command (invoke-error-exit-status c)) - (pause) - #f)) - (syslog "running command ~s~%" command) - (apply invoke command) - (syslog "command ~s succeeded~%" command) - (newline) - (pause) - #t)) - ;;; ;;; Logging. @@ -219,3 +180,118 @@ accepting socket." (current-clients (reverse remainder)) exp) + + +;;; +;;; Run commands. +;;; + +;; XXX: This is taken from (guix build utils) and could be factorized. +(define (open-pipe-with-stderr program . args) + "Run PROGRAM with ARGS in an input pipe, but, unlike 'open-pipe*', redirect +both its standard output and standard error to the pipe. Return two value: +the pipe to read PROGRAM's data from, and the PID of the child process running +PROGRAM." + ;; 'open-pipe*' doesn't attempt to capture stderr in any way, which is why + ;; we need to roll our own. + (match (pipe) + ((input . output) + (match (primitive-fork) + (0 + (dynamic-wind + (const #t) + (lambda () + (close-port input) + (close-port (syslog-port)) + (dup2 (fileno output) 1) + (dup2 (fileno output) 2) + (apply execlp program program args)) + (lambda () + (primitive-exit 127)))) + (pid + (close-port output) + (values input pid)))))) + +(define invoke-log-port + ;; Port used by INVOKE-WITH-LOG for logging. + (make-parameter #f)) + +(define* (invoke-with-log program . args) + "Invoke PROGRAM with ARGS and log PROGRAM's standard output and standard +error to INVOKE-LOG-PORT. If PROGRAM succeeds, print nothing and return the +unspecified value; otherwise, raise a '&message' error condition with the +status code. This procedure is very similar to INVOKE/QUIET with the +noticeable difference that the program output, that can be quite heavy, is not +stored but directly sent to INVOKE-LOG-PORT if defined." + (let-values (((pipe pid) + (apply open-pipe-with-stderr program args))) + (let loop () + (match (read-line pipe) + ((? eof-object?) + (close-port pipe) + (match (waitpid pid) + ((_ . status) + (unless (zero? status) + (raise + (condition ((@@ (guix build utils) &invoke-error) + (program program) + (arguments args) + (exit-status (status:exit-val status)) + (term-signal (status:term-sig status)) + (stop-signal (status:stop-sig status))))))))) + (line + (and=> (invoke-log-port) (cut format <> "~a~%" line)) + (loop)))))) + +(define* (run-command command #:key locale) + "Run COMMAND, a list of strings, in the given LOCALE. Return true if +COMMAND exited successfully, #f otherwise." + (define env (environ)) + + (define (pause) + (format #t (G_ "Press Enter to continue.~%")) + (send-to-clients '(pause)) + (environ env) ;restore environment variables + (match (select (cons (current-input-port) (current-clients)) + '() '()) + (((port _ ...) _ _) + (read-line port)))) + + (setenv "PATH" "/run/current-system/profile/bin") + + (when locale + (let ((supported? (false-if-exception + (setlocale LC_ALL locale)))) + ;; If LOCALE is not supported, then set LANGUAGE, which might at + ;; least give us translated messages. + (if supported? + (setenv "LC_ALL" locale) + (setenv "LANGUAGE" + (string-take locale + (or (string-index locale #\_) + (string-length locale))))))) + + (guard (c ((invoke-error? c) + (newline) + (format (current-error-port) + (G_ "Command failed with exit code ~a.~%") + (invoke-error-exit-status c)) + (syslog "command ~s failed with exit code ~a" + command (invoke-error-exit-status c)) + (pause) + #f)) + (syslog "running command ~s~%" command) + ;; If there are any connected clients, assume that we are running + ;; installation tests. In that case, dump the standard and error outputs + ;; to syslog. + (let ((testing? (not (null? (current-clients))))) + (if testing? + (parameterize ((invoke-log-port (syslog-port))) + (apply invoke-with-log command)) + (apply invoke command))) + (syslog "command ~s succeeded~%" command) + (newline) + (pause) + #t)) + +;;; utils.scm ends here -- 2.26.2 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 09 10:22:14 2020 Received: (at 41668) by debbugs.gnu.org; 9 Jun 2020 14:22:14 +0000 Received: from localhost ([127.0.0.1]:60514 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jif8n-0002A9-Ol for submit@debbugs.gnu.org; Tue, 09 Jun 2020 10:22:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47568) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jif8l-00029u-NF for 41668@debbugs.gnu.org; Tue, 09 Jun 2020 10:22:11 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:42272) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jif8f-0000qL-VX; Tue, 09 Jun 2020 10:22:05 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=60626 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jif8d-0005xz-Of; Tue, 09 Jun 2020 10:22:04 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <877dwidhf0.fsf@gnu.org> <87blluslc8.fsf@gnu.org> <87zh9eaqq9.fsf@gnu.org> <878sgxbshs.fsf@gnu.org> <87v9k0o5by.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 22 Prairial an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 09 Jun 2020 16:22:02 +0200 In-Reply-To: <87v9k0o5by.fsf@gnu.org> (Mathieu Othacehe's message of "Tue, 09 Jun 2020 13:18:25 +0200") Message-ID: <871rmojp4l.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Mathieu Othacehe skribis: > From 18754c8c62eabb341e0f710d83ff435ef950ca8e Mon Sep 17 00:00:00 2001 > From: Mathieu Othacehe > Date: Mon, 8 Jun 2020 15:14:49 +0200 > Subject: [PATCH] installer: utils: Dump command output to syslog when tes= ting. > > When debugging the installation tests, it can be very handy to be able to= read > "run-command" output, for instance when executing "guix system init". > > Introduce a new "invoke-with-log" procedure that is able to log a command > standard and error outputs to the syslog. Use it, only when running the > installation tests, to dump "run-command" output. > > * gnu/installer/utils.scm (open-pipe-with-stderr, invoke-with-log): New > procedures, > (invoke-log-port): new variable, > (run-command): move to the end of the file and use invoke-with-log when > running the installation tests. > --- > gnu/installer/utils.scm | 164 +++++++++++++++++++++++++++++----------- > 1 file changed, 120 insertions(+), 44 deletions(-) It=E2=80=99s nice, but also a bit complicated just to print stuff on the screen. :-) I found a stash with my debugging hack: --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/gnu/installer/final.scm b/gnu/installer/final.scm index 869be8814b..c084123064 100644 --- a/gnu/installer/final.scm +++ b/gnu/installer/final.scm @@ -137,7 +137,13 @@ or #f. Return #t on success and #f on failure." (lambda () (start-service 'cow-store (list (%installer-target-dir)))) (lambda () - (run-command install-command #:locale locale)) + (with-output-to-file "/dev/console" + (lambda () + (with-error-to-file "/dev/console" + (lambda () + (setvbuf (current-output-port) 'none) + (setvbuf (current-error-port) 'none) + (run-command install-command #:locale locale)))))) (lambda () (stop-service 'cow-store) ;; Remove the store overlay created at cow-store service start. --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable That=E2=80=99s enough to send the =E2=80=98guix system init=E2=80=99 output= to the console, since we use =E2=80=9Cconsole=3DttyS0=E2=80=9D. It=E2=80=99s a gross hack of course, but maybe we can do something along th= ese lines instead of setting up a pipe? Thanks, Ludo=E2=80=99. --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 09 10:24:10 2020 Received: (at 41668) by debbugs.gnu.org; 9 Jun 2020 14:24:10 +0000 Received: from localhost ([127.0.0.1]:60518 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jifAg-0002DI-5b for submit@debbugs.gnu.org; Tue, 09 Jun 2020 10:24:10 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47824) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jifAf-0002D5-DD for 41668@debbugs.gnu.org; Tue, 09 Jun 2020 10:24:09 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:42298) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jifAa-0001Fl-1b; Tue, 09 Jun 2020 10:24:04 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=60628 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jifAZ-000682-6a; Tue, 09 Jun 2020 10:24:03 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Danny Milosavljevic Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <877dwidhf0.fsf@gnu.org> <87blluslc8.fsf@gnu.org> <20200608162602.5bff8512@scratchpost.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 22 Prairial an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 09 Jun 2020 16:24:01 +0200 In-Reply-To: <20200608162602.5bff8512@scratchpost.org> (Danny Milosavljevic's message of "Mon, 8 Jun 2020 16:26:02 +0200") Message-ID: <87sgf4iagu.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, Mathieu Othacehe , Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Danny Milosavljevic skribis: > my CI setup for work automatically mails me failures. > > It would be really nice to have that for guix master eventually, too. > Maybe just mail failures to guix-devel as they happen. > > I don't check https://ci.guix.gnu.org/ so often, and even when I do the j= obset > names are kinda weird there, and there's too few stuff on one page--and j= ust in > general there's no good overview on there. I mean I can search, but a se= rver > can just automate that and just send me the results as they happen. > > It could also automatically mail failures to the last commiters of the so= urce > files that are relevant--but that's probably difficult to implement. Yeah, Hydra could do that on status change (success -> failure and vice ver= sa). IRC notifications might also be nice. Or Mastodon. There=E2=80=99s always a risk of flood though, at which point people stop p= aying attention to those notifications. Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 09 12:01:22 2020 Received: (at 41668) by debbugs.gnu.org; 9 Jun 2020 16:01:22 +0000 Received: from localhost ([127.0.0.1]:60649 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jiggg-0004i9-Ua for submit@debbugs.gnu.org; Tue, 09 Jun 2020 12:01:22 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34280) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jigge-0004hu-Gx for 41668@debbugs.gnu.org; Tue, 09 Jun 2020 12:01:17 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44340) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jiggY-0003CO-N7; Tue, 09 Jun 2020 12:01:10 -0400 Received: from [2a01:e0a:fa:a50:d939:1174:2a3f:75c2] (port=53496 helo=meru) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jiggV-0002yI-Ga; Tue, 09 Jun 2020 12:01:08 -0400 From: Mathieu Othacehe To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <877dwidhf0.fsf@gnu.org> <87blluslc8.fsf@gnu.org> <87zh9eaqq9.fsf@gnu.org> <878sgxbshs.fsf@gnu.org> <87v9k0o5by.fsf@gnu.org> <871rmojp4l.fsf@gnu.org> Date: Tue, 09 Jun 2020 18:01:04 +0200 In-Reply-To: <871rmojp4l.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Tue, 09 Jun 2020 16:22:02 +0200") Message-ID: <877dwgns8v.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hey Ludo, > It=E2=80=99s nice, but also a bit complicated just to print stuff on the > screen. :-) You're right, I went too far :p > That=E2=80=99s enough to send the =E2=80=98guix system init=E2=80=99 outp= ut to the console, > since we use =E2=80=9Cconsole=3DttyS0=E2=80=9D. > > It=E2=80=99s a gross hack of course, but maybe we can do something along = these > lines instead of setting up a pipe? Sure, I just applied a variant of your patch. Thanks, Mathieu From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 14 12:16:53 2020 Received: (at 41668) by debbugs.gnu.org; 14 Jun 2020 16:16:53 +0000 Received: from localhost ([127.0.0.1]:44353 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jkVJU-0002C6-UU for submit@debbugs.gnu.org; Sun, 14 Jun 2020 12:16:53 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58854) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jkVJU-0002Bu-1U for 41668@debbugs.gnu.org; Sun, 14 Jun 2020 12:16:52 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48160) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jkVJN-00043J-4H; Sun, 14 Jun 2020 12:16:45 -0400 Received: from [2a01:e0a:fa:a50:915b:bab1:f4c6:afb3] (port=57978 helo=meru) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jkVJM-0005dd-ID; Sun, 14 Jun 2020 12:16:44 -0400 From: Mathieu Othacehe To: Brice Waegeneire Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> Date: Sun, 14 Jun 2020 18:16:43 +0200 In-Reply-To: <87mu5glcse.fsf@gnu.org> (Mathieu Othacehe's message of "Sat, 06 Jun 2020 12:16:33 +0200") Message-ID: <87wo49ipw4.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, ludo@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, Some status here. > * The cow-store issue here: > https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00043.html. This should be fixed with 876a8d987085b8c64f32c8a320e4219575af285c. > > * A new issue I discovered, whereby 'guix system init' try to > build/download stuff, when it shouldn't: This is fixed by ea5d9bf9916d532005842f220f7da6e522151f6a. Regarding the initial OOM issue, this is no longer a thing because we are building the image on the host (and not in a VM with limited memory). I discovered another issue, related to HFS+ limits when producing the ISO image, this is fixed by: fd1e885236fcec0aec1e3293153524f02b010560. There is now a final issue, where the command restarting "guix-daemon" in "umount-cow-store" hangs forever. I can only reproduce it in "gui-installed-desktop-os-encrypted" test. I wonder if it's a Shepherd bug or anything else. I'll keep digging. Thanks, Mathieu From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 14 16:29:17 2020 Received: (at 41668) by debbugs.gnu.org; 14 Jun 2020 20:29:17 +0000 Received: from localhost ([127.0.0.1]:44567 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jkZFg-0008RK-Oj for submit@debbugs.gnu.org; Sun, 14 Jun 2020 16:29:17 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34498) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jkZFe-0008R4-C5 for 41668@debbugs.gnu.org; Sun, 14 Jun 2020 16:29:10 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52342) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jkZFX-0006UM-D9; Sun, 14 Jun 2020 16:29:03 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=43732 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jkZFV-0001U8-1Q; Sun, 14 Jun 2020 16:29:02 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <87wo49ipw4.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 27 Prairial an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 14 Jun 2020 22:28:59 +0200 In-Reply-To: <87wo49ipw4.fsf@gnu.org> (Mathieu Othacehe's message of "Sun, 14 Jun 2020 18:16:43 +0200") Message-ID: <87h7vdtmr8.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Mathieu, Mathieu Othacehe skribis: >> * The cow-store issue here: >> https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00043.html. > > This should be fixed with 876a8d987085b8c64f32c8a320e4219575af285c. Oh, fun! >> * A new issue I discovered, whereby 'guix system init' try to >> build/download stuff, when it shouldn't: > > This is fixed by ea5d9bf9916d532005842f220f7da6e522151f6a. > > Regarding the initial OOM issue, this is no longer a thing because we > are building the image on the host (and not in a VM with limited > memory). > > I discovered another issue, related to HFS+ limits when producing the > ISO image, this is fixed by: fd1e885236fcec0aec1e3293153524f02b010560. Woow. > There is now a final issue, where the command restarting "guix-daemon" > in "umount-cow-store" hangs forever. I can only reproduce it in > "gui-installed-desktop-os-encrypted" test. I wonder if it's a Shepherd > bug or anything else. I'll keep digging. If I can be of any help, we can chat on IRC. Great work already! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 16 11:41:08 2020 Received: (at 41668) by debbugs.gnu.org; 16 Jun 2020 15:41:08 +0000 Received: from localhost ([127.0.0.1]:49438 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jlDhz-00043W-UH for submit@debbugs.gnu.org; Tue, 16 Jun 2020 11:41:08 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34922) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jlDhy-00042u-1O for 41668@debbugs.gnu.org; Tue, 16 Jun 2020 11:41:06 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58647) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jlDhr-0008FH-7D; Tue, 16 Jun 2020 11:40:59 -0400 Received: from lfbn-ann-1-136-86.w86-200.abo.wanadoo.fr ([86.200.104.86]:37246 helo=meru) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jlDho-0007C3-S8; Tue, 16 Jun 2020 11:40:58 -0400 From: Mathieu Othacehe To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <87wo49ipw4.fsf@gnu.org> <87h7vdtmr8.fsf@gnu.org> Date: Tue, 16 Jun 2020 17:40:52 +0200 In-Reply-To: <87h7vdtmr8.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Sun, 14 Jun 2020 22:28:59 +0200") Message-ID: <871rmfvx17.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hey, >> There is now a final issue, where the command restarting "guix-daemon" >> in "umount-cow-store" hangs forever. I can only reproduce it in >> "gui-installed-desktop-os-encrypted" test. I wonder if it's a Shepherd >> bug or anything else. I'll keep digging. > > If I can be of any help, we can chat on IRC. > > Great work already! Thanks :) I'm a bit clueless about this one. Here's the output: --8<---------------cut here---------------start------------->8--- The following derivations will be built: /gnu/store/pz0h29lwddwv8iq755xbkp4jbrj645f6-install-bootloader.scm.drv /gnu/store/6fbpimwxl85b62rcxd9plk1mf1jb7hvy-module-import-compiled.drv /gnu/store/978myg8v0as05r07dzq8m4nlfj5cmkn6-module-import.drv building /gnu/store/978myg8v0as05r07dzq8m4nlfj5cmkn6-module-import.drv... building /gnu/store/6fbpimwxl85b62rcxd9plk1mf1jb7hvy-module-import-compiled.drv... building /gnu/store/pz0h29lwddwv8iq755xbkp4jbrj645f6-install-bootloader.scm.drv... guix system: bootloader successfully installed on '/dev/vda' Press Enter to contiJun 16 17:30:08 localhost installer[231]: command ("guix" "system" "init" "--fallback" "--no-grafts" "--no-substitutes" "/mnt/etc/config.scm" "/mnt") succeeded nue. conversation expecting p attern ((quote installation-complete)) Jun 16 17:30:10 localhost shepherd[1]: Service cow-store has been stopped. Jun 16 17:30:10 localhost installer[231]: Unmounting cow-store. Jun 16 17:30:10 localhost shepherd[1]: Service guix-daemon has been stopped. --8<---------------cut here---------------end--------------->8--- and it hangs forever. Looks like the guix-daemon restart could be the issue as it's the last thing that is logged. But hard to say for sure. I cannot reproduce it while installing manually a system with all DE and services enabled. Having an ssh access to this VM would be nice, but I think you already told me it was not possible. If you have any other idea, it would be great :) Thanks, Mathieu From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 17 08:33:41 2020 Received: (at 41668) by debbugs.gnu.org; 17 Jun 2020 12:33:41 +0000 Received: from localhost ([127.0.0.1]:50695 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jlXG8-0002Y4-Vs for submit@debbugs.gnu.org; Wed, 17 Jun 2020 08:33:41 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47318) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jlXG5-0002Xp-K2 for 41668@debbugs.gnu.org; Wed, 17 Jun 2020 08:33:39 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44337) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jlXFz-0003Cc-Ey; Wed, 17 Jun 2020 08:33:31 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41098 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jlXFu-0005Zx-VT; Wed, 17 Jun 2020 08:33:27 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <87wo49ipw4.fsf@gnu.org> <87h7vdtmr8.fsf@gnu.org> <871rmfvx17.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 30 Prairial an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 17 Jun 2020 14:33:24 +0200 In-Reply-To: <871rmfvx17.fsf@gnu.org> (Mathieu Othacehe's message of "Tue, 16 Jun 2020 17:40:52 +0200") Message-ID: <87h7v9rhwr.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668 Cc: 41668@debbugs.gnu.org, Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Mathieu Othacehe skribis: >>> There is now a final issue, where the command restarting "guix-daemon" >>> in "umount-cow-store" hangs forever. I can only reproduce it in >>> "gui-installed-desktop-os-encrypted" test. I wonder if it's a Shepherd >>> bug or anything else. I'll keep digging. >> >> If I can be of any help, we can chat on IRC. >> >> Great work already! > > Thanks :) I'm a bit clueless about this one. Here's the output: > > The following derivations will be built: > /gnu/store/pz0h29lwddwv8iq755xbkp4jbrj645f6-install-bootloader.scm.drv > /gnu/store/6fbpimwxl85b62rcxd9plk1mf1jb7hvy-module-import-compiled.drv > /gnu/store/978myg8v0as05r07dzq8m4nlfj5cmkn6-module-import.drv > building /gnu/store/978myg8v0as05r07dzq8m4nlfj5cmkn6-module-import.drv... > building /gnu/store/6fbpimwxl85b62rcxd9plk1mf1jb7hvy-module-import-compil= ed.drv... > building /gnu/store/pz0h29lwddwv8iq755xbkp4jbrj645f6-install-bootloader.s= cm.drv... > guix system: bootloader successfully installed on '/dev/vda' > > Press Enter to contiJun 16 17:30:08 localhost installer[231]: command ("g= uix" "system" "init" "--fallback" "--no-grafts" "--no-substitutes" "/mnt/et= c/config.scm" "/mnt") succeeded nue. > conversation expecting p > attern ((quote installation-complete)) > Jun 16 17:30:10 localhost shepherd[1]: Service cow-store has been stopped. > Jun 16 17:30:10 localhost installer[231]: Unmounting cow-store. > Jun 16 17:30:10 localhost shepherd[1]: Service guix-daemon has been stopp= ed. > > and it hangs forever. Looks like the guix-daemon restart could be the > issue as it's the last thing that is logged. But hard to say for > sure. I cannot reproduce it while installing manually a system with all > DE and services enabled. Is it deterministic? Same story with =E2=80=9Cgui-installed-os=E2=80=9D? I guess we can add more =E2=80=98syslog=E2=80=99 statements in the installe= r around the place where we restart services. Or we could run the installer entirely under strace. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 11 03:32:55 2020 Received: (at 41668-done) by debbugs.gnu.org; 11 Sep 2020 07:32:55 +0000 Received: from localhost ([127.0.0.1]:42208 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGdYE-0000gp-Vz for submit@debbugs.gnu.org; Fri, 11 Sep 2020 03:32:55 -0400 Received: from eggs.gnu.org ([209.51.188.92]:44518) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGdYD-0000gN-Ve for 41668-done@debbugs.gnu.org; Fri, 11 Sep 2020 03:32:54 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48511) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kGdY7-0002Io-LH; Fri, 11 Sep 2020 03:32:47 -0400 Received: from [2a01:e0a:19b:d9a0:9d9d:97cc:d92a:8ac0] (port=36958 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kGdY6-0005SO-Uu; Fri, 11 Sep 2020 03:32:47 -0400 From: Mathieu Othacehe To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <87wo49ipw4.fsf@gnu.org> <87h7vdtmr8.fsf@gnu.org> <871rmfvx17.fsf@gnu.org> <87h7v9rhwr.fsf@gnu.org> Date: Fri, 11 Sep 2020 09:32:45 +0200 In-Reply-To: <87h7v9rhwr.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Wed, 17 Jun 2020 14:33:24 +0200") Message-ID: <87k0x0zs2q.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668-done Cc: 41668-done@debbugs.gnu.org, Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, > Is it deterministic? Same story with =E2=80=9Cgui-installed-os=E2=80=9D? > > I guess we can add more =E2=80=98syslog=E2=80=99 statements in the instal= ler around the > place where we restart services. Or we could run the installer entirely > under strace. Now that the installation is done in a container, this one doesn't show up anymore. I also added further fixes with 3ad68ec251a008d6f5875f9235e75f5ce6399f15. All the installation tests are now operational. Thanks, Mathieu From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 11 10:43:00 2020 Received: (at 41668-done) by debbugs.gnu.org; 11 Sep 2020 14:43:00 +0000 Received: from localhost ([127.0.0.1]:45057 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGkGS-0004AC-70 for submit@debbugs.gnu.org; Fri, 11 Sep 2020 10:43:00 -0400 Received: from eggs.gnu.org ([209.51.188.92]:54782) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGkGP-00049y-1P for 41668-done@debbugs.gnu.org; Fri, 11 Sep 2020 10:42:59 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40224) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kGkGJ-0006Rq-K6; Fri, 11 Sep 2020 10:42:51 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=39266 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kGkGG-0003Ue-Cf; Fri, 11 Sep 2020 10:42:49 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted References: <3f66774219bb18c47b305b8760f58abe@waegenei.re> <87mu5glcse.fsf@gnu.org> <87wo49ipw4.fsf@gnu.org> <87h7vdtmr8.fsf@gnu.org> <871rmfvx17.fsf@gnu.org> <87h7v9rhwr.fsf@gnu.org> <87k0x0zs2q.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 26 Fructidor an 228 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 11 Sep 2020 16:42:41 +0200 In-Reply-To: <87k0x0zs2q.fsf@gnu.org> (Mathieu Othacehe's message of "Fri, 11 Sep 2020 09:32:45 +0200") Message-ID: <87y2lgtlwe.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41668-done Cc: 41668-done@debbugs.gnu.org, Brice Waegeneire X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Mathieu Othacehe skribis: >> Is it deterministic? Same story with =E2=80=9Cgui-installed-os=E2=80=9D? >> >> I guess we can add more =E2=80=98syslog=E2=80=99 statements in the insta= ller around the >> place where we restart services. Or we could run the installer entirely >> under strace. > > Now that the installation is done in a container, this one doesn't show > up anymore. I also added further fixes with > 3ad68ec251a008d6f5875f9235e75f5ce6399f15. > > All the installation tests are now operational. Yay, well done! Ludo=E2=80=99. From unknown Fri Jun 20 07:29:04 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 10 Oct 2020 11:24:07 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator