From unknown Fri Jun 20 07:10:31 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#34982 <34982@debbugs.gnu.org> To: bug#34982 <34982@debbugs.gnu.org> Subject: Status: [PATCH] guile-build-system: Support building in parallel. Reply-To: bug#34982 <34982@debbugs.gnu.org> Date: Fri, 20 Jun 2025 14:10:31 +0000 retitle 34982 [PATCH] guile-build-system: Support building in parallel. reassign 34982 guix-patches submitter 34982 Christopher Baines severity 34982 normal tag 34982 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 24 17:24:01 2019 Received: (at submit) by debbugs.gnu.org; 24 Mar 2019 21:24:01 +0000 Received: from localhost ([127.0.0.1]:57205 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1h8Ab2-0004NJ-Op for submit@debbugs.gnu.org; Sun, 24 Mar 2019 17:24:00 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34006) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1h8Ab1-0004N5-0h for submit@debbugs.gnu.org; Sun, 24 Mar 2019 17:23:59 -0400 Received: from lists.gnu.org ([209.51.188.17]:42918) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1h8Aav-0007SE-Sd for submit@debbugs.gnu.org; Sun, 24 Mar 2019 17:23:53 -0400 Received: from eggs.gnu.org ([209.51.188.92]:48386) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h8Aau-0004et-LJ for guix-patches@gnu.org; Sun, 24 Mar 2019 17:23:53 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_40,UNPARSEABLE_RELAY autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h8Aar-0007Pv-0O for guix-patches@gnu.org; Sun, 24 Mar 2019 17:23:50 -0400 Received: from li622-129.members.linode.com ([212.71.249.129]:48220 helo=mira.cbaines.net) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h8Aap-0007Ny-6p for guix-patches@gnu.org; Sun, 24 Mar 2019 17:23:48 -0400 Received: from localhost (cpc102582-walt20-2-0-cust14.13-2.cable.virginm.net [86.27.34.15]) by mira.cbaines.net (Postfix) with ESMTPSA id 5DB1D16DCA for ; Sun, 24 Mar 2019 21:23:45 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 1817ac54 for ; Sun, 24 Mar 2019 21:23:45 +0000 (UTC) From: Christopher Baines To: guix-patches@gnu.org Subject: [PATCH] guile-build-system: Support building in parallel. Date: Sun, 24 Mar 2019 21:23:45 +0000 Message-Id: <20190324212345.4697-1-mail@cbaines.net> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 212.71.249.129 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) * guix/build/guile-build-system.scm (build): Use n-par-for-each, instead = of for-each, to use multiple cores if available. --- guix/build/guile-build-system.scm | 43 +++++++++++++++++++------------ 1 file changed, 26 insertions(+), 17 deletions(-) diff --git a/guix/build/guile-build-system.scm b/guix/build/guile-build-s= ystem.scm index 0bed049436..a5741081bf 100644 --- a/guix/build/guile-build-system.scm +++ b/guix/build/guile-build-system.scm @@ -23,6 +23,7 @@ #:use-module (ice-9 match) #:use-module (ice-9 popen) #:use-module (ice-9 rdelim) + #:use-module (ice-9 threads) #:use-module (guix build utils) #:export (target-guile-effective-version %standard-phases @@ -101,24 +102,32 @@ Return #false if it cannot be determined." (match (getenv "GUILE_LOAD_COMPILED_PATH") (#f "") (path (string-append ":" path))))) - (for-each (lambda (file) - (let* ((go (string-append go-dir - (file-sans-extension file) - ".go"))) - ;; Install source module. - (install-file (string-append source-directory "/" file= ) - (string-append module-dir - "/" (dirname file))) + (n-par-for-each + (parallel-job-count) + (lambda (file) + (catch #t + (lambda () + (let* ((go (string-append go-dir + (file-sans-extension file) + ".go"))) + ;; Install source module. + (install-file (string-append source-directory "/" file) + (string-append module-dir + "/" (dirname file))) =20 - ;; Install and compile module. - (apply invoke guild "compile" "-L" source-directory - "-o" go - (string-append source-directory "/" file) - flags))) - - ;; Arrange to strip SOURCE-DIRECTORY from file names. - (with-directory-excursion source-directory - (find-files "." scheme-file-regexp))) + ;; Install and compile module. + (apply invoke guild "compile" "-L" source-directory + "-o" go + (string-append source-directory "/" file) + flags))) + (lambda (key . args) + ;; Since ports are not thread-safe as of Guile 2.0, reopen st= derr. + (let ((port (fdopen 2 "w0"))) + (print-exception port #f key args) + (primitive-exit 1))))) + ;; Arrange to strip SOURCE-DIRECTORY from file names. + (with-directory-excursion source-directory + (find-files "." scheme-file-regexp))) #t)) =20 (define* (install-documentation #:key outputs --=20 2.20.1 From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 30 06:56:49 2019 Received: (at 34982) by debbugs.gnu.org; 30 Mar 2019 10:56:49 +0000 Received: from localhost ([127.0.0.1]:36290 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hABfM-00043I-SL for submit@debbugs.gnu.org; Sat, 30 Mar 2019 06:56:49 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38384) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hABfK-000436-Sa for 34982@debbugs.gnu.org; Sat, 30 Mar 2019 06:56:47 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37268) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hABfB-0004IE-0N; Sat, 30 Mar 2019 06:56:39 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=57970 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hABf9-000538-Fz; Sat, 30 Mar 2019 06:56:36 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: [bug#34982] [PATCH] guile-build-system: Support building in parallel. References: <20190324212345.4697-1-mail@cbaines.net> Date: Sat, 30 Mar 2019 11:56:33 +0100 In-Reply-To: <20190324212345.4697-1-mail@cbaines.net> (Christopher Baines's message of "Sun, 24 Mar 2019 21:23:45 +0000") Message-ID: <87lg0w4om6.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 34982 Cc: 34982@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, Christopher Baines skribis: > * guix/build/guile-build-system.scm (build): Use n-par-for-each, instead = of > for-each, to use multiple cores if available. [...] > + (n-par-for-each > + (parallel-job-count) > + (lambda (file) > + (catch #t > + (lambda () > + (let* ((go (string-append go-dir > + (file-sans-extension file) > + ".go"))) > + ;; Install source module. > + (install-file (string-append source-directory "/" file) > + (string-append module-dir > + "/" (dirname file))) >=20=20 > - ;; Install and compile module. > - (apply invoke guild "compile" "-L" source-directory It probably doesn=E2=80=99t matter that much, but it feels wrong to create threads that do nothing but call =E2=80=98waitpid=E2=80=99, essentially. Commit f07041f7d25badb7d74b8fad6ee446a12af04f63 removed a =E2=80=98p-for-ea= ch=E2=80=99 procedure that could be useful here since it directly creates N processes and then does (waitpid WAITPID_ANY). Would it make sense to paste it here and use it in lieu of =E2=80=98n-par-for-each=E2=80=99? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 05 19:50:59 2019 Received: (at 34982) by debbugs.gnu.org; 5 Apr 2019 23:50:59 +0000 Received: from localhost ([127.0.0.1]:46370 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCYbq-0007ds-H2 for submit@debbugs.gnu.org; Fri, 05 Apr 2019 19:50:59 -0400 Received: from li622-129.members.linode.com ([212.71.249.129]:56016 helo=mira.cbaines.net) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hCYbn-0007df-9x for 34982@debbugs.gnu.org; Fri, 05 Apr 2019 19:50:56 -0400 Received: from localhost (unknown [194.168.26.181]) by mira.cbaines.net (Postfix) with ESMTPSA id A8B9316CEC; Sat, 6 Apr 2019 00:50:53 +0100 (BST) Received: from phact (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 5d26110b; Fri, 5 Apr 2019 23:50:53 +0000 (UTC) References: <20190324212345.4697-1-mail@cbaines.net> <87lg0w4om6.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#34982] [PATCH] guile-build-system: Support building in parallel. In-reply-to: <87lg0w4om6.fsf@gnu.org> Date: Sat, 06 Apr 2019 00:50:48 +0100 Message-ID: <87o95km2p3.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 34982 Cc: 34982@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Hi, > > Christopher Baines skribis: > >> * guix/build/guile-build-system.scm (build): Use n-par-for-each, instead= of >> for-each, to use multiple cores if available. > > [...] > >> + (n-par-for-each >> + (parallel-job-count) >> + (lambda (file) >> + (catch #t >> + (lambda () >> + (let* ((go (string-append go-dir >> + (file-sans-extension file) >> + ".go"))) >> + ;; Install source module. >> + (install-file (string-append source-directory "/" file) >> + (string-append module-dir >> + "/" (dirname file))) >> >> - ;; Install and compile module. >> - (apply invoke guild "compile" "-L" source-directory > > It probably doesn=E2=80=99t matter that much, but it feels wrong to create > threads that do nothing but call =E2=80=98waitpid=E2=80=99, essentially. > > Commit f07041f7d25badb7d74b8fad6ee446a12af04f63 removed a =E2=80=98p-for-= each=E2=80=99 > procedure that could be useful here since it directly creates N > processes and then does (waitpid WAITPID_ANY). Would it make sense to > paste it here and use it in lieu of =E2=80=98n-par-for-each=E2=80=99? I've sent a new patch with an updated approach now, I started with the n-par-for-each procedure, and adapted it. It seems to work, let me know what you think :) Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAlyn6dhfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XdQ6hAAmj+05jYoaZtaN3SW7NVnDb5aehXj1wC5FvFf0qOH6+9+halMhYKx03wA vrLM4bE5/j4yn7bxflej3pFqV7DTbIWqMC1Pm0KM0ziWNxdU9QIzaaUg9VfSYnFt r9nbwbgAV5/lZRuU9PoIATcGAo6UrZRvfjERTh9miCJQ9oAQJw8g/34vCQANw+i7 MAWLvyv7o44+Kk+cwU1qln+Yi4gWVv4xfWrALlo4LsAeFX0X2Tgc72s6WKxRRV5X mpMJ/dybRDvmig5kys7oN1Z32fg2a8ih/losWqlLg+iacVsbtaeglSCLEM5jyj/n xJiSdrA03hvv+mQ94BOhhZx92nhbovRz5FQ30ykaAbXxp5ITwjxT6i1TWPS9mmmx KKrZE7JhfsfS/nc66bKSWnWDkXFO0yVTYn4nn4oIl+6Q75nus3JsBE2FWLHU5KSQ i2vLZpE7iJd0mypr3mCRbzMiyJtaCLQuYIO2gpL0o97z1Ao/ykjAkt4zLV9vs/MX DgdQ0IzSZdpvjdbJgnxE9Ki7MpO2ZnB848QbXXf9U+RXx8TC/w3KEJ5XsKR94xAa eg15D/YCcMwnBOx/V9QFSC1Rv2jSxxphAdxT/CcovQ4BS9Ieon4zFXQItGnfN48D hr7S/IAVIAcU1RPYWX6shEBeq5kWRQcLe2pZvoS1Du8tzObVa08= =wJ6f -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 16 13:06:15 2019 Received: (at 34982) by debbugs.gnu.org; 16 Apr 2019 17:06:15 +0000 Received: from localhost ([127.0.0.1]:37945 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hGRXC-0005l9-PF for submit@debbugs.gnu.org; Tue, 16 Apr 2019 13:06:14 -0400 Received: from eggs.gnu.org ([209.51.188.92]:46694) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hGRXB-0005ku-2L for 34982@debbugs.gnu.org; Tue, 16 Apr 2019 13:06:13 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52530) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hGRX5-0003eN-89; Tue, 16 Apr 2019 13:06:07 -0400 Received: from [2a01:e0a:1d:7270:6a6c:dc17:fc02:cfda] (port=59938 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hGRX4-0001ue-MQ; Tue, 16 Apr 2019 13:06:07 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: [bug#34982] [PATCH] guile-build-system: Support building in parallel. References: <20190324212345.4697-1-mail@cbaines.net> <87lg0w4om6.fsf@gnu.org> <87o95km2p3.fsf@cbaines.net> Date: Tue, 16 Apr 2019 19:06:04 +0200 In-Reply-To: <87o95km2p3.fsf@cbaines.net> (Christopher Baines's message of "Sat, 06 Apr 2019 00:50:48 +0100") Message-ID: <87o955ev7n.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 34982 Cc: 34982@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello Christopher! Christopher Baines skribis: > I've sent a new patch with an updated approach now, I started with the > n-par-for-each procedure, and adapted it. It seems to work, let me know > what you think :) Sorry for the delay, but=E2=80=A6 where=E2=80=99s the new patch? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 16 14:13:34 2019 Received: (at 34982) by debbugs.gnu.org; 16 Apr 2019 18:13:34 +0000 Received: from localhost ([127.0.0.1]:37976 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hGSaM-0007QN-8D for submit@debbugs.gnu.org; Tue, 16 Apr 2019 14:13:34 -0400 Received: from li622-129.members.linode.com ([212.71.249.129]:36418 helo=mira.cbaines.net) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hGSaH-0007QC-GV for 34982@debbugs.gnu.org; Tue, 16 Apr 2019 14:13:32 -0400 Received: from localhost (cpc102582-walt20-2-0-cust14.13-2.cable.virginm.net [86.27.34.15]) by mira.cbaines.net (Postfix) with ESMTPSA id A163016E94 for <34982@debbugs.gnu.org>; Tue, 16 Apr 2019 19:13:26 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 8ea8c650 for <34982@debbugs.gnu.org>; Tue, 16 Apr 2019 18:13:26 +0000 (UTC) From: Christopher Baines To: 34982@debbugs.gnu.org Subject: [PATCH] guile-build-system: Support building in parallel. Date: Tue, 16 Apr 2019 19:13:26 +0100 Message-Id: <20190416181326.2416-1-mail@cbaines.net> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 34982 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * guix/build/guile-build-system.scm (build): Use invoke-each, instead of for-each, to use multiple cores if available. (invoke-each, report-build-process): New procedures. --- guix/build/guile-build-system.scm | 96 +++++++++++++++++++++++++------ 1 file changed, 78 insertions(+), 18 deletions(-) diff --git a/guix/build/guile-build-system.scm b/guix/build/guile-build-system.scm index 0bed049436..5ad728361a 100644 --- a/guix/build/guile-build-system.scm +++ b/guix/build/guile-build-system.scm @@ -23,6 +23,7 @@ #:use-module (ice-9 match) #:use-module (ice-9 popen) #:use-module (ice-9 rdelim) + #:use-module (ice-9 threads) #:use-module (guix build utils) #:export (target-guile-effective-version %standard-phases @@ -65,6 +66,59 @@ Return #false if it cannot be determined." (setenv "GUIX_LOCPATH" (string-append locales "/lib/locale")) #t))) +(define* (invoke-each commands + #:key (max-processes (current-processor-count)) + report-progress) + "Run each command in COMMANDS in a separate process, using up to +MAX-PROCESSES processes in parallel. Call REPORT-PROGRESS at each step. +Raise an error if one of the processes exit with non-zero." + (define total + (length commands)) + + (define (wait-for-one-process) + (match (waitpid WAIT_ANY) + ((_ . status) + (unless (zero? (status:exit-val status)) + (error "process failed" status))))) + + (define (fork-and-run-command command) + (match (primitive-fork) + (0 + (apply execlp command)) + (pid + #t))) + + (let loop ((commands commands) + (running 0) + (completed 0)) + (match commands + (() + (or (zero? running) + (let ((running (- running 1)) + (completed (+ completed 1))) + (wait-for-one-process) + (report-progress total completed) + (loop commands running completed)))) + ((command . rest) + (if (< running max-processes) + (let ((running (+ 1 running))) + (fork-and-run-command command) + (report-progress total completed) + (loop rest running completed)) + (let ((running (- running 1)) + (completed (+ completed 1))) + (wait-for-one-process) + (report-progress total completed) + (loop commands running completed))))))) + +(define* (report-build-progress total completed + #:optional (log-port (current-error-port))) + "Report that COMPLETED out of TOTAL files have been completed." + (display #\cr log-port) + (format log-port "compiling...\t~5,1f% of ~d files" ;FIXME: i18n + (* 100. (/ completed total)) total) + (force-output log-port)) + (define* (build #:key outputs inputs native-inputs (source-directory ".") (compile-flags '()) @@ -101,24 +155,30 @@ Return #false if it cannot be determined." (match (getenv "GUILE_LOAD_COMPILED_PATH") (#f "") (path (string-append ":" path))))) - (for-each (lambda (file) - (let* ((go (string-append go-dir - (file-sans-extension file) - ".go"))) - ;; Install source module. - (install-file (string-append source-directory "/" file) - (string-append module-dir - "/" (dirname file))) - - ;; Install and compile module. - (apply invoke guild "compile" "-L" source-directory - "-o" go - (string-append source-directory "/" file) - flags))) - - ;; Arrange to strip SOURCE-DIRECTORY from file names. - (with-directory-excursion source-directory - (find-files "." scheme-file-regexp))) + + (let ((source-files + (with-directory-excursion source-directory + (find-files "." scheme-file-regexp)))) + (invoke-each + (map (lambda (file) + (cons* guild + "guild" "compile" + "-L" source-directory + "-o" (string-append go-dir + (file-sans-extension file) + ".go") + (string-append source-directory "/" file) + flags)) + source-files) + #:max-processes (parallel-job-count) + #:report-progress report-build-progress) + + (for-each + (lambda (file) + (install-file (string-append source-directory "/" file) + (string-append module-dir + "/" (dirname file)))) + source-files)) #t)) (define* (install-documentation #:key outputs -- 2.21.0 From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 16 14:26:02 2019 Received: (at 34982) by debbugs.gnu.org; 16 Apr 2019 18:26:02 +0000 Received: from localhost ([127.0.0.1]:37980 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hGSmP-0007jB-Jg for submit@debbugs.gnu.org; Tue, 16 Apr 2019 14:26:01 -0400 Received: from li622-129.members.linode.com ([212.71.249.129]:36426 helo=mira.cbaines.net) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hGSmN-0007iv-Fy for 34982@debbugs.gnu.org; Tue, 16 Apr 2019 14:26:00 -0400 Received: from localhost (cpc102582-walt20-2-0-cust14.13-2.cable.virginm.net [86.27.34.15]) by mira.cbaines.net (Postfix) with ESMTPSA id 7F32216E88; Tue, 16 Apr 2019 19:25:57 +0100 (BST) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id e948a209; Tue, 16 Apr 2019 18:25:57 +0000 (UTC) References: <20190324212345.4697-1-mail@cbaines.net> <87lg0w4om6.fsf@gnu.org> <87o95km2p3.fsf@cbaines.net> <87o955ev7n.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#34982] [PATCH] guile-build-system: Support building in parallel. In-reply-to: <87o955ev7n.fsf@gnu.org> Date: Tue, 16 Apr 2019 19:25:56 +0100 Message-ID: <87y349dcy3.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 34982 Cc: 34982@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Hello Christopher! > > Christopher Baines skribis: > >> I've sent a new patch with an updated approach now, I started with the >> n-par-for-each procedure, and adapted it. It seems to work, let me know >> what you think :) > > Sorry for the delay, but=E2=80=A6 where=E2=80=99s the new patch? Hmm, I'm not sure. I thought I sent it, but seemingly not. I've just sent the updated patch, and it's definately arrived now. Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAly2HjRfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XdJ8RAAg9ipf0M/pmoT7KT5BeRqX9NkLzJh5Do0FXN9cWTYIHzR6D9oJDYlrbHH mZ/oN2QapKb9LY/hsz31qn1W1zEji8qnCtUTc+69vRHxfxI6Z4o/Dk/fOKPMGlXB c9dg8aPTlgeUN0lEPTPwNMxnEPWxVs00qQCiN7YLMwTGwnsfebtJkd3dyR+u9YAe UW6GFsiNc3DpyjHsowV6NzCZ86CRCmNJk6bEX3OdHf+6kt3Uc9uYFPOQBsjQBq1d sgvrnEZSwQjGSxQbembg7Cv6ags2VDVVHBgiJi2UNWgPdDref8xm+xoLvgts/jSa cIoJoHSHgLtR5JO2/tkELkuR9OpLjeOshY98gurp0883I8GBq/MDx0JhefBh1PR3 X9kkG/WXZL4O0B7yGOZBiom8khQLaHt3/UrGFxJvqM1lmLWgotmSIbHOBHTym/Hj b70p8O5HBys2fF8Ak3DoIuv/58NgjDcsYstueUgN6kme0JtkZMzs16thfPLnj981 21skCVORjCYtKK7ZJB6ClpcmT0oKaOcssIu3x5Yj20CxBtDFd7o4JZWRuXBv8RmR r8IabtJ6I8zOqnDPfWMqireF7rhHtpdeE/LD0pnwKULtrpeBRdN8Ew5lP7pMhXMg 6CoNX5pZp90MGwX0BbsaQivouEreZ9cPwxEsBoi8fz8t0NGA5oE= =V2rz -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 16 15:30:36 2019 Received: (at 34982) by debbugs.gnu.org; 16 Apr 2019 19:30:36 +0000 Received: from localhost ([127.0.0.1]:38018 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hGTmt-0000nw-UP for submit@debbugs.gnu.org; Tue, 16 Apr 2019 15:30:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:51315) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hGTms-0000nk-Af for 34982@debbugs.gnu.org; Tue, 16 Apr 2019 15:30:34 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:55018) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hGTmm-0000iZ-Q2; Tue, 16 Apr 2019 15:30:28 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48804 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hGTmj-00060E-Fo; Tue, 16 Apr 2019 15:30:26 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: [bug#34982] [PATCH] guile-build-system: Support building in parallel. References: <20190324212345.4697-1-mail@cbaines.net> <20190416181326.2416-1-mail@cbaines.net> Date: Tue, 16 Apr 2019 21:30:21 +0200 In-Reply-To: <20190416181326.2416-1-mail@cbaines.net> (Christopher Baines's message of "Tue, 16 Apr 2019 19:13:26 +0100") Message-ID: <87ftqheoj6.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 34982 Cc: 34982@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Christopher Baines skribis: > * guix/build/guile-build-system.scm (build): Use invoke-each, instead of > for-each, to use multiple cores if available. > (invoke-each, report-build-process): New procedures. [...] > + (define (fork-and-run-command command) > + (match (primitive-fork) > + (0 > + (apply execlp command)) > + (pid > + #t))) To be on the safe side, you should probably wrap the =E2=80=98execlp=E2=80= =99 call like this: (dynamic-wind (const #t) (lambda () (apply execlp command)) (lambda () (primitive-exit 127))) This ensures that the child process exits immediately if something goes wrong (e.g., =E2=80=98execlp=E2=80=99 raises an exception because the execu= table could not be found.) Otherwise LGTM, thank you! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 19 06:23:59 2019 Received: (at 34982-done) by debbugs.gnu.org; 19 Apr 2019 10:23:59 +0000 Received: from localhost ([127.0.0.1]:43482 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hHQgZ-00015J-8i for submit@debbugs.gnu.org; Fri, 19 Apr 2019 06:23:59 -0400 Received: from mira.cbaines.net ([212.71.252.8]:47866) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hHQgW-000153-Af for 34982-done@debbugs.gnu.org; Fri, 19 Apr 2019 06:23:57 -0400 Received: from localhost (unknown [51.6.38.71]) by mira.cbaines.net (Postfix) with ESMTPSA id BF3AE16E16; Fri, 19 Apr 2019 08:44:20 +0100 (BST) Received: from phact (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 1b87fba5; Fri, 19 Apr 2019 07:43:42 +0000 (UTC) References: <20190324212345.4697-1-mail@cbaines.net> <20190416181326.2416-1-mail@cbaines.net> <87ftqheoj6.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#34982] [PATCH] guile-build-system: Support building in parallel. In-reply-to: <87ftqheoj6.fsf@gnu.org> Date: Fri, 19 Apr 2019 08:43:39 +0100 Message-ID: <87y346tp78.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 34982-done Cc: 34982-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Christopher Baines skribis: > >> * guix/build/guile-build-system.scm (build): Use invoke-each, instead of >> for-each, to use multiple cores if available. >> (invoke-each, report-build-process): New procedures. > > [...] > >> + (define (fork-and-run-command command) >> + (match (primitive-fork) >> + (0 >> + (apply execlp command)) >> + (pid >> + #t))) > > To be on the safe side, you should probably wrap the =E2=80=98execlp=E2= =80=99 call like > this: > > (dynamic-wind > (const #t) > (lambda () > (apply execlp command)) > (lambda () > (primitive-exit 127))) > > This ensures that the child process exits immediately if something goes > wrong (e.g., =E2=80=98execlp=E2=80=99 raises an exception because the exe= cutable could > not be found.) > > Otherwise LGTM, thank you! Great, I've added in dynamic-wind, made some minor tweaks to the output, and pushed this as 3fdb9a375f1cee7dd302349a9527437df20b3f61. Thanks for taking a look :) Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAly5fCtfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XdZgBAAr/arAGXex8vefpWbUZQZ6eraG2JM67NdvoiEu8yX1FR5OmiENDpIrwxT XLBAv/ypZZ90ZGnOcFbTVQTiPX57Z/V+RWJI7/N1MlbbPo0OoJexeGOJ6b5MJuOf mieDHYfJiLYbLWCBAxkqQK1WC1w3W0z6tstZ0ToZZ9mNAwNfBG9oJsVxZdtBnMaw N/qhCRVna90VrzLkdVqRFsdzjY5DB6Rcv6228tGFsGpm8zVkVux95OLi+TD1Deas bhPCAZr6vXM5ebimHXtWc9vTbPLQhDf95ke6gY96F1yeHUBc9i67xhDIrJXMslWh 5E92c0n8RJoqeDcOkGqnd27o347ktFabkDlwtkvYC9l7DsrmrB3gb980oFR5RTY0 uYuw5+1WSoX83Sf0iDbAffcUFfxOoLvnLCyLe486bHogX/14AbvW+nCB8aaBblxw S+IR2dsND9cuWM/+INzmO3XEpqo9Jl8kJSSv+8VautG8T+NMs92TzZ5kUmaN3gLo WCzZGbax4oJaQ5nGefx1tU6vbh0tm7j77MYIhNFUK4F3GT1OLGh4GvhyW5smH+fO 4zbUmrDRJbyN4LjQHMh8v2s0m0xzZ4MUvxnz/Lof/g9TrvQULomzj9wIu3D43aap ldSfLSO8quD7vO+RPfozByE8yfpvBW53wupGuqvaLJ+K8A26XEY= =Jmg/ -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Jun 20 07:10:31 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 17 May 2019 11:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator