From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 0/4] Add package and services for the Guix Build Coordinator Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Sep 2020 18:35:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43494@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.160045409712468 (code B ref -1); Fri, 18 Sep 2020 18:35:01 +0000 Received: (at submit) by debbugs.gnu.org; 18 Sep 2020 18:34:57 +0000 Received: from localhost ([127.0.0.1]:44378 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLDl-0003F2-Ef for submit@debbugs.gnu.org; Fri, 18 Sep 2020 14:34:57 -0400 Received: from lists.gnu.org ([209.51.188.17]:46262) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLDk-0003Ev-4H for submit@debbugs.gnu.org; Fri, 18 Sep 2020 14:34:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47080) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kJLDi-0004WE-Aa for guix-patches@gnu.org; Fri, 18 Sep 2020 14:34:55 -0400 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:36565) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kJLDg-0006bo-G0 for guix-patches@gnu.org; Fri, 18 Sep 2020 14:34:53 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id C6F7B27BBE7 for ; Fri, 18 Sep 2020 19:34:49 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id cf0e3d57 for ; Fri, 18 Sep 2020 18:34:47 +0000 (UTC) User-agent: mu4e 1.4.13; emacs 26.3 From: Christopher Baines Date: Fri, 18 Sep 2020 19:34:47 +0100 Message-ID: <87o8m39bnc.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/18 14:34:50 X-ACL-Warn: Detected OS = ??? X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --=-=-= Content-Type: text/plain This patch series is not particularly well tested, like the Guix Build Coordinator itself, but it should hopefully still work at least in some configurations. Christopher Baines (4): gnu: Add guix-build-coordinator. services: guix: Add guix-build-coordinator-service-type. services: guix: Add guix-build-coordinator-agent-service-type. services: guix: Add guix-build-coordinator-queue-builds-service-type. doc/guix.texi | 143 +++++++++ gnu/packages/package-management.scm | 89 ++++++ gnu/services/guix.scm | 433 +++++++++++++++++++++++++++- gnu/tests/guix.scm | 75 ++++- 4 files changed, 736 insertions(+), 4 deletions(-) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl9k/cdfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9Xfrig//b1nSvUfhfTzlMrAc5stuOAEBdd7hrG82aj8exISjDZa2UfzFN0ox828z ggjWbz69KjKP41OJfQgNGJ9GXqk582CvUaFloQBPQXH+3tBNJR+DlZHYEPhO5KlU K0ZuatnpqCC704DQOsqTPAxKsaKuqSHYy9j7XFMieqilznvv0YMp8qpepTRyza6F Y/H0qG4Ohfk3EhV5c99c4Yyvxx6x16TB+Gr93HBClVh3AYNbiEW2xuN7xSS03ljl yr26rpna8fY4sXH3hUgbyOu7B1LNeiAveLRnEW2E3OtRBhXFB/nBc/L17Uz2tje4 zRD86Bl4nWZ26v9FKaTwQAKaCl0S5Fi5DSPZviO8FySVRXfDHYy3PaZks3HHpDDQ /vgMbLba9BbBJ0Zq7/uoGgk9rWLUCuUDkfIigxlvyMHUAuNCnh3PnvjwJ3zenK6B HGSJ2Xi2JDE0pHrt4Tzs4uBG2JC+JyDZAhgZjJSkFfWP4YdJaLKq7J8BkFJV5c/E x42nxLdf9LaqeR7zfCqS8u0mBocmrvl9bfOY7aafo4ezE6ktDDp1ecwnxzpVlgIE Pn3ItRAIrs/w1O8i0ITHIMArpUdS+Yk+rfyEs400QB4kSXnphaBQdUKL0arRIFXr nqzi+I857L7ZqKWR7AKGSmWAJQay3KzzuENVho+dLJtC4wZxwcg= =uTsG -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 4/4] services: guix: Add guix-build-coordinator-queue-builds-service-type. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Sep 2020 18:41:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160045444813104 (code B ref 43494); Fri, 18 Sep 2020 18:41:02 +0000 Received: (at 43494) by debbugs.gnu.org; 18 Sep 2020 18:40:48 +0000 Received: from localhost ([127.0.0.1]:44395 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLJP-0003PG-Eo for submit@debbugs.gnu.org; Fri, 18 Sep 2020 14:40:47 -0400 Received: from mira.cbaines.net ([212.71.252.8]:35124) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLJN-0003Oq-FC for 43494@debbugs.gnu.org; Fri, 18 Sep 2020 14:40:45 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id C96EE27BBEA for <43494@debbugs.gnu.org>; Fri, 18 Sep 2020 19:40:44 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 00c67d71 for <43494@debbugs.gnu.org>; Fri, 18 Sep 2020 18:40:42 +0000 (UTC) From: Christopher Baines Date: Fri, 18 Sep 2020 19:40:42 +0100 Message-Id: <20200918184042.22660-4-mail@cbaines.net> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200918184042.22660-1-mail@cbaines.net> References: <20200918184042.22660-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * gnu/services/guix.scm (): New record type. (guix-build-coordinator-queue-builds-configuration, guix-build-coordinator-queue-builds-configuration?, guix-build-coordinator-queue-builds-configuration-package, guix-build-coordinator-queue-builds-configuration-user, guix-build-coordinator-queue-builds-coordinator, guix-build-coordinator-queue-builds-configuration-systems, guix-build-coordinator-queue-builds-configuration-system-and-targets, guix-build-coordinator-queue-builds-configuration-guix-data-service, guix-build-coordinator-queue-builds-configuration-processed-commits-file, guix-build-coordinator-queue-builds-shepherd-services, guix-build-coordinator-queue-builds-activation, guix-build-coordinator-queue-builds-account): New procedures. (guix-build-coordinator-queue-builds-service-type): New variable. --- doc/guix.texi | 38 +++++++++++++++ gnu/services/guix.scm | 108 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 146 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index db91831197..1b47a01217 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -27590,6 +27590,44 @@ input store items aren't already available. @end table @end deftp +@defvar {Scheme Variable} guix-build-coordinator-queue-builds-service-type +Service type for the +guix-build-coordinator-queue-builds-from-guix-data-service script. Its +value must be a @code{guix-build-coordinator-queue-builds-configuration} +object. +@end defvar + +@deftp {Data Type} guix-build-coordinator-queue-builds-configuration +Data type representing the options to the queue builds from guix data +service script. + +@table @asis +@item @code{package} (default: @code{guix-build-coordinator}) +The Guix Build Coordinator package to use. + +@item @code{user} (default: @code{"guix-build-coordinator-queue-builds"}) +The system user to run the service as. + +@item @code{coordinator} (default: @code{"http://localhost:8745"}) +The URI to use when connecting to the coordinator. + +@item @code{systems} (default: @code{#f}) +The systems for which to fetch derivations to build. + +@item @code{systems-and-targets} (default: @code{#f}) +An association list of system and target pairs for which to fetch +derivations to build. + +@item @code{guix-data-service} (default: @code{"https://data.guix.gnu.org"}) +The Guix Data Service instance from which to query to find out about +derivations to build. + +@item @code{processed-commits-file} (default: @code{"/var/lib/guix-build-coordinator-queue-builds/processed-commits"}) +A file to record which commits have been processed, to avoid needlessly +processing them again if the service is restarted. + +@end table +@end deftp @subsubheading Guix Data Service The @uref{http://data.guix.gnu.org,Guix Data Service} processes, stores diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm index 71e58237da..975febebd1 100644 --- a/gnu/services/guix.scm +++ b/gnu/services/guix.scm @@ -65,6 +65,18 @@ guix-build-coordinator-agent-service-type + guix-build-coordinator-queue-builds-configuration + guix-build-coordinator-queue-builds-configuration? + guix-build-coordinator-queue-builds-configuration-package + guix-build-coordinator-queue-builds-configuration-user + guix-build-coordinator-queue-builds-coordinator + guix-build-coordinator-queue-builds-configuration-systems + guix-build-coordinator-queue-builds-configuration-system-and-targets + guix-build-coordinator-queue-builds-configuration-guix-data-service + guix-build-coordinator-queue-builds-configuration-processed-commits-file + + guix-build-coordinator-queue-builds-service-type + guix-data-service-configuration guix-data-service-configuration? @@ -137,6 +149,28 @@ guix-build-coordinator-agent-configuration-non-derivation-substitute-urls (default #f))) +(define-record-type* + guix-build-coordinator-queue-builds-configuration + make-guix-build-coordinator-queue-builds-configuration + guix-build-coordinator-queue-builds-configuration? + (package guix-build-coordinator-queue-builds-configuration-package + (default guix-build-coordinator)) + (user guix-build-coordinator-queue-builds-configuration-user + (default "guix-build-coordinator-queue-builds")) + (coordinator guix-build-coordinator-queue-builds-coordinator + (default "http://localhost:8745")) + (systems guix-build-coordinator-queue-builds-configuration-systems + (default #f)) + (systems-and-targets + guix-build-coordinator-queue-builds-configuration-system-and-targets + (default #f)) + (guix-data-service + guix-build-coordinator-queue-builds-configuration-guix-data-service + (default "https://data.guix.gnu.org")) + (processed-commits-file + guix-build-coordinator-queue-builds-configuration-processed-commits-file + (default "/var/lib/guix-build-coordinator-queue-builds/processed-commits"))) + (define* (make-guix-build-coordinator-start-script database-uri-string allocation-strategy pid-file @@ -364,6 +398,80 @@ (description "Run an instance of the Guix Build Coordinator."))) +(define (guix-build-coordinator-queue-builds-shepherd-services config) + (match-record config + (package user coordinator systems systems-and-targets + guix-data-service processed-commits-file) + (list + (shepherd-service + (documentation "Guix Build Coordinator queue builds from Guix Data Service") + (provision '(guix-build-coordinator-queue-builds)) + (requirement '(networking)) + (start + #~(make-forkexec-constructor + (list + #$(file-append + package + "/bin/guix-build-coordinator-queue-builds-from-guix-data-service") + #$(string-append "--coordinator=" coordinator) + #$@(map (lambda (system) + (string-append "--system=" system)) + (or systems '())) + #$@(map (match-lambda + ((system . target) + (string-append "--system-and-target=" system "=" target))) + (or systems-and-targets '())) + #$@(if guix-data-service + #~(#$(string-append "--guix-data-service=" guix-data-service)) + #~()) + #$@(if processed-commits-file + #~(#$(string-append "--processed-commits-file=" + processed-commits-file)) + #~())) + #:user #$user + #:pid-file "/var/run/guix-build-coordinator-queue-builds/pid" + #:environment-variables + `(,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") + #:log-file "/var/log/guix-build-coordinator/queue-builds.log")) + (stop #~(make-kill-destructor)))))) + +(define (guix-build-coordinator-queue-builds-activation config) + #~(begin + (use-modules (guix build utils)) + + (mkdir-p "/var/log/guix-build-coordinator") + + ;; Allow writing the PID file + (mkdir-p "/var/run/guix-build-coordinator-queue-builds") + (chown "/var/run/guix-build-coordinator-queue-builds" + (passwd:uid %user) + (passwd:gid %user)))) + +(define (guix-build-coordinator-queue-builds-account config) + (list (user-account + (name (guix-build-coordinator-queue-builds-configuration-user config)) + (group "nogroup") + (system? #t) + (comment "Guix Build Coordinator queue-builds user") + (home-directory "/var/empty") + (shell (file-append shadow "/sbin/nologin"))))) + +(define guix-build-coordinator-queue-builds-service-type + (service-type + (name 'guix-build-coordinator-queue-builds) + (extensions + (list + (service-extension shepherd-root-service-type + guix-build-coordinator-queue-builds-shepherd-services) + (service-extension activation-service-type + guix-build-coordinator-queue-builds-activation) + (service-extension account-service-type + guix-build-coordinator-queue-builds-account))) + (description + "Run the Guix Build Coordinator queue builds script."))) + ;;; ;;; Guix Data Service -- 2.28.0 From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 1/4] gnu: Add guix-build-coordinator. References: <87o8m39bnc.fsf@cbaines.net> In-Reply-To: <87o8m39bnc.fsf@cbaines.net> Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Sep 2020 18:41:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160045444813111 (code B ref 43494); Fri, 18 Sep 2020 18:41:02 +0000 Received: (at 43494) by debbugs.gnu.org; 18 Sep 2020 18:40:48 +0000 Received: from localhost ([127.0.0.1]:44397 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLJP-0003PJ-UA for submit@debbugs.gnu.org; Fri, 18 Sep 2020 14:40:48 -0400 Received: from mira.cbaines.net ([212.71.252.8]:35118) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLJN-0003On-FD for 43494@debbugs.gnu.org; Fri, 18 Sep 2020 14:40:45 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id 66D1427BBE7 for <43494@debbugs.gnu.org>; Fri, 18 Sep 2020 19:40:44 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 3452e035 for <43494@debbugs.gnu.org>; Fri, 18 Sep 2020 18:40:42 +0000 (UTC) From: Christopher Baines Date: Fri, 18 Sep 2020 19:40:39 +0100 Message-Id: <20200918184042.22660-1-mail@cbaines.net> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * gnu/packages/package-management.scm (guix-build-coordinator): New variable. --- gnu/packages/package-management.scm | 89 +++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm index ec87226197..4272d3134d 100644 --- a/gnu/packages/package-management.scm +++ b/gnu/packages/package-management.scm @@ -50,6 +50,7 @@ #:use-module (gnu packages cpio) #:use-module (gnu packages crypto) #:use-module (gnu packages curl) + #:use-module (gnu packages databases) #:use-module (gnu packages dbm) #:use-module (gnu packages docbook) #:use-module (gnu packages file) @@ -981,6 +982,94 @@ environments.") ;; and the fonts included in this package are licensed OFL1.1. (license (list license:gpl3+ license:agpl3+ license:silofl1.1)))) +(define-public guix-build-coordinator + (let ((commit "5e8408c833e209efddfa0159114b90400c1aaf4d") + (revision "0")) + (package + (name "guix-build-coordinator") + (version (git-version "0" revision commit)) + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://git.cbaines.net/git/guix/build-coordinator") + (commit commit))) + (sha256 + (base32 + "0f7m1zg9mlb2m22qyblglaa36h8f49b810jc9j5b0hsb42ijwh4b")) + (file-name (string-append name "-" version "-checkout")))) + (build-system gnu-build-system) + (arguments + '(#:modules ((guix build utils) + (guix build gnu-build-system) + (ice-9 ftw) + (ice-9 match) + (ice-9 rdelim) + (ice-9 popen)) + #:phases + (modify-phases %standard-phases + (add-before 'build 'set-GUILE_AUTO_COMPILE + (lambda _ + ;; To avoid warnings relating to 'guild'. + (setenv "GUILE_AUTO_COMPILE" "0") + #t)) + (add-after 'install 'wrap-executable + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (bin (string-append out "/bin")) + (guile (assoc-ref inputs "guile")) + (guile-effective-version + (read-line + (open-pipe* OPEN_READ + (string-append guile "/bin/guile") + "-c" "(display (effective-version))"))) + (scm (string-append out "/share/guile/site/" + guile-effective-version)) + (go (string-append out "/lib/guile/" + guile-effective-version + "/site-ccache"))) + (for-each + (lambda (file) + (simple-format (current-error-port) + "wrapping: ~A\n" + (string-append bin "/" file)) + (wrap-program (string-append bin "/" file) + `("PATH" ":" prefix + (,bin ,(assoc-ref inputs "sqitch"))) + `("PERL5LIB" ":" prefix + (,(getenv "PERL5LIB"))) + `("GUILE_LOAD_PATH" ":" prefix + (,scm ,(getenv "GUILE_LOAD_PATH"))) + `("GUILE_LOAD_COMPILED_PATH" ":" prefix + (,go ,(getenv "GUILE_LOAD_COMPILED_PATH"))))) + (scandir bin + (match-lambda + ((or "." "..") #f) + (_ #t)))) + #t))) + (delete 'strip)))) ; As the .go files aren't compatible + (native-inputs + `(("pkg-config" ,pkg-config) + ("autoconf" ,autoconf) + ("automake" ,automake) + ("perl" ,perl))) + (inputs + `(("guile" ,guile-3.0-latest) + ("guile-fibers" ,guile-fibers) + ("guile-prometheus" ,guile-prometheus) + ("guile-gcrypt" ,guile-gcrypt) + ("guile-json" ,guile-json-3) + ("guile-lzlib" ,guile-lzlib) + ("guile-sqlite3" ,guile-sqlite3) + ("guix" ,guix) + ("sqlite" ,sqlite) + ("sqitch" ,sqitch) + ("perl-dbd-sqlite" ,perl-dbd-sqlite))) + (home-page "https://git.cbaines.net/guile/guix/build-coordinator") + (synopsis "") + (description + "") + (license license:gpl3+)))) + (define-public guix-jupyter (package (name "guix-jupyter") -- 2.28.0 From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 3/4] services: guix: Add guix-build-coordinator-agent-service-type. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Sep 2020 18:41:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160045444813119 (code B ref 43494); Fri, 18 Sep 2020 18:41:03 +0000 Received: (at 43494) by debbugs.gnu.org; 18 Sep 2020 18:40:48 +0000 Received: from localhost ([127.0.0.1]:44399 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLJQ-0003PQ-8j for submit@debbugs.gnu.org; Fri, 18 Sep 2020 14:40:48 -0400 Received: from mira.cbaines.net ([212.71.252.8]:35122) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLJN-0003Op-FK for 43494@debbugs.gnu.org; Fri, 18 Sep 2020 14:40:46 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id 9FBED27BBE9 for <43494@debbugs.gnu.org>; Fri, 18 Sep 2020 19:40:44 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 39d6d5cb for <43494@debbugs.gnu.org>; Fri, 18 Sep 2020 18:40:42 +0000 (UTC) From: Christopher Baines Date: Fri, 18 Sep 2020 19:40:41 +0100 Message-Id: <20200918184042.22660-3-mail@cbaines.net> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200918184042.22660-1-mail@cbaines.net> References: <20200918184042.22660-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * gnu/services/guix.scm (): New record type. (guix-build-coordinator-agent-configuration, guix-build-coordinator-agent-configuration?, guix-build-coordinator-agent-configuration-package, guix-build-coordinator-agent-configuration-user, guix-build-coordinator-agent-configuration-coordinator, guix-build-coordinator-agent-configuration-uuid), guix-build-coordinator-agent-configuration-password, guix-build-coordinator-agent-configuration-password-file, guix-build-coordinator-agent-configuration-systems, guix-build-coordinator-agent-configuration-max-parallel-builds, guix-build-coordinator-agent-configuration-derivation-substitute-urls, guix-build-coordinator-agent-configuration-non-derivation-substitute-urls, guix-build-coordinator-agent-shepherd-services, guix-build-coordinator-agent-activation, guix-build-coordinator-agent-account): New procedures. (guix-build-coordinator-agent-service-type): New variable. * doc/guix.texi (Guix Services): Document it. --- doc/guix.texi | 50 +++++++++++++++++ gnu/services/guix.scm | 121 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 171 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index e83008c177..db91831197 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -27541,6 +27541,56 @@ The Guile package with which to run the Guix Build Coordinator. @end table @end deftp +@defvar {Scheme Variable} guix-build-coordinator-agent-service-type +Service type for a Guix Build Coordinator agent. Its value must be a +@code{guix-build-coordinator-agent-configuration} object. +@end defvar + +@deftp {Data Type} guix-build-coordinator-agent-configuration +Data type representing the configuration a Guix Build Coordinator agent. + +@table @asis +@item @code{package} (default: @code{guix-build-coordinator}) +The Guix Build Coordinator package to use. + +@item @code{user} (default: @code{"guix-build-coordinator-agent"}) +The system user to run the service as. + +@item @code{coordinator} (default: @code{"http://localhost:8745"}) +The URI to use when connecting to the coordinator. + +@item @code{uuid} +The UUID of the agent. This should be generated by the coordinator +process, stored in the coordinator database, and used by the intended +agent. + +@item @code{password} (default: @code{#f}) +The password to use when connecting to the coordinator. A file to read +the password from can also be specified, and this is more secure. + +@item @code{password-file} (default: @code{#f}) +A file containing the password to use when connecting to the +coordinator. + +@item @code{systems} (default: @var{#f}) +The systems for which this agent should fetch builds. The agent process +will use the current system it's running on as the default. + +@item @code{max-parallel-builds} (default: @code{1}) +The number of builds to perform in parallel. + +@item @code{derivation-substitute-urls} (default: @code{1}) +URLs from which to attempt to fetch substitutes for derivations, if the +derivations aren't already available. + +@item @code{non-derivation-substitute-urls} (default: @code{1}) +URLs from which to attempt to fetch substitutes for build inputs, if the +input store items aren't already available. + +@end table +@end deftp + + @subsubheading Guix Data Service The @uref{http://data.guix.gnu.org,Guix Data Service} processes, stores and provides data about GNU Guix. This includes information about diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm index de14fcadb2..71e58237da 100644 --- a/gnu/services/guix.scm +++ b/gnu/services/guix.scm @@ -50,6 +50,21 @@ guix-build-coordinator-service-type + guix-build-coordinator-agent-configuration + guix-build-coordinator-agent-configuration? + guix-build-coordinator-agent-configuration-package + guix-build-coordinator-agent-configuration-user + guix-build-coordinator-agent-configuration-coordinator + guix-build-coordinator-agent-configuration-uuid) + guix-build-coordinator-agent-configuration-password + guix-build-coordinator-agent-configuration-password-file + guix-build-coordinator-agent-configuration-systems + guix-build-coordinator-agent-configuration-max-parallel-builds + guix-build-coordinator-agent-configuration-derivation-substitute-urls + guix-build-coordinator-agent-configuration-non-derivation-substitute-urls + + guix-build-coordinator-agent-service-type + guix-data-service-configuration guix-data-service-configuration? @@ -95,6 +110,33 @@ (guile guix-build-coordinator-configuration-guile (default guile-3.0-latest))) +(define-record-type* + guix-build-coordinator-agent-configuration + make-guix-build-coordinator-agent-configuration + guix-build-coordinator-agent-configuration? + (package guix-build-coordinator-agent-configuration-package + (default guix-build-coordinator)) + (user guix-build-coordinator-agent-configuration-user + (default "guix-build-coordinator-agent")) + (coordinator guix-build-coordinator-agent-configuration-coordinator + (default "http://localhost:8745")) + (uuid guix-build-coordinator-agent-configuration-uuid) + (password guix-build-coordinator-agent-configuration-password + (default #f)) + (password-file guix-build-coordinator-agent-configuration-password-file + (default #f)) + (systems guix-build-coordinator-agent-configuration-systems + (default #f)) + (max-parallel-builds + guix-build-coordinator-agent-configuration-max-parallel-builds + (default 1)) + (derivation-substitute-urls + guix-build-coordinator-agent-configuration-derivation-substitute-urls + (default #f)) + (non-derivation-substitute-urls + guix-build-coordinator-agent-configuration-non-derivation-substitute-urls + (default #f))) + (define* (make-guix-build-coordinator-start-script database-uri-string allocation-strategy pid-file @@ -243,6 +285,85 @@ (description "Run an instance of the Guix Build Coordinator."))) +(define (guix-build-coordinator-agent-shepherd-services config) + (match-record config + (package user coordinator uuid password password-file max-parallel-builds + derivation-substitute-urls non-derivation-substitute-urls + systems) + (list + (shepherd-service + (documentation "Guix Build Coordinator Agent") + (provision '(guix-build-coordinator-agent)) + (requirement '(networking)) + (start #~(make-forkexec-constructor + (list #$(file-append package "/bin/guix-build-coordinator-agent") + #$(string-append "--coordinator=" coordinator) + #$(string-append "--uuid=" uuid) + #$@(if password + #~(#$(string-append "--password=" password)) + #~()) + #$@(if password-file + #~(#$(string-append "--password-file=" password-file)) + #~()) + #$(simple-format #f "--max-parallel-builds=~A" + max-parallel-builds) + #$@(if derivation-substitute-urls + #~(#$(string-append + "--derivation-substitute-urls=" + (string-join derivation-substitute-urls " "))) + #~()) + #$@(if non-derivation-substitute-urls + #~(#$(string-append + "--non-derivation-substitute-urls=" + (string-join derivation-substitute-urls " "))) + #~()) + #$@(map (lambda (system) + (string-append "--system=" system)) + (or systems '()))) + #:user #$user + #:pid-file "/var/run/guix-build-coordinator-agent/pid" + #:environment-variables + `(,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") + #:log-file "/var/log/guix-build-coordinator/agent.log")) + (stop #~(make-kill-destructor)))))) + +(define (guix-build-coordinator-agent-activation config) + #~(begin + (use-modules (guix build utils)) + + (mkdir-p "/var/log/guix-build-coordinator") + + ;; Allow writing the PID file + (mkdir-p "/var/run/guix-build-coordinator-agent") + (chown "/var/run/guix-build-coordinator-agent" + (passwd:uid %user) + (passwd:gid %user)))) + +(define (guix-build-coordinator-agent-account config) + (list (user-account + (name (guix-build-coordinator-agent-configuration-user config)) + (group "nogroup") + (system? #t) + (comment "Guix Build Coordinator agent user") + (home-directory "/var/empty") + (shell (file-append shadow "/sbin/nologin"))))) + +(define guix-build-coordinator-agent-service-type + (service-type + (name 'guix-build-coordinator-agent) + (extensions + (list + (service-extension shepherd-root-service-type + guix-build-coordinator-agent-shepherd-services) + (service-extension activation-service-type + guix-build-coordinator-agent-activation) + (service-extension account-service-type + guix-build-coordinator-agent-account))) + (description + "Run an instance of the Guix Build Coordinator."))) + ;;; ;;; Guix Data Service -- 2.28.0 From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 2/4] services: guix: Add guix-build-coordinator-service-type. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Sep 2020 18:41:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160045444913125 (code B ref 43494); Fri, 18 Sep 2020 18:41:03 +0000 Received: (at 43494) by debbugs.gnu.org; 18 Sep 2020 18:40:49 +0000 Received: from localhost ([127.0.0.1]:44401 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLJQ-0003Pa-Qz for submit@debbugs.gnu.org; Fri, 18 Sep 2020 14:40:49 -0400 Received: from mira.cbaines.net ([212.71.252.8]:35120) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLJN-0003Oo-FC for 43494@debbugs.gnu.org; Fri, 18 Sep 2020 14:40:46 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id 850CC27BBE8 for <43494@debbugs.gnu.org>; Fri, 18 Sep 2020 19:40:44 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 5dee5f60 for <43494@debbugs.gnu.org>; Fri, 18 Sep 2020 18:40:42 +0000 (UTC) From: Christopher Baines Date: Fri, 18 Sep 2020 19:40:40 +0100 Message-Id: <20200918184042.22660-2-mail@cbaines.net> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200918184042.22660-1-mail@cbaines.net> References: <20200918184042.22660-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * gnu/services/guix.scm (): New record type. (guix-build-coordinator-configuration, guix-build-coordinator-configuration?, guix-build-coordinator-configuration-package, guix-build-coordinator-configuration-user, guix-build-coordinator-configuration-group, guix-build-coordinator-configuration-datastore-uri-string, guix-build-coordinator-configuration-agent-communication-uri-string, guix-build-coordinator-configuration-client-communication-uri-string, guix-build-coordinator-configuration-allocation-strategy, guix-build-coordinator-configuration-hooks, guix-build-coordinator-configuration-guile, make-guix-build-coordinator-start-script, guix-build-coordinator-shepherd-services, guix-build-coordinator-activation, guix-build-coordinator-account): New procedures. (guix-build-coordinator-service-type): New variable. * gnu/tests/guix.scm (%test-guix-build-coordinator): New variable. * doc/guix.texi (Guix Services): Document it. --- doc/guix.texi | 55 ++++++++++++ gnu/services/guix.scm | 204 +++++++++++++++++++++++++++++++++++++++++- gnu/tests/guix.scm | 75 +++++++++++++++- 3 files changed, 330 insertions(+), 4 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 88128a4b3a..e83008c177 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -27486,6 +27486,61 @@ The complete list of possible options can be found in the man page for @node Guix Services @subsection Guix Services +@subsubheading Guix Build Coordinator +The @uref{https://git.cbaines.net/guix/build-coordinator/,Guix Build +Coordinator} aids in building derivations. The Guix Daemon is still +used to build the derivations, but the Guix Build Coordinator provides +ways of performing builds across many machines and working with the +results of builds. + +There is a script to run the coordinator component of the Guix Build +Coordinator, but the Guix service uses a custom Guile script instead, to +provide better integration with G-expressions used in the configuration. + +@defvar {Scheme Variable} guix-build-coordinator-service-type +Service type for the Guix Build Coordinator. Its value must be a +@code{guix-build-coordinator-configuration} object. +@end defvar + +@deftp {Data Type} guix-build-coordinator-configuration +Data type representing the configuration of the Guix Build Coordinator. + +@table @asis +@item @code{package} (default: @code{guix-build-coordinator}) +The Guix Build Coordinator package to use. + +@item @code{user} (default: @code{"guix-build-coordinator"}) +The system user to run the service as. + +@item @code{group} (default: @code{"guix-build-coordinator"}) +The system group to run the service as. + +@item @code{database-uri-string} (default: @code{"sqlite:///var/lib/guix-build-coordinator/guix_build_coordinator.db"}) +The URI to use for the database. + +@item @code{agent-communication-uri} (default: @code{"http://0.0.0.0:8745"}) +The URI describing how to listen to requests from agent processes. + +@item @code{client-communication-uri} (default: @code{"http://127.0.0.1:8746"}) +The URI describing how to listen to requests from clients. The client +API allows submitting builds and currently isn't authenticated, so take +care when configuring this value. + +@item @code{allocation-strategy} (default: @code{#~basic-build-allocation-strategy}) +A G-expression for the allocation strategy to be used. This is a +procedure that takes the datastore as an argument and populates the +allocation plan in the database. + +@item @code{hooks} (default: @var{'()}) +An association list of hooks. These provide a way to execute arbitrary +code upon certian events, like a build result being processed. + +@item @code{guile} (default: @code{guile-3.0-latest}) +The Guile package with which to run the Guix Build Coordinator. + +@end table +@end deftp + @subsubheading Guix Data Service The @uref{http://data.guix.gnu.org,Guix Data Service} processes, stores and provides data about GNU Guix. This includes information about diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm index 10a8581a62..de14fcadb2 100644 --- a/gnu/services/guix.scm +++ b/gnu/services/guix.scm @@ -17,20 +17,40 @@ ;;; along with GNU Guix. If not, see . (define-module (gnu services guix) + #:use-module (srfi srfi-1) #:use-module (ice-9 match) #:use-module (guix gexp) #:use-module (guix records) + #:use-module (guix packages) #:use-module ((gnu packages base) #:select (glibc-utf8-locales)) #:use-module (gnu packages admin) + #:use-module (gnu packages databases) #:use-module (gnu packages web) + #:use-module (gnu packages guile) + #:use-module (gnu packages guile-xyz) + #:use-module (gnu packages package-management) #:use-module (gnu services) #:use-module (gnu services base) #:use-module (gnu services admin) #:use-module (gnu services shepherd) #:use-module (gnu services getmail) #:use-module (gnu system shadow) - #:export ( + #:export (guix-build-coordinator-configuration + guix-build-coordinator-configuration? + guix-build-coordinator-configuration-package + guix-build-coordinator-configuration-user + guix-build-coordinator-configuration-group + guix-build-coordinator-configuration-datastore-uri-string + guix-build-coordinator-configuration-agent-communication-uri-string + guix-build-coordinator-configuration-client-communication-uri-string + guix-build-coordinator-configuration-allocation-strategy + guix-build-coordinator-configuration-hooks + guix-build-coordinator-configuration-guile + + guix-build-coordinator-service-type + + guix-data-service-configuration guix-data-service-configuration? guix-data-service-package @@ -45,11 +65,189 @@ ;;;; Commentary: ;;; -;;; This module implements a service that to run instances of the Guix Data -;;; Service, which provides data about Guix over time. +;;; Services specifically related to GNU Guix. ;;; ;;;; Code: +(define-record-type* + guix-build-coordinator-configuration make-guix-build-coordinator-configuration + guix-build-coordinator-configuration? + (package guix-build-coordinator-configuration-package + (default guix-build-coordinator)) + (user guix-build-coordinator-configuration-user + (default "guix-build-coordinator")) + (group guix-build-coordinator-configuration-group + (default "guix-build-coordinator")) + (database-uri-string + guix-build-coordinator-configuration-datastore-uri-string + (default "sqlite:///var/lib/guix-build-coordinator/guix_build_coordinator.db")) + (agent-communication-uri-string + guix-build-coordinator-configuration-agent-communication-uri-string + (default "http://0.0.0.0:8745")) + (client-communication-uri-string + guix-build-coordinator-configuration-client-communication-uri-string + (default "http://127.0.0.1:8746")) + (allocation-strategy + guix-build-coordinator-configuration-allocation-strategy + (default #~basic-build-allocation-strategy)) + (hooks guix-build-coordinator-configuration-hooks + (default '())) + (guile guix-build-coordinator-configuration-guile + (default guile-3.0-latest))) + +(define* (make-guix-build-coordinator-start-script database-uri-string + allocation-strategy + pid-file + guix-build-coordinator-package + #:key + agent-communication-uri-string + client-communication-uri-string + (hooks '()) + (guile guile-3.0)) + (program-file + "start-guix-build-coordinator" + (with-extensions (cons guix-build-coordinator-package + ;; This is a poorly constructed Guile load path, + ;; since it contains things that aren't Guile + ;; libraries, but it means that the Guile libraries + ;; needed for the Guix Build Coordinator don't need + ;; to be individually specified here. + (map second (package-inputs + guix-build-coordinator-package))) + #~(begin + (use-modules (srfi srfi-1) + (ice-9 match) + (web uri) + (prometheus) + (guix-build-coordinator hooks) + (guix-build-coordinator datastore) + (guix-build-coordinator build-allocator) + (guix-build-coordinator coordinator)) + + ;; Otherwise sqitch can't use SQlite + (setenv "PERL5LIB" + #$(file-append perl-dbd-sqlite "/lib/perl5/site_perl")) + + (let* ((metrics-registry (make-metrics-registry + #:namespace + "guixbuildcoordinator_")) + (datastore (database-uri->datastore + #$database-uri-string + #:metrics-registry metrics-registry)) + (hooks + (list #$@(map (match-lambda + ((name . hook-gexp) + #~(cons name #$hook-gexp))) + hooks))) + (hooks-with-defaults + `(,@hooks + ,@(remove (match-lambda + ((name . _) (assq-ref hooks name))) + %default-hooks))) + (build-coordinator (make-build-coordinator + #:datastore datastore + #:hooks hooks-with-defaults + #:metrics-registry metrics-registry + #:allocation-strategy #$allocation-strategy))) + + (run-coordinator-service + build-coordinator + #:update-datastore? #t + #:pid-file #$pid-file + #:agent-communication-uri (string->uri + #$agent-communication-uri-string) + #:client-communication-uri (string->uri + #$client-communication-uri-string))))) + #:guile guile)) + +(define (guix-build-coordinator-shepherd-services config) + (match-record config + (package user group database-uri-string + agent-communication-uri-string + client-communication-uri-string + allocation-strategy + hooks + guile) + (list + (shepherd-service + (documentation "Guix Build Coordinator") + (provision '(guix-build-coordinator)) + (requirement '(networking)) + (start #~(make-forkexec-constructor + (list #$(make-guix-build-coordinator-start-script + database-uri-string + allocation-strategy + "/var/run/guix-build-coordinator/pid" + package + #:agent-communication-uri-string + agent-communication-uri-string + #:client-communication-uri-string + client-communication-uri-string + #:hooks hooks + #:guile guile)) + #:user #$user + #:group #$group + #:pid-file "/var/run/guix-build-coordinator/pid" + ;; Allow time for migrations to run + #:pid-file-timeout 60 + #:environment-variables + `(,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") + #:log-file "/var/log/guix-build-coordinator/coordinator.log")) + (stop #~(make-kill-destructor)))))) + +(define (guix-build-coordinator-activation config) + #~(begin + (use-modules (guix build utils)) + + (define %user (getpw "guix-build-coordinator")) + + (chmod "/var/lib/guix-build-coordinator" #o755) + + (mkdir-p "/var/log/guix-build-coordinator") + + ;; Allow writing the PID file + (mkdir-p "/var/run/guix-build-coordinator") + (chown "/var/run/guix-build-coordinator" + (passwd:uid %user) + (passwd:gid %user)))) + +(define (guix-build-coordinator-account config) + (match-record config + (user group) + (list (user-group + (name group) + (system? #t)) + (user-account + (name user) + (group group) + (system? #t) + (comment "Guix Build Coordinator user") + (home-directory "/var/lib/guix-build-coordinator") + (shell (file-append shadow "/sbin/nologin")))))) + +(define guix-build-coordinator-service-type + (service-type + (name 'guix-build-coordinator) + (extensions + (list + (service-extension shepherd-root-service-type + guix-build-coordinator-shepherd-services) + (service-extension activation-service-type + guix-build-coordinator-activation) + (service-extension account-service-type + guix-build-coordinator-account))) + (default-value + (guix-build-coordinator-configuration)) + (description + "Run an instance of the Guix Build Coordinator."))) + + +;;; +;;; Guix Data Service +;;; + (define-record-type* guix-data-service-configuration make-guix-data-service-configuration guix-data-service-configuration? diff --git a/gnu/tests/guix.scm b/gnu/tests/guix.scm index 6139e31cf0..20b67d55d3 100644 --- a/gnu/tests/guix.scm +++ b/gnu/tests/guix.scm @@ -35,7 +35,80 @@ #:use-module (guix store) #:use-module (guix utils) #:use-module (ice-9 match) - #:export (%test-guix-data-service)) + #:export (%test-guix-build-coordinator + %test-guix-data-service)) + +;;; +;;; Guix Build Coordinator +;;; + +(define %guix-build-coordinator-os + (simple-operating-system + (service dhcp-client-service-type) + (service guix-build-coordinator-service-type))) + +(define (run-guix-build-coordinator-test) + (define os + (marionette-operating-system + %guix-build-coordinator-os + #:imported-modules '((gnu services herd) + (guix combinators)))) + + (define forwarded-port 8745) + + (define vm + (virtual-machine + (operating-system os) + (memory-size 1024) + (port-forwardings `((,forwarded-port . 8745))))) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-11) (srfi srfi-64) + (gnu build marionette) + (web uri) + (web client) + (web response)) + + (define marionette + (make-marionette (list #$vm))) + + (mkdir #$output) + (chdir #$output) + + (test-begin "guix-build-coordinator") + + (test-assert "service running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'guix-build-coordinator) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (test-equal "http-get" + 200 + (let-values + (((response text) + (http-get #$(simple-format + #f "http://localhost:~A/metrics" forwarded-port) + #:decode-body? #t))) + (response-code response))) + + (test-end) + (exit (= (test-runner-fail-count (test-runner-current)) 0))))) + + (gexp->derivation "guix-build-coordinator-test" test)) + +(define %test-guix-build-coordinator + (system-test + (name "guix-build-coordinator") + (description "Connect to a running Guix Build Coordinator.") + (value (run-guix-build-coordinator-test)))) ;;; -- 2.28.0 From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 1/4] gnu: Add guix-build-coordinator. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Sep 2020 20:10:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Christopher Baines Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160045974621304 (code B ref 43494); Fri, 18 Sep 2020 20:10:01 +0000 Received: (at 43494) by debbugs.gnu.org; 18 Sep 2020 20:09:06 +0000 Received: from localhost ([127.0.0.1]:44514 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJMgs-0005XY-8w for submit@debbugs.gnu.org; Fri, 18 Sep 2020 16:09:06 -0400 Received: from eggs.gnu.org ([209.51.188.92]:32966) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJMgq-0005X5-2F for 43494@debbugs.gnu.org; Fri, 18 Sep 2020 16:09:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:55939) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kJMgk-0001Ex-R1; Fri, 18 Sep 2020 16:08:58 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=45400 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kJMgk-0006Dg-Ez; Fri, 18 Sep 2020 16:08:58 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87o8m39bnc.fsf@cbaines.net> <20200918184042.22660-1-mail@cbaines.net> Date: Fri, 18 Sep 2020 22:08:56 +0200 In-Reply-To: <20200918184042.22660-1-mail@cbaines.net> (Christopher Baines's message of "Fri, 18 Sep 2020 19:40:39 +0100") Message-ID: <87lfh6vodj.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Christopher Baines skribis: > * gnu/packages/package-management.scm (guix-build-coordinator): New varia= ble. [...] > + (lambda* (#:key inputs outputs #:allow-other-keys) > + (let* ((out (assoc-ref outputs "out")) > + (bin (string-append out "/bin")) > + (guile (assoc-ref inputs "guile")) > + (guile-effective-version > + (read-line > + (open-pipe* OPEN_READ > + (string-append guile "/bin/guile") > + "-c" "(display (effective-version))"))) Maybe use =E2=80=98target-guile-effective-version=E2=80=99 from (guix build guile-build-system). > + (wrap-program (string-append bin "/" file) > + `("PATH" ":" prefix > + (,bin ,(assoc-ref inputs "sqitch"))) > + `("PERL5LIB" ":" prefix > + (,(getenv "PERL5LIB"))) Do we really need PERL5LIB here? Shouldn=E2=80=99t it be done in the wrapp= er of =E2=80=98sqitch=E2=80=99 directly? > + (scandir bin > + (match-lambda > + ((or "." "..") #f) > + (_ #t)))) I think you could just use (find-files bin). > + (inputs > + `(("guile" ,guile-3.0-latest) Rather ,@(assoc-ref (package-native-inputs guix) "guile"). > + ("sqitch" ,sqitch) > + ("perl-dbd-sqlite" ,perl-dbd-sqlite))) Shouldn=E2=80=99t perl-dbd-sqlite be a dependency of sqitch? > + (home-page "https://git.cbaines.net/guile/guix/build-coordinator") > + (synopsis "") > + (description > + "") Missing! :-) LGTM with changes along these lines! Ludo=E2=80=99. From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 2/4] services: guix: Add guix-build-coordinator-service-type. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Sep 2020 20:22:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Christopher Baines Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160046047022430 (code B ref 43494); Fri, 18 Sep 2020 20:22:01 +0000 Received: (at 43494) by debbugs.gnu.org; 18 Sep 2020 20:21:10 +0000 Received: from localhost ([127.0.0.1]:44530 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJMsY-0005pi-B4 for submit@debbugs.gnu.org; Fri, 18 Sep 2020 16:21:10 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35840) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJMsV-0005pU-SA for 43494@debbugs.gnu.org; Fri, 18 Sep 2020 16:21:09 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:56360) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kJMsQ-0002s3-Gb; Fri, 18 Sep 2020 16:21:02 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=45402 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kJMsO-0007V7-Fb; Fri, 18 Sep 2020 16:21:01 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20200918184042.22660-1-mail@cbaines.net> <20200918184042.22660-2-mail@cbaines.net> Date: Fri, 18 Sep 2020 22:20:58 +0200 In-Reply-To: <20200918184042.22660-2-mail@cbaines.net> (Christopher Baines's message of "Fri, 18 Sep 2020 19:40:40 +0100") Message-ID: <87a6xmvnth.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Christopher Baines skribis: > * gnu/services/guix.scm (): New rec= ord > type. > (guix-build-coordinator-configuration, guix-build-coordinator-configurati= on?, > guix-build-coordinator-configuration-package, > guix-build-coordinator-configuration-user, > guix-build-coordinator-configuration-group, > guix-build-coordinator-configuration-datastore-uri-string, > guix-build-coordinator-configuration-agent-communication-uri-string, > guix-build-coordinator-configuration-client-communication-uri-string, > guix-build-coordinator-configuration-allocation-strategy, > guix-build-coordinator-configuration-hooks, > guix-build-coordinator-configuration-guile, > make-guix-build-coordinator-start-script, > guix-build-coordinator-shepherd-services, guix-build-coordinator-activati= on, > guix-build-coordinator-account): New procedures. > (guix-build-coordinator-service-type): New variable. > * gnu/tests/guix.scm (%test-guix-build-coordinator): New variable. > * doc/guix.texi (Guix Services): Document it. Yay! > +@subsubheading Guix Build Coordinator > +The @uref{https://git.cbaines.net/guix/build-coordinator/,Guix Build > +Coordinator} aids in building derivations. The Guix Daemon is still ^ Maybe something like: =E2=80=9Cin distributing derivation builds among mach= ines running an @dfn{agent}=E2=80=9D. Also, s/Guix Daemon/build daemon/ or similar. (In general I=E2=80=99m in favor of avoiding =E2=80=9Cbrands=E2=80=9D in do= cumentation and concept names.) Maybe add a =E2=80=9C@quotation Note=E2=80=9D stating that it=E2=80=99s wor= k in progress and subject to change. > +(define* (make-guix-build-coordinator-start-script database-uri-string > + allocation-strategy > + pid-file > + guix-build-coordinato= r-package > + #:key > + agent-communication-u= ri-string > + client-communication-= uri-string > + (hooks '()) > + (guile guile-3.0)) > + (program-file > + "start-guix-build-coordinator" > + (with-extensions (cons guix-build-coordinator-package > + ;; This is a poorly constructed Guile load pat= h, > + ;; since it contains things that aren't Guile > + ;; libraries, but it means that the Guile libr= aries > + ;; needed for the Guix Build Coordinator don't= need > + ;; to be individually specified here. > + (map second (package-inputs > + guix-build-coordinator-package))) Perhaps there should eventually be a =E2=80=98guix-build-coordinator=E2=80= =99 command in the package itself? Otherwise LGTM, thanks! One thing we discussed on IRC is the name and ways to set up one=E2=80=99s monitor so that guix-build-coordinator-configuration-client-communication-uri-string fits on one line. :-) An idea that came to mind was =E2=80=9C(Guix) dispatch(er)=E2=80=9D, which = is synonymous. You could use the (guix dispatch =E2=80=A6) name space. Food for thought! (No rush, of course!) Ludo=E2=80=99. From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 3/4] services: guix: Add guix-build-coordinator-agent-service-type. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Sep 2020 20:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Christopher Baines Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160046071822813 (code B ref 43494); Fri, 18 Sep 2020 20:26:01 +0000 Received: (at 43494) by debbugs.gnu.org; 18 Sep 2020 20:25:18 +0000 Received: from localhost ([127.0.0.1]:44534 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJMwX-0005vr-UD for submit@debbugs.gnu.org; Fri, 18 Sep 2020 16:25:18 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37292) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJMwW-0005vf-Lq for 43494@debbugs.gnu.org; Fri, 18 Sep 2020 16:25:16 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:56572) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kJMwR-0003Qf-9E; Fri, 18 Sep 2020 16:25:11 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=45404 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kJMwM-00081U-HI; Fri, 18 Sep 2020 16:25:07 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20200918184042.22660-1-mail@cbaines.net> <20200918184042.22660-3-mail@cbaines.net> Date: Fri, 18 Sep 2020 22:25:05 +0200 In-Reply-To: <20200918184042.22660-3-mail@cbaines.net> (Christopher Baines's message of "Fri, 18 Sep 2020 19:40:41 +0100") Message-ID: <87363evnmm.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Christopher Baines skribis: > * gnu/services/guix.scm (): N= ew > record type. > (guix-build-coordinator-agent-configuration, > guix-build-coordinator-agent-configuration?, > guix-build-coordinator-agent-configuration-package, > guix-build-coordinator-agent-configuration-user, > guix-build-coordinator-agent-configuration-coordinator, > guix-build-coordinator-agent-configuration-uuid), > guix-build-coordinator-agent-configuration-password, > guix-build-coordinator-agent-configuration-password-file, > guix-build-coordinator-agent-configuration-systems, > guix-build-coordinator-agent-configuration-max-parallel-builds, > guix-build-coordinator-agent-configuration-derivation-substitute-urls, > guix-build-coordinator-agent-configuration-non-derivation-substitute-urls, > guix-build-coordinator-agent-shepherd-services, > guix-build-coordinator-agent-activation, > guix-build-coordinator-agent-account): New procedures. > (guix-build-coordinator-agent-service-type): New variable. > * doc/guix.texi (Guix Services): Document it. [...] > +@defvar {Scheme Variable} guix-build-coordinator-agent-service-type > +Service type for a Guix Build Coordinator agent. Its value must be a Perhaps =E2=80=9Ccoordinator=E2=80=9D and =E2=80=9Cagent=E2=80=9D should be= defined in a few sentences above to clarify what this is about. > +@item @code{derivation-substitute-urls} (default: @code{1}) > +URLs from which to attempt to fetch substitutes for derivations, if the > +derivations aren't already available. > + > +@item @code{non-derivation-substitute-urls} (default: @code{1}) > +URLs from which to attempt to fetch substitutes for build inputs, if the > +input store items aren't already available. This is interesting, I wonder how you can distinguish between the two in code. You have to open different sessions, right? Or to call =E2=80=98set-build-options=E2=80=99 again? > + (description > + "Run an instance of the Guix Build Coordinator."))) + =E2=80=9Cagent=E2=80=9D Otherwise LGTM! Ludo=E2=80=99. From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 4/4] services: guix: Add guix-build-coordinator-queue-builds-service-type. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Sep 2020 20:32:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Christopher Baines Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160046108323491 (code B ref 43494); Fri, 18 Sep 2020 20:32:02 +0000 Received: (at 43494) by debbugs.gnu.org; 18 Sep 2020 20:31:23 +0000 Received: from localhost ([127.0.0.1]:44551 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJN2R-00066p-6w for submit@debbugs.gnu.org; Fri, 18 Sep 2020 16:31:23 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38366) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJN2P-00066d-V2 for 43494@debbugs.gnu.org; Fri, 18 Sep 2020 16:31:22 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:56643) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kJN2K-0004Eb-Kq; Fri, 18 Sep 2020 16:31:16 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=45406 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kJN2J-0002vg-Sz; Fri, 18 Sep 2020 16:31:16 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20200918184042.22660-1-mail@cbaines.net> <20200918184042.22660-4-mail@cbaines.net> Date: Fri, 18 Sep 2020 22:31:14 +0200 In-Reply-To: <20200918184042.22660-4-mail@cbaines.net> (Christopher Baines's message of "Fri, 18 Sep 2020 19:40:42 +0100") Message-ID: <87pn6iu8rx.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Christopher Baines skribis: > * gnu/services/guix.scm (): > New record type. > (guix-build-coordinator-queue-builds-configuration, > guix-build-coordinator-queue-builds-configuration?, > guix-build-coordinator-queue-builds-configuration-package, > guix-build-coordinator-queue-builds-configuration-user, > guix-build-coordinator-queue-builds-coordinator, > guix-build-coordinator-queue-builds-configuration-systems, > guix-build-coordinator-queue-builds-configuration-system-and-targets, > guix-build-coordinator-queue-builds-configuration-guix-data-service, > guix-build-coordinator-queue-builds-configuration-processed-commits-file, > guix-build-coordinator-queue-builds-shepherd-services, > guix-build-coordinator-queue-builds-activation, > guix-build-coordinator-queue-builds-account): New procedures. > (guix-build-coordinator-queue-builds-service-type): New variable. [...] > +@defvar {Scheme Variable} guix-build-coordinator-queue-builds-service-ty= pe > +Service type for the > +guix-build-coordinator-queue-builds-from-guix-data-service script. Its Oh! :-) =E2=80=98guix-build-queue=E2=80=99 maybe? Because after all, it can queue = builds coming from anywhere, not just from the Data Service, right? It=E2=80=99d be nice to add a sentence or two explaining how the =E2=80=9Cq= ueue=E2=80=9D fits in the big picture (I=E2=80=99m not quite sure :-)). > +@item @code{systems-and-targets} (default: @code{#f}) > +An association list of system and target pairs for which to fetch > +derivations to build. Cross-compilation target triplets are a notion that=E2=80=99s not available= at the derivation level; it only exists for packages. Are we mixing things here? > +@item @code{guix-data-service} (default: @code{"https://data.guix.gnu.or= g"}) > +The Guix Data Service instance from which to query to find out about > +derivations to build. Ah so the queue pulls from the Data Service. Got it! > +@item @code{processed-commits-file} (default: @code{"/var/lib/guix-build= -coordinator-queue-builds/processed-commits"}) > +A file to record which commits have been processed, to avoid needlessly > +processing them again if the service is restarted. Maybe in /var/cache by default, no? > + (processed-commits-file > + guix-build-coordinator-queue-builds-configuration-processed-commits-f= ile > + (default "/var/lib/guix-build-coordinator-queue-builds/processed-comm= its"))) Maybe =E2=80=9Cstate-file=E2=80=9D? In addition to saving space :-), it=E2= =80=99d leave room for additional bits of state. That=E2=80=99s all, thank you! Ludo=E2=80=99. From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 1/4] gnu: Add guix-build-coordinator. Resent-From: Jonathan Brielmaier Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Sep 2020 23:36:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.16004721218669 (code B ref 43494); Fri, 18 Sep 2020 23:36:01 +0000 Received: (at 43494) by debbugs.gnu.org; 18 Sep 2020 23:35:21 +0000 Received: from localhost ([127.0.0.1]:45348 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJPuT-0002Fl-0y for submit@debbugs.gnu.org; Fri, 18 Sep 2020 19:35:21 -0400 Received: from mout.web.de ([212.227.15.14]:38459) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJPuQ-0002FV-6V for 43494@debbugs.gnu.org; Fri, 18 Sep 2020 19:35:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1600472111; bh=9LoAQ2SjyJxEmXyuq1UqEymANLnKC23r+cknb1HoIJk=; h=X-UI-Sender-Class:Subject:To:Cc:References:From:Date:In-Reply-To; b=JgNW3ypcIs18gUG54Wq3iLU8PqHemfF1INrJvoeXsRFZLb44pkYzoun0F6+7Df4oW 16KkMH4t1/Ts+zEDeHNxRQ2zLq98xz2CkbX5TV6eIrn9AzbFXPcLlJBFRd4i4koKEF ZYWYaYPDMC7qK8qPNhTdBOlNl0QU7LS0q7dxQNpE= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from [192.168.178.25] ([88.152.185.253]) by smtp.web.de (mrweb003 [213.165.67.108]) with ESMTPSA (Nemesis) id 0M8zRF-1kBTwQ2aFB-00CSaw; Sat, 19 Sep 2020 01:35:11 +0200 References: <87o8m39bnc.fsf@cbaines.net> <20200918184042.22660-1-mail@cbaines.net> <87lfh6vodj.fsf@gnu.org> From: Jonathan Brielmaier Message-ID: <4332de6e-8002-1967-4838-3448df888242@web.de> Date: Sat, 19 Sep 2020 01:34:57 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Icedove/68.11.0 MIME-Version: 1.0 In-Reply-To: <87lfh6vodj.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Language: fr Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:hKuWGkNiMYXkIicv2UtTtFNtS09V+GMqHoQb3UzhiLcRC9rekLJ YCTkGRvKBOGvTxjkIWaYNwtjnBD5YFqfzOELaSyGJVlG/mXXSQAhT6QaYUMwPMdUSZU8eIl 8LSbjOl0K4GRcqeu2G6U9i85ZL+W5sRYozTSwbXBvJbMtf7AjMCU9GbqbWIKWYzZwLQYuIR J2uvuvOfLub9KP93vz/RA== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:Ch3BN8OkIyY=:pFCdx6aK+bBytNUzw9Ll5s nPmcZSW3OXkGwxhz/d0qhxgKcZToOdNdE6YBLWhzif2sxkXixjHo6qw3aBgfgK8M8o/PwDhEa 47Ejv2bqDIxmH8DELpN4fuQc4Do3K3OSX9liFgeFlSGpW4PwnE1ka0kbxotFmtKVeb7Vb6BXb PiYlnOLVVFDIgp2EE5gLfLxeXina9wcTX5SO2fN7pKDa2LT9yQ+BHCy5KoFZjiFMOPrhphpVb 1eC60t4DQ1lgLmyTkCZQHgBNrkfdAPjhnNxq5/DvT+sYLTFo5woybE4FU0J1m/+Gn7tLuvjJl tpc+kFrfvOHZTzg1+Pubtyez/qq6xDoAPqFS+FHqKxPK8GJ/lVrsEOd15OMfHdizJ3TcR6Qct H71dTdR+/3FnOiiXE6dDP5c+PQnCLIRiGqyG+as01HlnC2bQPYNgHt8JKGRDksNnh9WdIEPFW VIUpchBxWm4G5Ws+UqaVx8NgtN9C93a46WNutIZPaO6m/G+6+EqP5+05kOVq2dKUDUKlSaaWz 9ko0Pkm5ijBtaVDYgg1igCQa2Sej8WSFxgRV9y8CZDwa+s4bDBupRkXe3D4CFqAppcrFwArz5 VYO3Zwi1q6r2k4C2s602wE3YC1TTJ3QHKDMsJbYe2LHblwNCZDBfKpMoCHdyKlJhDHqC3LGph jQEA8Ps/SpPCh3TI9sw+Xwq2oyN40dPKFqt9D9WftzDIeLrl64C19JOoueEP3+LzJDzIjAiyO kHy39S7nA5kDNYprKrrltrsj73cWqaewiFxJB5u2RBg0zV1bpE/K5sKbz1Ay70wNvsu7quh1I 5rYiC7I3n2lOxC9PbhcsfoulE11pcacgXFMH4JlxTwoYwnXveSsmdkq/z3hITHh7HeiVhcRS0 SQo8/RGIpcsVSrJjaSYg3oDqlgkz6nzKvIacc/69Csj1ZFkwOsy03zl0PGhobcMcdTX6VKdSf 8MlszGTF96ccq4vywdJIx0qTXwf1NVTxBgixfBU/AS8n+/u+/HjbdCX1pAx/pFPY+tHUeqpBT +PXU6nzX8ni0WAbcS31qEfYENlWJ6uLCpAsU2tScgJyUBGEs75ZMWmgqEnj8dp9jWmEseBvdh L+A/FjVyTCJ6Wtir0BbHvS7PV4Buizn5kFckMCdJS0L4f/BhR99SrJ88AvxCA4HwPJTGFH6dN TX9YKtTOC0u9zLSIcfudaQbfeCO9hxEWNKSUPZ1YECCobe2d2RCs36nAEUj9KEkBA0DX7wUNJ vEHyDS5RAG/K4Mo+DuNeUWYE7ij6CYu92GILDfA== X-Spam-Score: -2.6 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.6 (---) On 18.09.20 22:08, Ludovic Court=C3=A8s wrote: >> + (home-page "https://git.cbaines.net/guile/guix/build-coordinator") It should be https://git.cbaines.net/guix/build-coordinator/ From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 1/4] gnu: Add guix-build-coordinator. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 19 Sep 2020 09:02:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Jonathan Brielmaier Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.16005060638156 (code B ref 43494); Sat, 19 Sep 2020 09:02:01 +0000 Received: (at 43494) by debbugs.gnu.org; 19 Sep 2020 09:01:03 +0000 Received: from localhost ([127.0.0.1]:45631 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJYju-00027C-Vh for submit@debbugs.gnu.org; Sat, 19 Sep 2020 05:01:03 -0400 Received: from mira.cbaines.net ([212.71.252.8]:57718) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJYjt-00024l-LS for 43494@debbugs.gnu.org; Sat, 19 Sep 2020 05:01:02 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id 654D827BBE7; Sat, 19 Sep 2020 10:01:00 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id dfb594e7; Sat, 19 Sep 2020 09:00:57 +0000 (UTC) References: <87o8m39bnc.fsf@cbaines.net> <20200918184042.22660-1-mail@cbaines.net> <87lfh6vodj.fsf@gnu.org> <4332de6e-8002-1967-4838-3448df888242@web.de> User-agent: mu4e 1.4.13; emacs 26.3 From: Christopher Baines In-reply-to: <4332de6e-8002-1967-4838-3448df888242@web.de> Date: Sat, 19 Sep 2020 10:00:54 +0100 Message-ID: <87ft7e9m49.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Jonathan Brielmaier writes: > On 18.09.20 22:08, Ludovic Court=C3=A8s wrote: >>> + (home-page "https://git.cbaines.net/guile/guix/build-coordinator") > > It should be https://git.cbaines.net/guix/build-coordinator/ Thanks Jonathan, I've fixed this now :) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl9lyMZfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XcELBAApxwnjfJu/B3H9p8cXiZCTl99ejaJXvKfT8Utx4aB4YpFg+ZR7KqlqiJ0 dUr1hXpMzRX31IoUrcIk2mc2DUw5GQjM2lq3mUxESsW8A7ODEU8WfW5KcwLfes4g 7M+jZlPMEJYpfzV2ryvFyBrWErv76u/9d4vcAXIvjIznhEPZbrMyEaW5T0CMUCdz ULZuBB6PuN7NlV6atYh0Jy7FL83LY5hN+EX0T8ecJ+3pOrlZQWcTJitRsaoCtbvn nYi3pSQjhuERfb6c4n+lK3NpX4StyMJWW86hF65CsYZhpVsG5xT/kI2R3xf2Ux/h H4oVrhVxzLIRKPHZEFu2+48O6Pno2QkPam+VYl2Jc6pM5qJuRWrASbXPCu5lhleP ZLoZPtVqkqIdAZDc3ItUW47hzihpEzsn2a7/rw5c6yh3L53q/edF8jFDYFYF2/xa j46BG2UKYFpFSvnwUcbJJvNiVpZECcSzFpOYN1kb4z21TuuILL/BTzdEyUcPlH/Q jVz1lP4ElGAVJPnOltTQYW8syElS0BCc0yhz9KOGeFGJmn3u5TL32ViBJ29p9q2l R2OitOhLwRZxJwFm64i7+smjkps3DxYEaDkZJnHdA5bGYlk4ojxk7MsDFD42NYh6 kRoBQF7l1kjHXI9PGxFRMI8T9Gri1HZESEo4bVCTnS6BW6TqPc4= =+xCv -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 1/4] gnu: Add guix-build-coordinator. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 19 Sep 2020 09:06:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160050631113216 (code B ref 43494); Sat, 19 Sep 2020 09:06:01 +0000 Received: (at 43494) by debbugs.gnu.org; 19 Sep 2020 09:05:11 +0000 Received: from localhost ([127.0.0.1]:45635 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJYnv-0003R4-Iq for submit@debbugs.gnu.org; Sat, 19 Sep 2020 05:05:11 -0400 Received: from mira.cbaines.net ([212.71.252.8]:57726) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJYnt-0003Qx-O4 for 43494@debbugs.gnu.org; Sat, 19 Sep 2020 05:05:10 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id 4518E27BBE7; Sat, 19 Sep 2020 10:05:09 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 68cd3d6e; Sat, 19 Sep 2020 09:05:07 +0000 (UTC) References: <87o8m39bnc.fsf@cbaines.net> <20200918184042.22660-1-mail@cbaines.net> <87lfh6vodj.fsf@gnu.org> User-agent: mu4e 1.4.13; emacs 26.3 From: Christopher Baines In-reply-to: <87lfh6vodj.fsf@gnu.org> Date: Sat, 19 Sep 2020 10:05:06 +0100 Message-ID: <87d02i9lx9.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Hi! > > Christopher Baines skribis: > >> * gnu/packages/package-management.scm (guix-build-coordinator): New vari= able. > > [...] > >> + (lambda* (#:key inputs outputs #:allow-other-keys) >> + (let* ((out (assoc-ref outputs "out")) >> + (bin (string-append out "/bin")) >> + (guile (assoc-ref inputs "guile")) >> + (guile-effective-version >> + (read-line >> + (open-pipe* OPEN_READ >> + (string-append guile "/bin/guile") >> + "-c" "(display (effective-version))")= )) > > Maybe use =E2=80=98target-guile-effective-version=E2=80=99 from (guix bui= ld > guile-build-system). I've changed this. I'll also try and remember and update the guix-data-service package accordingly, as that's where I copied this from. >> + (wrap-program (string-append bin "/" file) >> + `("PATH" ":" prefix >> + (,bin ,(assoc-ref inputs "sqitch"))) >> + `("PERL5LIB" ":" prefix >> + (,(getenv "PERL5LIB"))) > > Do we really need PERL5LIB here? Shouldn=E2=80=99t it be done in the wra= pper of > =E2=80=98sqitch=E2=80=99 directly? Hmm, when I packaged sqitch, I didn't add all the database drivers which are optional dependencies, but perl-dbd-pg did end up in there somehow (probably because that's needed by the guix-data-service). I've now gone ahead and added the database drivers which Guix has packages for [1], it does increase the closure size a bit, but it was quite big anyway. 1: http://git.savannah.gnu.org/cgit/guix.git/commit/?id=3D6a1c3a908c8aadfde5b5= fe405132839683a01776 All that means I can remove the wrapping here. >> + (scandir bin >> + (match-lambda >> + ((or "." "..") #f) >> + (_ #t)))) > > I think you could just use (find-files bin). Done. >> + (inputs >> + `(("guile" ,guile-3.0-latest) > > Rather ,@(assoc-ref (package-native-inputs guix) "guile"). Done. >> + ("sqitch" ,sqitch) >> + ("perl-dbd-sqlite" ,perl-dbd-sqlite))) > > Shouldn=E2=80=99t perl-dbd-sqlite be a dependency of sqitch? See above. >> + (home-page "https://git.cbaines.net/guile/guix/build-coordinator") >> + (synopsis "") >> + (description >> + "") > > Missing! :-) Ah, yeah, I should have remembered to lint the package. I've attempted to write something now, I'm still working on trying to explain what the Guix Build Coordinator is meant to be. > LGTM with changes along these lines! Thanks! --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl9lycJfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XdAdQ/8CgG11hrFpUwT8jxkToEexip5q3Gxt9MwtUK7nEmN6YmiRKO+5aEiWbGl rbLvKuxgtg673rb5+Th3LTUcprxlgh3SGFIXINHHqy3ybzyGm2vOFBDlZ1I8rbsS Hr195bVVHsirWQJzqPk0c4ukfaxEKc9V/QFa+VllJs/LlTobbuo8oqwkQ5wC6wUJ O+ZTJrXZpjDclFXGNGDEGGXJ2KnhxG7CMLt/SrgTrMy/0tpLyHsPYP1OXOOe7YQC dDXAx5zN3EkNSOCGWygi2QGiocRunVKJzhAWJr5WE62Vzn9/XxbuMxECs30kQuza AVKHA5q5hCL2kAFm80goM8nmdYzxZ/tMENpgySCb5FF1pcGQVpLjlsodao4AHPl9 c2x3cgr4DB8wiNx26O5ENEuR1vfInwlQVQz8zvd9tseD8BRh4bk+3f28ZXWs/VDz ZGCQ67Vp83+3poAV12+ScHvFu9MNX3jwICEHtISNnhtYlw8GS+icOpe1pmFCf4D6 4kxyXmj595OcZjv2xfVUso32XccTmTl9+c52Djj+SLlu91mJ20jJt/flxvv7uK6u T5SwfpHX+RThnJMFHjjwoFPV7CsyuTb9sfKDjgrf7hMepHYuX7jGqDjjrMMoXvco 4/0YlIIKP+gR3OvjTyg+YZVP/Fulbjm7xkgkHNZPxS6bXQJrKho= =hrmM -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 2/4] services: guix: Add guix-build-coordinator-service-type. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 19 Sep 2020 09:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160050684714100 (code B ref 43494); Sat, 19 Sep 2020 09:15:02 +0000 Received: (at 43494) by debbugs.gnu.org; 19 Sep 2020 09:14:07 +0000 Received: from localhost ([127.0.0.1]:45650 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJYwZ-0003fL-4f for submit@debbugs.gnu.org; Sat, 19 Sep 2020 05:14:07 -0400 Received: from mira.cbaines.net ([212.71.252.8]:57734) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJYwX-0003fE-Ne for 43494@debbugs.gnu.org; Sat, 19 Sep 2020 05:14:06 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id 1430727BBE7; Sat, 19 Sep 2020 10:14:05 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id ffee01bd; Sat, 19 Sep 2020 09:14:02 +0000 (UTC) References: <20200918184042.22660-1-mail@cbaines.net> <20200918184042.22660-2-mail@cbaines.net> <87a6xmvnth.fsf@gnu.org> User-agent: mu4e 1.4.13; emacs 26.3 From: Christopher Baines In-reply-to: <87a6xmvnth.fsf@gnu.org> Date: Sat, 19 Sep 2020 10:13:58 +0100 Message-ID: <87a6xm9lih.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Christopher Baines skribis: > >> * gnu/services/guix.scm (): New re= cord >> type. >> (guix-build-coordinator-configuration, guix-build-coordinator-configurat= ion?, >> guix-build-coordinator-configuration-package, >> guix-build-coordinator-configuration-user, >> guix-build-coordinator-configuration-group, >> guix-build-coordinator-configuration-datastore-uri-string, >> guix-build-coordinator-configuration-agent-communication-uri-string, >> guix-build-coordinator-configuration-client-communication-uri-string, >> guix-build-coordinator-configuration-allocation-strategy, >> guix-build-coordinator-configuration-hooks, >> guix-build-coordinator-configuration-guile, >> make-guix-build-coordinator-start-script, >> guix-build-coordinator-shepherd-services, guix-build-coordinator-activat= ion, >> guix-build-coordinator-account): New procedures. >> (guix-build-coordinator-service-type): New variable. >> * gnu/tests/guix.scm (%test-guix-build-coordinator): New variable. >> * doc/guix.texi (Guix Services): Document it. > > Yay! > >> +@subsubheading Guix Build Coordinator >> +The @uref{https://git.cbaines.net/guix/build-coordinator/,Guix Build >> +Coordinator} aids in building derivations. The Guix Daemon is still > ^ > Maybe something like: =E2=80=9Cin distributing derivation builds among ma= chines > running an @dfn{agent}=E2=80=9D. > > Also, s/Guix Daemon/build daemon/ or similar. > > (In general I=E2=80=99m in favor of avoiding =E2=80=9Cbrands=E2=80=9D in = documentation and > concept names.) > > Maybe add a =E2=80=9C@quotation Note=E2=80=9D stating that it=E2=80=99s w= ork in progress and > subject to change. I've made these changes now. >> +(define* (make-guix-build-coordinator-start-script database-uri-string >> + allocation-strategy >> + pid-file >> + guix-build-coordinat= or-package >> + #:key >> + agent-communication-= uri-string >> + client-communication= -uri-string >> + (hooks '()) >> + (guile guile-3.0)) >> + (program-file >> + "start-guix-build-coordinator" >> + (with-extensions (cons guix-build-coordinator-package >> + ;; This is a poorly constructed Guile load pa= th, >> + ;; since it contains things that aren't Guile >> + ;; libraries, but it means that the Guile lib= raries >> + ;; needed for the Guix Build Coordinator don'= t need >> + ;; to be individually specified here. >> + (map second (package-inputs >> + guix-build-coordinator-package))) > > Perhaps there should eventually be a =E2=80=98guix-build-coordinator=E2= =80=99 command in > the package itself? There actually is, one thing I've had in mind for a while now though is to use a scheme script constructed by the Guix service to run the coordinator. For guix.cbaines.net, I'm using the script, but with the hooks passed in on the command line, the command is rather long, and it means that backtraces don't work well with the hooks. I'm unsure how well this has worked out, I didn't anticipate the issues with the Guile load path, and because of the formatting stripping, both the code and backtraces will be unreadable anyway... but it is useful in allowing G-expressions to be used as part of the configuration. > One thing we discussed on IRC is the name and ways to set up one=E2=80=99s > monitor so that > guix-build-coordinator-configuration-client-communication-uri-string > fits on one line. :-) > > An idea that came to mind was =E2=80=9C(Guix) dispatch(er)=E2=80=9D, whic= h is > synonymous. You could use the (guix dispatch =E2=80=A6) name space. > Food for thought! (No rush, of course!) Indeed, I think that's a strong candidate :) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl9ly9ZfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XdVWg/+KZ3LZDolesoD7M43Bg2HBcI4SB2nou7+rT5iiYXgcX17kZjX/PG6SWbS DgdfJ+su/n59Hc9galUyZTLkbr52Yl8gByj35CQmWJJ0SJPzzvEdml6Ng7/KPOMr LN1YHvFxVfhXn+tZbExxV98LBGqWIualpoUd2I7qJvrA+11QAUXyLOybkhCk/lJz rrJRCjKoXP4mjxVBj6EgFB1s6CNXPI97kmJJvEHEp1V4kC8rym3kOkfYiuaZAmnZ aZil9CslNLOtFBhiheXaZWgGzvzx7ifFe/7oV63zfghg/AzoPwHVjvIdRrly7b1t eFJ5O3E3OOawXMAMV/K1+rPx/eTHaVyufex27wZcZpEv1UkX/7oMv6/XSVp0gOOD UpmDCNwSAmrvApE+rh1LowGIDBHYBa+P03RZ+4DYH68jT87C/O3Vz9WbDDTbPow9 KanFLJbR80sYaD/xO5Ni8cZ+o6DmxrjLPv55EdLqds1TQ2Bp7x2AG4Ki3VoMrFol H94YCS99hck5gXghZJZpsrUaclSDBUDET541OrBFtctfG8WksstGovRMnWO4u2wO 3JjQJTTdfr4yAqOVWUi3PE1JdZB1aZENE4XgZeNPBZCKVNt5noAV0OH/SN66noDk od5DslxrE/KUmgagTiEWOAl6noCmIRH+jSBrgqUJrGMSL93HwTM= =tSTq -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 3/4] services: guix: Add guix-build-coordinator-agent-service-type. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 19 Sep 2020 09:50:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160050895025590 (code B ref 43494); Sat, 19 Sep 2020 09:50:01 +0000 Received: (at 43494) by debbugs.gnu.org; 19 Sep 2020 09:49:10 +0000 Received: from localhost ([127.0.0.1]:45668 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZUT-0006eg-Oo for submit@debbugs.gnu.org; Sat, 19 Sep 2020 05:49:10 -0400 Received: from mira.cbaines.net ([212.71.252.8]:57748) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZUS-0006eY-H6 for 43494@debbugs.gnu.org; Sat, 19 Sep 2020 05:49:09 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id 68B8927BBE7; Sat, 19 Sep 2020 10:49:07 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 35f96f3f; Sat, 19 Sep 2020 09:49:04 +0000 (UTC) References: <20200918184042.22660-1-mail@cbaines.net> <20200918184042.22660-3-mail@cbaines.net> <87363evnmm.fsf@gnu.org> User-agent: mu4e 1.4.13; emacs 26.3 From: Christopher Baines In-reply-to: <87363evnmm.fsf@gnu.org> Date: Sat, 19 Sep 2020 10:49:04 +0100 Message-ID: <875z8a9jvz.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Christopher Baines skribis: > >> * gnu/services/guix.scm (): = New >> record type. >> (guix-build-coordinator-agent-configuration, >> guix-build-coordinator-agent-configuration?, >> guix-build-coordinator-agent-configuration-package, >> guix-build-coordinator-agent-configuration-user, >> guix-build-coordinator-agent-configuration-coordinator, >> guix-build-coordinator-agent-configuration-uuid), >> guix-build-coordinator-agent-configuration-password, >> guix-build-coordinator-agent-configuration-password-file, >> guix-build-coordinator-agent-configuration-systems, >> guix-build-coordinator-agent-configuration-max-parallel-builds, >> guix-build-coordinator-agent-configuration-derivation-substitute-urls, >> guix-build-coordinator-agent-configuration-non-derivation-substitute-url= s, >> guix-build-coordinator-agent-shepherd-services, >> guix-build-coordinator-agent-activation, >> guix-build-coordinator-agent-account): New procedures. >> (guix-build-coordinator-agent-service-type): New variable. >> * doc/guix.texi (Guix Services): Document it. > > [...] > >> +@defvar {Scheme Variable} guix-build-coordinator-agent-service-type >> +Service type for a Guix Build Coordinator agent. Its value must be a > > Perhaps =E2=80=9Ccoordinator=E2=80=9D and =E2=80=9Cagent=E2=80=9D should = be defined in a few sentences > above to clarify what this is about. I've added a bit more information, I want to at some point actually give a workable minimal example configuration, but I need to do some more thinking about that. >> +@item @code{derivation-substitute-urls} (default: @code{1}) >> +URLs from which to attempt to fetch substitutes for derivations, if the >> +derivations aren't already available. >> + >> +@item @code{non-derivation-substitute-urls} (default: @code{1}) >> +URLs from which to attempt to fetch substitutes for build inputs, if the >> +input store items aren't already available. > > This is interesting, I wonder how you can distinguish between the two in > code. You have to open different sessions, right? Or to call > =E2=80=98set-build-options=E2=80=99 again? Yeah, it's not strictly necessary to have some separation. In fact, I can't remember exactly, but I think it should be possible to deploy the coordinator and agents without explicit configuration of the substitute URLs, but I need to check that. These options were added mostly because it useful if you have two seperate sources of substitutes, one for just derivations (say an instance of the Guix Data Service), and one just for build outputs (say populated by one of the hooks) [1]. 1: https://git.cbaines.net/guix/build-coordinator/commit/?id=3Dbda7d58853ed= 4fba976cac92a70c4dc68db263aa >> + (description >> + "Run an instance of the Guix Build Coordinator."))) > > + =E2=80=9Cagent=E2=80=9D > > Otherwise LGTM! Thanks! --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl9l1BBfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XdqNA/+PAeN8zL9P608PFjf1IDPZZWx76WSAMZ6BfCStsRvGNR/X+ZOgzabH6+K ZiJoAbvC0RIzFgGh90CfYI2GabG6SYTkQMAi4DN/Cs3iKFVG+DDvdpprmk9xDTFb k+MnarQzw0sSidtZb8OL9E9u+c3evfDO6AMl9y2vKhfGDV6A5IvIcNJy1t8TM8OL S/qcd8O2kIlLX2LBKoGJuhiszXt1MFYrXmbSLDEd+iZ0wcFXHaG2TV/y5UbqydP3 0MaCKAym2YN5NNK98u+z94TImZeppCeQGGY5OnN0Cq29TaIn8egDX1NJjFkV4OzT +WwM2jY14oFvOZHblS5uqd0gd3I5yi/NX41K6CFjAOX43ADlpQSeLF6CtPZAJbWS wNn+3pmBdtyuRJibe0GbtaAn5+0CvJ1L58gHzXIp22Vaz+yW5qoAVEOMuEx0a6ft uMJ6RzjW9FGtvNzuyzcvCbeMmB5Wc3IuoTiPA6ZHBuj1KvdDBwD32zskPwUKk3Cu 1G7KBkHmAMOF28Z/Ci0Lr1vewKW6iR+6pIiXD6gxRiaGLZaz/8Ck3eweYa+llidU YBkz/UzGMZggWegnABCMv+6bTRgDTwVFTH9QNlBmx1gJRYayx8+xclGNR/njcHH5 VR93AvkYiHmXPnpEGeXncGJwN8jTFoW4UP5Mb1BPT1geWj8xmzQ= =RwqV -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 4/4] services: guix: Add guix-build-coordinator-queue-builds-service-type. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 19 Sep 2020 10:07:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160050996727137 (code B ref 43494); Sat, 19 Sep 2020 10:07:01 +0000 Received: (at 43494) by debbugs.gnu.org; 19 Sep 2020 10:06:07 +0000 Received: from localhost ([127.0.0.1]:45687 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZkt-00073d-7o for submit@debbugs.gnu.org; Sat, 19 Sep 2020 06:06:07 -0400 Received: from mira.cbaines.net ([212.71.252.8]:57754) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZkr-00073V-8G for 43494@debbugs.gnu.org; Sat, 19 Sep 2020 06:06:05 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id 492B927BBE7; Sat, 19 Sep 2020 11:06:04 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 1e7a9b66; Sat, 19 Sep 2020 10:06:01 +0000 (UTC) References: <20200918184042.22660-1-mail@cbaines.net> <20200918184042.22660-4-mail@cbaines.net> <87pn6iu8rx.fsf@gnu.org> User-agent: mu4e 1.4.13; emacs 26.3 From: Christopher Baines In-reply-to: <87pn6iu8rx.fsf@gnu.org> Date: Sat, 19 Sep 2020 11:05:57 +0100 Message-ID: <87363e9j3u.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Christopher Baines skribis: > >> * gnu/services/guix.scm (): >> New record type. >> (guix-build-coordinator-queue-builds-configuration, >> guix-build-coordinator-queue-builds-configuration?, >> guix-build-coordinator-queue-builds-configuration-package, >> guix-build-coordinator-queue-builds-configuration-user, >> guix-build-coordinator-queue-builds-coordinator, >> guix-build-coordinator-queue-builds-configuration-systems, >> guix-build-coordinator-queue-builds-configuration-system-and-targets, >> guix-build-coordinator-queue-builds-configuration-guix-data-service, >> guix-build-coordinator-queue-builds-configuration-processed-commits-file, >> guix-build-coordinator-queue-builds-shepherd-services, >> guix-build-coordinator-queue-builds-activation, >> guix-build-coordinator-queue-builds-account): New procedures. >> (guix-build-coordinator-queue-builds-service-type): New variable. > > [...] > >> +@defvar {Scheme Variable} guix-build-coordinator-queue-builds-service-t= ype >> +Service type for the >> +guix-build-coordinator-queue-builds-from-guix-data-service script. Its > > Oh! :-) > > =E2=80=98guix-build-queue=E2=80=99 maybe? Because after all, it can queu= e builds coming > from anywhere, not just from the Data Service, right? > > It=E2=80=99d be nice to add a sentence or two explaining how the =E2=80= =9Cqueue=E2=80=9D fits in > the big picture (I=E2=80=99m not quite sure :-)). So, this got me thinking a bit, maybe this script should actually exist outside of the guix-build-coordinator repository... You can run `guix-build-coordinator build ...` to build a derivation, and what this does is just make the relevant HTTP request to the coordinator. What this script does is just fetch derivation names from an instance of the Guix Data Service, and ask the Guix Build Coordinator to build them. >> +@item @code{systems-and-targets} (default: @code{#f}) >> +An association list of system and target pairs for which to fetch >> +derivations to build. > > Cross-compilation target triplets are a notion that=E2=80=99s not availab= le at > the derivation level; it only exists for packages. > > Are we mixing things here? This setting is basically configuing the query parameters to use for this page for example [1]. You're right that it's not available at the derivation level, but you can ask the Guix Data Service for derivations for packages which were computed for a specific system and target. 1: http://data.guix.gnu.org/revision/70ef8b24550c54cc8e9f20026bfd24b8680499= b4/package-derivations >> +@item @code{guix-data-service} (default: @code{"https://data.guix.gnu.o= rg"}) >> +The Guix Data Service instance from which to query to find out about >> +derivations to build. > > Ah so the queue pulls from the Data Service. Got it! I wouldn't read too much in to "queue" in the name here, "submit" might be a better word to use, since that's the term used on the Guix Build Coordinator side. There's also nothing resemling a queue anywhere... >> +@item @code{processed-commits-file} (default: @code{"/var/lib/guix-buil= d-coordinator-queue-builds/processed-commits"}) >> +A file to record which commits have been processed, to avoid needlessly >> +processing them again if the service is restarted. > > Maybe in /var/cache by default, no? Ah, yeah, I've changed it to be in /var/cache, as that's fine. >> + (processed-commits-file >> + guix-build-coordinator-queue-builds-configuration-processed-commits-= file >> + (default "/var/lib/guix-build-coordinator-queue-builds/processed-com= mits"))) > > Maybe =E2=80=9Cstate-file=E2=80=9D? In addition to saving space :-), it= =E2=80=99d leave room > for additional bits of state. Currently, it's just a file with each commit on its own line. Given that, if there's a need to store more state, it might just end up in different files. If that happens, this setting could become a directory, rather than a specific file to avoid adding extra configuration options. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl9l2AVfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XdCjQ/8DIg3yovjZeLdZteOJnB4JMRyFhx+VTZPbTVEDJcT8BL8gwqiZrVKsiLW Cz+vu62Bwiohk0zrXKLTuqTfsvq/NGnqWm4lt+cz6OiH8guX5CfvuXeLNXWh0t6S PA6O8ieRry/ar+DFQBUEF20XNUIb/hCGhykcdjKC1vLFac4npBaYxRyIwCwiyMLG PrugkmFveh93Xe9E7qqByT6DEPa22zw44zDrzRFWUP3Zbpv+fFuA+MwDg/BjS7lf xGuc0tfI8XVBWwsbnmdMUUNuHneBf5mYRhMDBFqjP+ZrWDq8Z2PHyIUFvie9iaC1 phgwQvjUwXe5w2H5k0rav+izwyjYe2Lg8GgBZsn2j9DV0FlRq3AhM0Ex96wCs6js xbDcAKeuKqaLKwbBoRd40uo+0Aoo5FBo7lVije0M3S4JaQjqGPxqR1HSCPz8VMxo XoKzq36kmZQEly8XLgSf/lWFLkVvEgLvBR4g5bFMVLKxKEK2lHSEa1jfxDc1Lz6V 6/4qVF/bMN8SUqU7ZvRr6klUUbQ2OaTV5Csl5fG1jlZNWDXOTlEKXcgUq+jbU0QE CoM1OED+rppjSXe1zbaubWZKKXk1Vq3Ix7uAIg3G7fwd0rD5FPjTXEdqtAZ4KLOO REYJt6fooebFrPdxqPOGEJQ8IHXBvlOK305U3/GplY2iP70kxrg= =d1v4 -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH v2 1/4] gnu: Add guix-build-coordinator. References: <87o8m39bnc.fsf@cbaines.net> In-Reply-To: <87o8m39bnc.fsf@cbaines.net> Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 19 Sep 2020 10:12:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160051026427608 (code B ref 43494); Sat, 19 Sep 2020 10:12:01 +0000 Received: (at 43494) by debbugs.gnu.org; 19 Sep 2020 10:11:04 +0000 Received: from localhost ([127.0.0.1]:45699 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZpf-0007Ay-6f for submit@debbugs.gnu.org; Sat, 19 Sep 2020 06:11:03 -0400 Received: from mira.cbaines.net ([212.71.252.8]:57764) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZpd-0007Aa-Rp for 43494@debbugs.gnu.org; Sat, 19 Sep 2020 06:11:02 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id 507D927BBE7 for <43494@debbugs.gnu.org>; Sat, 19 Sep 2020 11:11:01 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id e31c660e for <43494@debbugs.gnu.org>; Sat, 19 Sep 2020 10:10:59 +0000 (UTC) From: Christopher Baines Date: Sat, 19 Sep 2020 11:10:56 +0100 Message-Id: <20200919101059.8930-1-mail@cbaines.net> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * gnu/packages/package-management.scm (guix-build-coordinator): New variable. --- gnu/packages/package-management.scm | 74 +++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm index ec87226197..afa45d6789 100644 --- a/gnu/packages/package-management.scm +++ b/gnu/packages/package-management.scm @@ -50,6 +50,7 @@ #:use-module (gnu packages cpio) #:use-module (gnu packages crypto) #:use-module (gnu packages curl) + #:use-module (gnu packages databases) #:use-module (gnu packages dbm) #:use-module (gnu packages docbook) #:use-module (gnu packages file) @@ -981,6 +982,79 @@ environments.") ;; and the fonts included in this package are licensed OFL1.1. (license (list license:gpl3+ license:agpl3+ license:silofl1.1)))) +(define-public guix-build-coordinator + (let ((commit "5e8408c833e209efddfa0159114b90400c1aaf4d") + (revision "0")) + (package + (name "guix-build-coordinator") + (version (git-version "0" revision commit)) + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://git.cbaines.net/git/guix/build-coordinator") + (commit commit))) + (sha256 + (base32 + "0f7m1zg9mlb2m22qyblglaa36h8f49b810jc9j5b0hsb42ijwh4b")) + (file-name (string-append name "-" version "-checkout")))) + (build-system gnu-build-system) + (arguments + `(#:modules (((guix build guile-build-system) + #:select (target-guile-effective-version)) + ,@%gnu-build-system-modules) + #:imported-modules ((guix build guile-build-system) + ,@%gnu-build-system-modules) + #:phases + (modify-phases %standard-phases + (add-before 'build 'set-GUILE_AUTO_COMPILE + (lambda _ + ;; To avoid warnings relating to 'guild'. + (setenv "GUILE_AUTO_COMPILE" "0") + #t)) + (add-after 'install 'wrap-executable + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (bin (string-append out "/bin")) + (guile (assoc-ref inputs "guile")) + (version (target-guile-effective-version)) + (scm (string-append out "/share/guile/site/" version)) + (go (string-append out "/lib/guile/" version "/site-ccache"))) + (for-each + (lambda (file) + (simple-format (current-error-port) "wrapping: ~A\n" file) + (wrap-program file + `("PATH" ":" prefix + (,bin ,(assoc-ref inputs "sqitch"))) + `("GUILE_LOAD_PATH" ":" prefix + (,scm ,(getenv "GUILE_LOAD_PATH"))) + `("GUILE_LOAD_COMPILED_PATH" ":" prefix + (,go ,(getenv "GUILE_LOAD_COMPILED_PATH"))))) + (find-files bin))) + #t)) + (delete 'strip)))) ; As the .go files aren't compatible + (native-inputs + `(("pkg-config" ,pkg-config) + ("autoconf" ,autoconf) + ("automake" ,automake))) + (inputs + `(("guile" ,@(assoc-ref (package-native-inputs guix) "guile")) + ("guile-fibers" ,guile-fibers) + ("guile-prometheus" ,guile-prometheus) + ("guile-gcrypt" ,guile-gcrypt) + ("guile-json" ,guile-json-3) + ("guile-lzlib" ,guile-lzlib) + ("guile-sqlite3" ,guile-sqlite3) + ("guix" ,guix) + ("sqlite" ,sqlite) + ("sqitch" ,sqitch))) + (home-page "https://git.cbaines.net/guix/build-coordinator/") + (synopsis "Tool to help build derivations") + (description + "The Guix Build Coordinator helps with performing lots of builds across +potentially many machines, and with doing something with the results and +outputs of those builds.") + (license license:gpl3+)))) + (define-public guix-jupyter (package (name "guix-jupyter") -- 2.28.0 From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH v2 3/4] services: guix: Add guix-build-coordinator-agent-service-type. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 19 Sep 2020 10:12:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160051026727619 (code B ref 43494); Sat, 19 Sep 2020 10:12:02 +0000 Received: (at 43494) by debbugs.gnu.org; 19 Sep 2020 10:11:07 +0000 Received: from localhost ([127.0.0.1]:45701 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZpj-0007BN-0V for submit@debbugs.gnu.org; Sat, 19 Sep 2020 06:11:07 -0400 Received: from mira.cbaines.net ([212.71.252.8]:57768) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZpe-0007Ae-2j for 43494@debbugs.gnu.org; Sat, 19 Sep 2020 06:11:03 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id 95E4927BBE9 for <43494@debbugs.gnu.org>; Sat, 19 Sep 2020 11:11:01 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 67c71c7c for <43494@debbugs.gnu.org>; Sat, 19 Sep 2020 10:10:59 +0000 (UTC) From: Christopher Baines Date: Sat, 19 Sep 2020 11:10:58 +0100 Message-Id: <20200919101059.8930-3-mail@cbaines.net> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200919101059.8930-1-mail@cbaines.net> References: <20200919101059.8930-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * gnu/services/guix.scm (): New record type. (guix-build-coordinator-agent-configuration, guix-build-coordinator-agent-configuration?, guix-build-coordinator-agent-configuration-package, guix-build-coordinator-agent-configuration-user, guix-build-coordinator-agent-configuration-coordinator, guix-build-coordinator-agent-configuration-uuid), guix-build-coordinator-agent-configuration-password, guix-build-coordinator-agent-configuration-password-file, guix-build-coordinator-agent-configuration-systems, guix-build-coordinator-agent-configuration-max-parallel-builds, guix-build-coordinator-agent-configuration-derivation-substitute-urls, guix-build-coordinator-agent-configuration-non-derivation-substitute-urls, guix-build-coordinator-agent-shepherd-services, guix-build-coordinator-agent-activation, guix-build-coordinator-agent-account): New procedures. (guix-build-coordinator-agent-service-type): New variable. * doc/guix.texi (Guix Services): Document it. --- doc/guix.texi | 50 +++++++++++++++++ gnu/services/guix.scm | 121 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 171 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index 4f3f0e851f..b2dde30ddb 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -27554,6 +27554,56 @@ The Guile package with which to run the Guix Build Coordinator. @end table @end deftp +@defvar {Scheme Variable} guix-build-coordinator-agent-service-type +Service type for a Guix Build Coordinator agent. Its value must be a +@code{guix-build-coordinator-agent-configuration} object. +@end defvar + +@deftp {Data Type} guix-build-coordinator-agent-configuration +Data type representing the configuration a Guix Build Coordinator agent. + +@table @asis +@item @code{package} (default: @code{guix-build-coordinator}) +The Guix Build Coordinator package to use. + +@item @code{user} (default: @code{"guix-build-coordinator-agent"}) +The system user to run the service as. + +@item @code{coordinator} (default: @code{"http://localhost:8745"}) +The URI to use when connecting to the coordinator. + +@item @code{uuid} +The UUID of the agent. This should be generated by the coordinator +process, stored in the coordinator database, and used by the intended +agent. + +@item @code{password} (default: @code{#f}) +The password to use when connecting to the coordinator. A file to read +the password from can also be specified, and this is more secure. + +@item @code{password-file} (default: @code{#f}) +A file containing the password to use when connecting to the +coordinator. + +@item @code{systems} (default: @var{#f}) +The systems for which this agent should fetch builds. The agent process +will use the current system it's running on as the default. + +@item @code{max-parallel-builds} (default: @code{1}) +The number of builds to perform in parallel. + +@item @code{derivation-substitute-urls} (default: @code{1}) +URLs from which to attempt to fetch substitutes for derivations, if the +derivations aren't already available. + +@item @code{non-derivation-substitute-urls} (default: @code{1}) +URLs from which to attempt to fetch substitutes for build inputs, if the +input store items aren't already available. + +@end table +@end deftp + + @subsubheading Guix Data Service The @uref{http://data.guix.gnu.org,Guix Data Service} processes, stores and provides data about GNU Guix. This includes information about diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm index 1bacd61190..e4f5cf57b9 100644 --- a/gnu/services/guix.scm +++ b/gnu/services/guix.scm @@ -50,6 +50,21 @@ guix-build-coordinator-service-type + guix-build-coordinator-agent-configuration + guix-build-coordinator-agent-configuration? + guix-build-coordinator-agent-configuration-package + guix-build-coordinator-agent-configuration-user + guix-build-coordinator-agent-configuration-coordinator + guix-build-coordinator-agent-configuration-uuid + guix-build-coordinator-agent-configuration-password + guix-build-coordinator-agent-configuration-password-file + guix-build-coordinator-agent-configuration-systems + guix-build-coordinator-agent-configuration-max-parallel-builds + guix-build-coordinator-agent-configuration-derivation-substitute-urls + guix-build-coordinator-agent-configuration-non-derivation-substitute-urls + + guix-build-coordinator-agent-service-type + guix-data-service-configuration guix-data-service-configuration? @@ -95,6 +110,33 @@ (guile guix-build-coordinator-configuration-guile (default guile-3.0-latest))) +(define-record-type* + guix-build-coordinator-agent-configuration + make-guix-build-coordinator-agent-configuration + guix-build-coordinator-agent-configuration? + (package guix-build-coordinator-agent-configuration-package + (default guix-build-coordinator)) + (user guix-build-coordinator-agent-configuration-user + (default "guix-build-coordinator-agent")) + (coordinator guix-build-coordinator-agent-configuration-coordinator + (default "http://localhost:8745")) + (uuid guix-build-coordinator-agent-configuration-uuid) + (password guix-build-coordinator-agent-configuration-password + (default #f)) + (password-file guix-build-coordinator-agent-configuration-password-file + (default #f)) + (systems guix-build-coordinator-agent-configuration-systems + (default #f)) + (max-parallel-builds + guix-build-coordinator-agent-configuration-max-parallel-builds + (default 1)) + (derivation-substitute-urls + guix-build-coordinator-agent-configuration-derivation-substitute-urls + (default #f)) + (non-derivation-substitute-urls + guix-build-coordinator-agent-configuration-non-derivation-substitute-urls + (default #f))) + (define* (make-guix-build-coordinator-start-script database-uri-string allocation-strategy pid-file @@ -239,6 +281,85 @@ (description "Run an instance of the Guix Build Coordinator."))) +(define (guix-build-coordinator-agent-shepherd-services config) + (match-record config + (package user coordinator uuid password password-file max-parallel-builds + derivation-substitute-urls non-derivation-substitute-urls + systems) + (list + (shepherd-service + (documentation "Guix Build Coordinator Agent") + (provision '(guix-build-coordinator-agent)) + (requirement '(networking)) + (start #~(make-forkexec-constructor + (list #$(file-append package "/bin/guix-build-coordinator-agent") + #$(string-append "--coordinator=" coordinator) + #$(string-append "--uuid=" uuid) + #$@(if password + #~(#$(string-append "--password=" password)) + #~()) + #$@(if password-file + #~(#$(string-append "--password-file=" password-file)) + #~()) + #$(simple-format #f "--max-parallel-builds=~A" + max-parallel-builds) + #$@(if derivation-substitute-urls + #~(#$(string-append + "--derivation-substitute-urls=" + (string-join derivation-substitute-urls " "))) + #~()) + #$@(if non-derivation-substitute-urls + #~(#$(string-append + "--non-derivation-substitute-urls=" + (string-join derivation-substitute-urls " "))) + #~()) + #$@(map (lambda (system) + (string-append "--system=" system)) + (or systems '()))) + #:user #$user + #:pid-file "/var/run/guix-build-coordinator-agent/pid" + #:environment-variables + `(,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") + #:log-file "/var/log/guix-build-coordinator/agent.log")) + (stop #~(make-kill-destructor)))))) + +(define (guix-build-coordinator-agent-activation config) + #~(begin + (use-modules (guix build utils)) + + (mkdir-p "/var/log/guix-build-coordinator") + + ;; Allow writing the PID file + (mkdir-p "/var/run/guix-build-coordinator-agent") + (chown "/var/run/guix-build-coordinator-agent" + (passwd:uid %user) + (passwd:gid %user)))) + +(define (guix-build-coordinator-agent-account config) + (list (user-account + (name (guix-build-coordinator-agent-configuration-user config)) + (group "nogroup") + (system? #t) + (comment "Guix Build Coordinator agent user") + (home-directory "/var/empty") + (shell (file-append shadow "/sbin/nologin"))))) + +(define guix-build-coordinator-agent-service-type + (service-type + (name 'guix-build-coordinator-agent) + (extensions + (list + (service-extension shepherd-root-service-type + guix-build-coordinator-agent-shepherd-services) + (service-extension activation-service-type + guix-build-coordinator-agent-activation) + (service-extension account-service-type + guix-build-coordinator-agent-account))) + (description + "Run a Guix Build Coordinator agent."))) + ;;; ;;; Guix Data Service -- 2.28.0 From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH v2 2/4] services: guix: Add guix-build-coordinator-service-type. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 19 Sep 2020 10:12:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160051026827627 (code B ref 43494); Sat, 19 Sep 2020 10:12:02 +0000 Received: (at 43494) by debbugs.gnu.org; 19 Sep 2020 10:11:08 +0000 Received: from localhost ([127.0.0.1]:45703 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZpj-0007BQ-GH for submit@debbugs.gnu.org; Sat, 19 Sep 2020 06:11:08 -0400 Received: from mira.cbaines.net ([212.71.252.8]:57766) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZpd-0007Ac-UH for 43494@debbugs.gnu.org; Sat, 19 Sep 2020 06:11:03 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id 7EDCC27BBE8 for <43494@debbugs.gnu.org>; Sat, 19 Sep 2020 11:11:01 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id ca682f73 for <43494@debbugs.gnu.org>; Sat, 19 Sep 2020 10:10:59 +0000 (UTC) From: Christopher Baines Date: Sat, 19 Sep 2020 11:10:57 +0100 Message-Id: <20200919101059.8930-2-mail@cbaines.net> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200919101059.8930-1-mail@cbaines.net> References: <20200919101059.8930-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * gnu/services/guix.scm (): New record type. (guix-build-coordinator-configuration, guix-build-coordinator-configuration?, guix-build-coordinator-configuration-package, guix-build-coordinator-configuration-user, guix-build-coordinator-configuration-group, guix-build-coordinator-configuration-datastore-uri-string, guix-build-coordinator-configuration-agent-communication-uri-string, guix-build-coordinator-configuration-client-communication-uri-string, guix-build-coordinator-configuration-allocation-strategy, guix-build-coordinator-configuration-hooks, guix-build-coordinator-configuration-guile, make-guix-build-coordinator-start-script, guix-build-coordinator-shepherd-services, guix-build-coordinator-activation, guix-build-coordinator-account): New procedures. (guix-build-coordinator-service-type): New variable. * gnu/tests/guix.scm (%test-guix-build-coordinator): New variable. * doc/guix.texi (Guix Services): Document it. --- doc/guix.texi | 67 ++++++++++++++ gnu/services/guix.scm | 200 +++++++++++++++++++++++++++++++++++++++++- gnu/tests/guix.scm | 75 +++++++++++++++- 3 files changed, 338 insertions(+), 4 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index f7e2204b53..4f3f0e851f 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -27487,6 +27487,73 @@ The complete list of possible options can be found in the man page for @node Guix Services @subsection Guix Services +@subsubheading Guix Build Coordinator +The @uref{https://git.cbaines.net/guix/build-coordinator/,Guix Build +Coordinator} aids in distributing derivation builds among machines +running an @dfn{agent}. The build daemon is still used to build the +derivations, but the Guix Build Coordinator manages allocating builds +and working with the results. + +@quotation Note +This service is considered experimental. Configuration options may be +changed in a backwards-incompatible manner, and not all features have +been thorougly tested. +@end quotation + +The Guix Build Coordinator consists of one @dfn{coordinator}, and one or +more connected @dfn{agent} processes. The coordinator process handles +clients submitting builds, and allocating builds to agents. The agent +processes talk to a build daemon to actually perform the builds, then +send the results back to the coordinator. + +There is a script to run the coordinator component of the Guix Build +Coordinator, but the Guix service uses a custom Guile script instead, to +provide better integration with G-expressions used in the configuration. + +@defvar {Scheme Variable} guix-build-coordinator-service-type +Service type for the Guix Build Coordinator. Its value must be a +@code{guix-build-coordinator-configuration} object. +@end defvar + +@deftp {Data Type} guix-build-coordinator-configuration +Data type representing the configuration of the Guix Build Coordinator. + +@table @asis +@item @code{package} (default: @code{guix-build-coordinator}) +The Guix Build Coordinator package to use. + +@item @code{user} (default: @code{"guix-build-coordinator"}) +The system user to run the service as. + +@item @code{group} (default: @code{"guix-build-coordinator"}) +The system group to run the service as. + +@item @code{database-uri-string} (default: @code{"sqlite:///var/lib/guix-build-coordinator/guix_build_coordinator.db"}) +The URI to use for the database. + +@item @code{agent-communication-uri} (default: @code{"http://0.0.0.0:8745"}) +The URI describing how to listen to requests from agent processes. + +@item @code{client-communication-uri} (default: @code{"http://127.0.0.1:8746"}) +The URI describing how to listen to requests from clients. The client +API allows submitting builds and currently isn't authenticated, so take +care when configuring this value. + +@item @code{allocation-strategy} (default: @code{#~basic-build-allocation-strategy}) +A G-expression for the allocation strategy to be used. This is a +procedure that takes the datastore as an argument and populates the +allocation plan in the database. + +@item @code{hooks} (default: @var{'()}) +An association list of hooks. These provide a way to execute arbitrary +code upon certian events, like a build result being processed. + +@item @code{guile} (default: @code{guile-3.0-latest}) +The Guile package with which to run the Guix Build Coordinator. + +@end table +@end deftp + @subsubheading Guix Data Service The @uref{http://data.guix.gnu.org,Guix Data Service} processes, stores and provides data about GNU Guix. This includes information about diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm index 10a8581a62..1bacd61190 100644 --- a/gnu/services/guix.scm +++ b/gnu/services/guix.scm @@ -17,20 +17,40 @@ ;;; along with GNU Guix. If not, see . (define-module (gnu services guix) + #:use-module (srfi srfi-1) #:use-module (ice-9 match) #:use-module (guix gexp) #:use-module (guix records) + #:use-module (guix packages) #:use-module ((gnu packages base) #:select (glibc-utf8-locales)) #:use-module (gnu packages admin) + #:use-module (gnu packages databases) #:use-module (gnu packages web) + #:use-module (gnu packages guile) + #:use-module (gnu packages guile-xyz) + #:use-module (gnu packages package-management) #:use-module (gnu services) #:use-module (gnu services base) #:use-module (gnu services admin) #:use-module (gnu services shepherd) #:use-module (gnu services getmail) #:use-module (gnu system shadow) - #:export ( + #:export (guix-build-coordinator-configuration + guix-build-coordinator-configuration? + guix-build-coordinator-configuration-package + guix-build-coordinator-configuration-user + guix-build-coordinator-configuration-group + guix-build-coordinator-configuration-datastore-uri-string + guix-build-coordinator-configuration-agent-communication-uri-string + guix-build-coordinator-configuration-client-communication-uri-string + guix-build-coordinator-configuration-allocation-strategy + guix-build-coordinator-configuration-hooks + guix-build-coordinator-configuration-guile + + guix-build-coordinator-service-type + + guix-data-service-configuration guix-data-service-configuration? guix-data-service-package @@ -45,11 +65,185 @@ ;;;; Commentary: ;;; -;;; This module implements a service that to run instances of the Guix Data -;;; Service, which provides data about Guix over time. +;;; Services specifically related to GNU Guix. ;;; ;;;; Code: +(define-record-type* + guix-build-coordinator-configuration make-guix-build-coordinator-configuration + guix-build-coordinator-configuration? + (package guix-build-coordinator-configuration-package + (default guix-build-coordinator)) + (user guix-build-coordinator-configuration-user + (default "guix-build-coordinator")) + (group guix-build-coordinator-configuration-group + (default "guix-build-coordinator")) + (database-uri-string + guix-build-coordinator-configuration-datastore-uri-string + (default "sqlite:///var/lib/guix-build-coordinator/guix_build_coordinator.db")) + (agent-communication-uri-string + guix-build-coordinator-configuration-agent-communication-uri-string + (default "http://0.0.0.0:8745")) + (client-communication-uri-string + guix-build-coordinator-configuration-client-communication-uri-string + (default "http://127.0.0.1:8746")) + (allocation-strategy + guix-build-coordinator-configuration-allocation-strategy + (default #~basic-build-allocation-strategy)) + (hooks guix-build-coordinator-configuration-hooks + (default '())) + (guile guix-build-coordinator-configuration-guile + (default guile-3.0-latest))) + +(define* (make-guix-build-coordinator-start-script database-uri-string + allocation-strategy + pid-file + guix-build-coordinator-package + #:key + agent-communication-uri-string + client-communication-uri-string + (hooks '()) + (guile guile-3.0)) + (program-file + "start-guix-build-coordinator" + (with-extensions (cons guix-build-coordinator-package + ;; This is a poorly constructed Guile load path, + ;; since it contains things that aren't Guile + ;; libraries, but it means that the Guile libraries + ;; needed for the Guix Build Coordinator don't need + ;; to be individually specified here. + (map second (package-inputs + guix-build-coordinator-package))) + #~(begin + (use-modules (srfi srfi-1) + (ice-9 match) + (web uri) + (prometheus) + (guix-build-coordinator hooks) + (guix-build-coordinator datastore) + (guix-build-coordinator build-allocator) + (guix-build-coordinator coordinator)) + + (let* ((metrics-registry (make-metrics-registry + #:namespace + "guixbuildcoordinator_")) + (datastore (database-uri->datastore + #$database-uri-string + #:metrics-registry metrics-registry)) + (hooks + (list #$@(map (match-lambda + ((name . hook-gexp) + #~(cons name #$hook-gexp))) + hooks))) + (hooks-with-defaults + `(,@hooks + ,@(remove (match-lambda + ((name . _) (assq-ref hooks name))) + %default-hooks))) + (build-coordinator (make-build-coordinator + #:datastore datastore + #:hooks hooks-with-defaults + #:metrics-registry metrics-registry + #:allocation-strategy #$allocation-strategy))) + + (run-coordinator-service + build-coordinator + #:update-datastore? #t + #:pid-file #$pid-file + #:agent-communication-uri (string->uri + #$agent-communication-uri-string) + #:client-communication-uri (string->uri + #$client-communication-uri-string))))) + #:guile guile)) + +(define (guix-build-coordinator-shepherd-services config) + (match-record config + (package user group database-uri-string + agent-communication-uri-string + client-communication-uri-string + allocation-strategy + hooks + guile) + (list + (shepherd-service + (documentation "Guix Build Coordinator") + (provision '(guix-build-coordinator)) + (requirement '(networking)) + (start #~(make-forkexec-constructor + (list #$(make-guix-build-coordinator-start-script + database-uri-string + allocation-strategy + "/var/run/guix-build-coordinator/pid" + package + #:agent-communication-uri-string + agent-communication-uri-string + #:client-communication-uri-string + client-communication-uri-string + #:hooks hooks + #:guile guile)) + #:user #$user + #:group #$group + #:pid-file "/var/run/guix-build-coordinator/pid" + ;; Allow time for migrations to run + #:pid-file-timeout 60 + #:environment-variables + `(,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") + #:log-file "/var/log/guix-build-coordinator/coordinator.log")) + (stop #~(make-kill-destructor)))))) + +(define (guix-build-coordinator-activation config) + #~(begin + (use-modules (guix build utils)) + + (define %user (getpw "guix-build-coordinator")) + + (chmod "/var/lib/guix-build-coordinator" #o755) + + (mkdir-p "/var/log/guix-build-coordinator") + + ;; Allow writing the PID file + (mkdir-p "/var/run/guix-build-coordinator") + (chown "/var/run/guix-build-coordinator" + (passwd:uid %user) + (passwd:gid %user)))) + +(define (guix-build-coordinator-account config) + (match-record config + (user group) + (list (user-group + (name group) + (system? #t)) + (user-account + (name user) + (group group) + (system? #t) + (comment "Guix Build Coordinator user") + (home-directory "/var/lib/guix-build-coordinator") + (shell (file-append shadow "/sbin/nologin")))))) + +(define guix-build-coordinator-service-type + (service-type + (name 'guix-build-coordinator) + (extensions + (list + (service-extension shepherd-root-service-type + guix-build-coordinator-shepherd-services) + (service-extension activation-service-type + guix-build-coordinator-activation) + (service-extension account-service-type + guix-build-coordinator-account))) + (default-value + (guix-build-coordinator-configuration)) + (description + "Run an instance of the Guix Build Coordinator."))) + + +;;; +;;; Guix Data Service +;;; + (define-record-type* guix-data-service-configuration make-guix-data-service-configuration guix-data-service-configuration? diff --git a/gnu/tests/guix.scm b/gnu/tests/guix.scm index 6139e31cf0..20b67d55d3 100644 --- a/gnu/tests/guix.scm +++ b/gnu/tests/guix.scm @@ -35,7 +35,80 @@ #:use-module (guix store) #:use-module (guix utils) #:use-module (ice-9 match) - #:export (%test-guix-data-service)) + #:export (%test-guix-build-coordinator + %test-guix-data-service)) + +;;; +;;; Guix Build Coordinator +;;; + +(define %guix-build-coordinator-os + (simple-operating-system + (service dhcp-client-service-type) + (service guix-build-coordinator-service-type))) + +(define (run-guix-build-coordinator-test) + (define os + (marionette-operating-system + %guix-build-coordinator-os + #:imported-modules '((gnu services herd) + (guix combinators)))) + + (define forwarded-port 8745) + + (define vm + (virtual-machine + (operating-system os) + (memory-size 1024) + (port-forwardings `((,forwarded-port . 8745))))) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-11) (srfi srfi-64) + (gnu build marionette) + (web uri) + (web client) + (web response)) + + (define marionette + (make-marionette (list #$vm))) + + (mkdir #$output) + (chdir #$output) + + (test-begin "guix-build-coordinator") + + (test-assert "service running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'guix-build-coordinator) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (test-equal "http-get" + 200 + (let-values + (((response text) + (http-get #$(simple-format + #f "http://localhost:~A/metrics" forwarded-port) + #:decode-body? #t))) + (response-code response))) + + (test-end) + (exit (= (test-runner-fail-count (test-runner-current)) 0))))) + + (gexp->derivation "guix-build-coordinator-test" test)) + +(define %test-guix-build-coordinator + (system-test + (name "guix-build-coordinator") + (description "Connect to a running Guix Build Coordinator.") + (value (run-guix-build-coordinator-test)))) ;;; -- 2.28.0 From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH v2 4/4] services: guix: Add guix-build-coordinator-queue-builds-service-type. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 19 Sep 2020 10:12:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160051026827633 (code B ref 43494); Sat, 19 Sep 2020 10:12:03 +0000 Received: (at 43494) by debbugs.gnu.org; 19 Sep 2020 10:11:08 +0000 Received: from localhost ([127.0.0.1]:45705 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZpk-0007BY-9y for submit@debbugs.gnu.org; Sat, 19 Sep 2020 06:11:08 -0400 Received: from mira.cbaines.net ([212.71.252.8]:57770) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJZpe-0007An-7U for 43494@debbugs.gnu.org; Sat, 19 Sep 2020 06:11:03 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id BBC6D27BBEA for <43494@debbugs.gnu.org>; Sat, 19 Sep 2020 11:11:01 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id be83c506 for <43494@debbugs.gnu.org>; Sat, 19 Sep 2020 10:10:59 +0000 (UTC) From: Christopher Baines Date: Sat, 19 Sep 2020 11:10:59 +0100 Message-Id: <20200919101059.8930-4-mail@cbaines.net> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200919101059.8930-1-mail@cbaines.net> References: <20200919101059.8930-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * gnu/services/guix.scm (): New record type. (guix-build-coordinator-queue-builds-configuration, guix-build-coordinator-queue-builds-configuration?, guix-build-coordinator-queue-builds-configuration-package, guix-build-coordinator-queue-builds-configuration-user, guix-build-coordinator-queue-builds-coordinator, guix-build-coordinator-queue-builds-configuration-systems, guix-build-coordinator-queue-builds-configuration-system-and-targets, guix-build-coordinator-queue-builds-configuration-guix-data-service, guix-build-coordinator-queue-builds-configuration-processed-commits-file, guix-build-coordinator-queue-builds-shepherd-services, guix-build-coordinator-queue-builds-activation, guix-build-coordinator-queue-builds-account): New procedures. (guix-build-coordinator-queue-builds-service-type): New variable. --- doc/guix.texi | 38 +++++++++++++++ gnu/services/guix.scm | 108 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 146 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index b2dde30ddb..767c99c5cb 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -27603,6 +27603,44 @@ input store items aren't already available. @end table @end deftp +@defvar {Scheme Variable} guix-build-coordinator-queue-builds-service-type +Service type for the +guix-build-coordinator-queue-builds-from-guix-data-service script. Its +value must be a @code{guix-build-coordinator-queue-builds-configuration} +object. +@end defvar + +@deftp {Data Type} guix-build-coordinator-queue-builds-configuration +Data type representing the options to the queue builds from guix data +service script. + +@table @asis +@item @code{package} (default: @code{guix-build-coordinator}) +The Guix Build Coordinator package to use. + +@item @code{user} (default: @code{"guix-build-coordinator-queue-builds"}) +The system user to run the service as. + +@item @code{coordinator} (default: @code{"http://localhost:8745"}) +The URI to use when connecting to the coordinator. + +@item @code{systems} (default: @code{#f}) +The systems for which to fetch derivations to build. + +@item @code{systems-and-targets} (default: @code{#f}) +An association list of system and target pairs for which to fetch +derivations to build. + +@item @code{guix-data-service} (default: @code{"https://data.guix.gnu.org"}) +The Guix Data Service instance from which to query to find out about +derivations to build. + +@item @code{processed-commits-file} (default: @code{"/var/cache/guix-build-coordinator-queue-builds/processed-commits"}) +A file to record which commits have been processed, to avoid needlessly +processing them again if the service is restarted. + +@end table +@end deftp @subsubheading Guix Data Service The @uref{http://data.guix.gnu.org,Guix Data Service} processes, stores diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm index e4f5cf57b9..1493a24be5 100644 --- a/gnu/services/guix.scm +++ b/gnu/services/guix.scm @@ -65,6 +65,18 @@ guix-build-coordinator-agent-service-type + guix-build-coordinator-queue-builds-configuration + guix-build-coordinator-queue-builds-configuration? + guix-build-coordinator-queue-builds-configuration-package + guix-build-coordinator-queue-builds-configuration-user + guix-build-coordinator-queue-builds-coordinator + guix-build-coordinator-queue-builds-configuration-systems + guix-build-coordinator-queue-builds-configuration-system-and-targets + guix-build-coordinator-queue-builds-configuration-guix-data-service + guix-build-coordinator-queue-builds-configuration-processed-commits-file + + guix-build-coordinator-queue-builds-service-type + guix-data-service-configuration guix-data-service-configuration? @@ -137,6 +149,28 @@ guix-build-coordinator-agent-configuration-non-derivation-substitute-urls (default #f))) +(define-record-type* + guix-build-coordinator-queue-builds-configuration + make-guix-build-coordinator-queue-builds-configuration + guix-build-coordinator-queue-builds-configuration? + (package guix-build-coordinator-queue-builds-configuration-package + (default guix-build-coordinator)) + (user guix-build-coordinator-queue-builds-configuration-user + (default "guix-build-coordinator-queue-builds")) + (coordinator guix-build-coordinator-queue-builds-coordinator + (default "http://localhost:8745")) + (systems guix-build-coordinator-queue-builds-configuration-systems + (default #f)) + (systems-and-targets + guix-build-coordinator-queue-builds-configuration-system-and-targets + (default #f)) + (guix-data-service + guix-build-coordinator-queue-builds-configuration-guix-data-service + (default "https://data.guix.gnu.org")) + (processed-commits-file + guix-build-coordinator-queue-builds-configuration-processed-commits-file + (default "/var/cache/guix-build-coordinator-queue-builds/processed-commits"))) + (define* (make-guix-build-coordinator-start-script database-uri-string allocation-strategy pid-file @@ -360,6 +394,80 @@ (description "Run a Guix Build Coordinator agent."))) +(define (guix-build-coordinator-queue-builds-shepherd-services config) + (match-record config + (package user coordinator systems systems-and-targets + guix-data-service processed-commits-file) + (list + (shepherd-service + (documentation "Guix Build Coordinator queue builds from Guix Data Service") + (provision '(guix-build-coordinator-queue-builds)) + (requirement '(networking)) + (start + #~(make-forkexec-constructor + (list + #$(file-append + package + "/bin/guix-build-coordinator-queue-builds-from-guix-data-service") + #$(string-append "--coordinator=" coordinator) + #$@(map (lambda (system) + (string-append "--system=" system)) + (or systems '())) + #$@(map (match-lambda + ((system . target) + (string-append "--system-and-target=" system "=" target))) + (or systems-and-targets '())) + #$@(if guix-data-service + #~(#$(string-append "--guix-data-service=" guix-data-service)) + #~()) + #$@(if processed-commits-file + #~(#$(string-append "--processed-commits-file=" + processed-commits-file)) + #~())) + #:user #$user + #:pid-file "/var/run/guix-build-coordinator-queue-builds/pid" + #:environment-variables + `(,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") + #:log-file "/var/log/guix-build-coordinator/queue-builds.log")) + (stop #~(make-kill-destructor)))))) + +(define (guix-build-coordinator-queue-builds-activation config) + #~(begin + (use-modules (guix build utils)) + + (mkdir-p "/var/log/guix-build-coordinator") + + ;; Allow writing the PID file + (mkdir-p "/var/run/guix-build-coordinator-queue-builds") + (chown "/var/run/guix-build-coordinator-queue-builds" + (passwd:uid %user) + (passwd:gid %user)))) + +(define (guix-build-coordinator-queue-builds-account config) + (list (user-account + (name (guix-build-coordinator-queue-builds-configuration-user config)) + (group "nogroup") + (system? #t) + (comment "Guix Build Coordinator queue-builds user") + (home-directory "/var/empty") + (shell (file-append shadow "/sbin/nologin"))))) + +(define guix-build-coordinator-queue-builds-service-type + (service-type + (name 'guix-build-coordinator-queue-builds) + (extensions + (list + (service-extension shepherd-root-service-type + guix-build-coordinator-queue-builds-shepherd-services) + (service-extension activation-service-type + guix-build-coordinator-queue-builds-activation) + (service-extension account-service-type + guix-build-coordinator-queue-builds-account))) + (description + "Run the Guix Build Coordinator queue builds script."))) + ;;; ;;; Guix Data Service -- 2.28.0 From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 2/4] services: guix: Add guix-build-coordinator-service-type. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 25 Sep 2020 09:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Christopher Baines Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160102694726987 (code B ref 43494); Fri, 25 Sep 2020 09:43:02 +0000 Received: (at 43494) by debbugs.gnu.org; 25 Sep 2020 09:42:27 +0000 Received: from localhost ([127.0.0.1]:42676 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kLkFG-00071C-Vd for submit@debbugs.gnu.org; Fri, 25 Sep 2020 05:42:27 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56172) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kLkFF-000710-FO for 43494@debbugs.gnu.org; Fri, 25 Sep 2020 05:42:25 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52894) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLkF9-00035R-UD; Fri, 25 Sep 2020 05:42:19 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=60764 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kLkF9-0002HP-1f; Fri, 25 Sep 2020 05:42:19 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20200918184042.22660-1-mail@cbaines.net> <20200918184042.22660-2-mail@cbaines.net> <87a6xmvnth.fsf@gnu.org> <87a6xm9lih.fsf@cbaines.net> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 =?UTF-8?Q?Vend=C3=A9miaire?= an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 25 Sep 2020 11:42:17 +0200 In-Reply-To: <87a6xm9lih.fsf@cbaines.net> (Christopher Baines's message of "Sat, 19 Sep 2020 10:13:58 +0100") Message-ID: <87imc22nwm.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Christopher Baines skribis: > Ludovic Court=C3=A8s writes: [...] >>> + (program-file >>> + "start-guix-build-coordinator" >>> + (with-extensions (cons guix-build-coordinator-package >>> + ;; This is a poorly constructed Guile load p= ath, >>> + ;; since it contains things that aren't Guile >>> + ;; libraries, but it means that the Guile li= braries >>> + ;; needed for the Guix Build Coordinator don= 't need >>> + ;; to be individually specified here. >>> + (map second (package-inputs >>> + guix-build-coordinator-package)= )) >> >> Perhaps there should eventually be a =E2=80=98guix-build-coordinator=E2= =80=99 command in >> the package itself? > > There actually is, one thing I've had in mind for a while now though is > to use a scheme script constructed by the Guix service to run the > coordinator. > > For guix.cbaines.net, I'm using the script, but with the hooks passed in > on the command line, the command is rather long, and it means that > backtraces don't work well with the hooks. You mean because the hooks are interpreted, and so all you see in the backtrace is a bunch of =E2=80=98eval=E2=80=99 calls? > I'm unsure how well this has worked out, I didn't anticipate the issues > with the Guile load path, and because of the formatting stripping, both > the code and backtraces will be unreadable anyway... but it is useful in > allowing G-expressions to be used as part of the configuration. OK. Thanks, Ludo=E2=80=99. From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH 2/4] services: guix: Add guix-build-coordinator-service-type. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 26 Sep 2020 08:44:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.16011098253311 (code B ref 43494); Sat, 26 Sep 2020 08:44:01 +0000 Received: (at 43494) by debbugs.gnu.org; 26 Sep 2020 08:43:45 +0000 Received: from localhost ([127.0.0.1]:45777 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kM5o0-0000rL-Ok for submit@debbugs.gnu.org; Sat, 26 Sep 2020 04:43:44 -0400 Received: from mira.cbaines.net ([212.71.252.8]:41902) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kM5ny-0000rC-Aq for 43494@debbugs.gnu.org; Sat, 26 Sep 2020 04:43:43 -0400 Received: from localhost (unknown [46.237.174.91]) by mira.cbaines.net (Postfix) with ESMTPSA id 6C7F927BBE8; Sat, 26 Sep 2020 09:43:41 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 2973c718; Sat, 26 Sep 2020 08:43:38 +0000 (UTC) References: <20200918184042.22660-1-mail@cbaines.net> <20200918184042.22660-2-mail@cbaines.net> <87a6xmvnth.fsf@gnu.org> <87a6xm9lih.fsf@cbaines.net> <87imc22nwm.fsf@gnu.org> User-agent: mu4e 1.4.13; emacs 26.3 From: Christopher Baines In-reply-to: <87imc22nwm.fsf@gnu.org> Date: Sat, 26 Sep 2020 09:43:31 +0100 Message-ID: <87mu1dnd1o.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Hi! > > Christopher Baines skribis: > >> Ludovic Court=C3=A8s writes: > > [...] > >>>> + (program-file >>>> + "start-guix-build-coordinator" >>>> + (with-extensions (cons guix-build-coordinator-package >>>> + ;; This is a poorly constructed Guile load = path, >>>> + ;; since it contains things that aren't Gui= le >>>> + ;; libraries, but it means that the Guile l= ibraries >>>> + ;; needed for the Guix Build Coordinator do= n't need >>>> + ;; to be individually specified here. >>>> + (map second (package-inputs >>>> + guix-build-coordinator-package= ))) >>> >>> Perhaps there should eventually be a =E2=80=98guix-build-coordinator=E2= =80=99 command in >>> the package itself? >> >> There actually is, one thing I've had in mind for a while now though is >> to use a scheme script constructed by the Guix service to run the >> coordinator. >> >> For guix.cbaines.net, I'm using the script, but with the hooks passed in >> on the command line, the command is rather long, and it means that >> backtraces don't work well with the hooks. > > You mean because the hooks are interpreted, and so all you see in the > backtrace is a bunch of =E2=80=98eval=E2=80=99 calls? Yeah, I haven't done much testing of this, but that's my assumption. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl9u/zNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9Xdr5Q//QN7KCG9MfVAAMx7d9lor18eZMdbw5JJNRXwbPrTIhTd+rWadYU0w/8fm y59oFNhv3MFFHtOYtKt8r/axRDey57CIRnzRPcOfd9Gr31hJ8wq5vqgDPl1xmOsG 4W9rhxi6MAdAfr0qcz4yxq9MHGujAuSmwsj5gP8S+UWVw5YbS3IemhAWg7lkWHlB brakM/A2A6ZD4GBwbJuYi5ZnIK1Af8gyPuc4VSCjTPmrutFzaEkOFy5Mdlx/Ct49 v/VKSzmcHkkRYP3jZZv3kFZn9gsePFtK30uh5iPQg2MwRUdbq5p5DIBN6PSNuyAN MMKKtyeXGnVM9iOwDFPli+s4B577W5Bo/Heixg+4RpituHKe5HP5eSg0jalQJkhK uGaiTOCRZbDSTo9ZsXEvTwCK6v0rdlE3Er/oEATqcatkUSgOYU2qhxkVNwipHf0O PmWjwDFMD3eYNjGxnCPePNHm2xxP7LjsSDh5r1ez6ck8zdKOxu5Ifv+3OQTl54bp MimxV5gxd0tV8XmjE1U1cNlyV9VTReZKsrBEiXypz/5aYRbie6AdfR7ZPi4Faved dm2le0sJTxnH1xkB4/k3zelqF2NCI9RztVvOnXZsn/AH8TpTW9hsMq8acgHvT2H0 A0SAZEhTsXYoznJOaEoT10PBH9xSmfXxiggCNNiOU6DlvxAIYcs= =9BXV -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Sep 05 19:18:42 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#43494] [PATCH v2 4/4] services: guix: Add guix-build-coordinator-queue-builds-service-type. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 05 Oct 2020 08:02:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Christopher Baines Cc: 43494@debbugs.gnu.org Received: via spool by 43494-submit@debbugs.gnu.org id=B43494.160188486610786 (code B ref 43494); Mon, 05 Oct 2020 08:02:02 +0000 Received: (at 43494) by debbugs.gnu.org; 5 Oct 2020 08:01:06 +0000 Received: from localhost ([127.0.0.1]:47728 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kPLQf-0002n4-Ba for submit@debbugs.gnu.org; Mon, 05 Oct 2020 04:01:06 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52872) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kPLQd-0002gN-DB for 43494@debbugs.gnu.org; Mon, 05 Oct 2020 04:01:03 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54259) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kPLQX-0004PD-On; Mon, 05 Oct 2020 04:00:57 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=33900 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kPLQW-0007z2-Ck; Mon, 05 Oct 2020 04:00:57 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20200919101059.8930-1-mail@cbaines.net> <20200919101059.8930-4-mail@cbaines.net> Date: Mon, 05 Oct 2020 10:00:55 +0200 In-Reply-To: <20200919101059.8930-4-mail@cbaines.net> (Christopher Baines's message of "Sat, 19 Sep 2020 11:10:59 +0100") Message-ID: <87y2kl5cg8.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Chris, Sorry for the delay. Patches 1=E2=80=933 LGTM! One minor comment on patch= 4: Christopher Baines skribis: > * gnu/services/guix.scm (): > New record type. > (guix-build-coordinator-queue-builds-configuration, > guix-build-coordinator-queue-builds-configuration?, > guix-build-coordinator-queue-builds-configuration-package, > guix-build-coordinator-queue-builds-configuration-user, > guix-build-coordinator-queue-builds-coordinator, > guix-build-coordinator-queue-builds-configuration-systems, > guix-build-coordinator-queue-builds-configuration-system-and-targets, > guix-build-coordinator-queue-builds-configuration-guix-data-service, > guix-build-coordinator-queue-builds-configuration-processed-commits-file, > guix-build-coordinator-queue-builds-shepherd-services, > guix-build-coordinator-queue-builds-activation, > guix-build-coordinator-queue-builds-account): New procedures. > (guix-build-coordinator-queue-builds-service-type): New variable. [...] >=20=20 > +@defvar {Scheme Variable} guix-build-coordinator-queue-builds-service-ty= pe > +Service type for the > +guix-build-coordinator-queue-builds-from-guix-data-service script. Its > +value must be a @code{guix-build-coordinator-queue-builds-configuration} > +object. > +@end defvar Could you add a paragraph right above this explaining what this service is about? > +(define guix-build-coordinator-queue-builds-service-type > + (service-type > + (name 'guix-build-coordinator-queue-builds) > + (extensions > + (list > + (service-extension shepherd-root-service-type > + guix-build-coordinator-queue-builds-shepherd-ser= vices) > + (service-extension activation-service-type > + guix-build-coordinator-queue-builds-activation) > + (service-extension account-service-type > + guix-build-coordinator-queue-builds-account))) > + (description > + "Run the Guix Build Coordinator queue builds script."))) Likewise it=E2=80=99d be great if you could add a few works here. OK to push with these changes, thank you! Ludo=E2=80=99. From unknown Fri Sep 05 19:18:42 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Christopher Baines Subject: bug#43494: closed (Re: [bug#43494] [PATCH v2 4/4] services: guix: Add guix-build-coordinator-queue-builds-service-type.) Message-ID: References: <87a6x01tll.fsf@cbaines.net> <87o8m39bnc.fsf@cbaines.net> X-Gnu-PR-Message: they-closed 43494 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 43494@debbugs.gnu.org Date: Mon, 05 Oct 2020 17:17:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1601918222-23527-1" This is a multi-part message in MIME format... ------------=_1601918222-23527-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #43494: [PATCH 0/4] Add package and services for the Guix Build Coordinator which was filed against the guix-patches package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 43494@debbugs.gnu.org. --=20 43494: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D43494 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1601918222-23527-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 43494-done) by debbugs.gnu.org; 5 Oct 2020 17:16:31 +0000 Received: from localhost ([127.0.0.1]:50433 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kPU6B-00066j-BY for submit@debbugs.gnu.org; Mon, 05 Oct 2020 13:16:31 -0400 Received: from mira.cbaines.net ([212.71.252.8]:48358) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kPU68-00066V-7g for 43494-done@debbugs.gnu.org; Mon, 05 Oct 2020 13:16:29 -0400 Received: from localhost (92.41.184.114.threembb.co.uk [92.41.184.114]) by mira.cbaines.net (Postfix) with ESMTPSA id 4A31F27BBE8; Mon, 5 Oct 2020 18:16:27 +0100 (BST) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 15edbad6; Mon, 5 Oct 2020 17:16:24 +0000 (UTC) References: <20200919101059.8930-1-mail@cbaines.net> <20200919101059.8930-4-mail@cbaines.net> <87y2kl5cg8.fsf@gnu.org> User-agent: mu4e 1.4.13; emacs 27.1 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#43494] [PATCH v2 4/4] services: guix: Add guix-build-coordinator-queue-builds-service-type. In-reply-to: <87y2kl5cg8.fsf@gnu.org> Date: Mon, 05 Oct 2020 18:16:22 +0100 Message-ID: <87a6x01tll.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 43494-done Cc: 43494-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Hi Chris, > > Sorry for the delay. Patches 1=E2=80=933 LGTM! One minor comment on pat= ch 4: > > Christopher Baines skribis: > >> * gnu/services/guix.scm (): >> New record type. >> (guix-build-coordinator-queue-builds-configuration, >> guix-build-coordinator-queue-builds-configuration?, >> guix-build-coordinator-queue-builds-configuration-package, >> guix-build-coordinator-queue-builds-configuration-user, >> guix-build-coordinator-queue-builds-coordinator, >> guix-build-coordinator-queue-builds-configuration-systems, >> guix-build-coordinator-queue-builds-configuration-system-and-targets, >> guix-build-coordinator-queue-builds-configuration-guix-data-service, >> guix-build-coordinator-queue-builds-configuration-processed-commits-file, >> guix-build-coordinator-queue-builds-shepherd-services, >> guix-build-coordinator-queue-builds-activation, >> guix-build-coordinator-queue-builds-account): New procedures. >> (guix-build-coordinator-queue-builds-service-type): New variable. > > [...] > >>=20=20 >> +@defvar {Scheme Variable} guix-build-coordinator-queue-builds-service-t= ype >> +Service type for the >> +guix-build-coordinator-queue-builds-from-guix-data-service script. Its >> +value must be a @code{guix-build-coordinator-queue-builds-configuration} >> +object. >> +@end defvar > > Could you add a paragraph right above this explaining what this service > is about? > >> +(define guix-build-coordinator-queue-builds-service-type >> + (service-type >> + (name 'guix-build-coordinator-queue-builds) >> + (extensions >> + (list >> + (service-extension shepherd-root-service-type >> + guix-build-coordinator-queue-builds-shepherd-se= rvices) >> + (service-extension activation-service-type >> + guix-build-coordinator-queue-builds-activation) >> + (service-extension account-service-type >> + guix-build-coordinator-queue-builds-account))) >> + (description >> + "Run the Guix Build Coordinator queue builds script."))) > > Likewise it=E2=80=99d be great if you could add a few works here. > > OK to push with these changes, thank you! I've tried to add some clarity in the places you mention. I'm still very tempted to make a separate repository/package for the queue builds from Guix Data Service script... I'm not quite decided yet though. Anyway, thanks for taking another look. I've gone ahead and pushed these patches as c14714cbbfe9239410aec2ed73282192a64fbbdc. I did tweak the package definition a bit though to move closer to being able to cross-compile it (you can now if you tweak the inputs). Thanks again, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl97VOZfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XelCRAAi3Ihs57Jxpd29ZfQX/6QqK8LTMWGs8dQ juPk6/Yl0DzaxcEeuzf+TPR0t/zuDCzucv9g0+bx4ohPTN08MPoGT6y+tdY1zDNh 7MxSYiLiM8IOLxDwPoB6LoFTyUcsV3EJNbfrdf3fN3IEwTd8y9HOBrRuCT46lRNc 03ku7yL44JhiHotzg1YWY+si2y/OAAPb4s0iP7Chx9ppLdvyqANcV0leuVJrtP3y itb23Gb9BCmSfFobEeI3xo4se6LsXqpYj8fq3kGipC4DCoWkZiA99gW4UiY3C4+h d2zmElR4+x2lcAQD2Z8SLWK/5bej1iWeJ7q8OvuAgRJS6LlDTkxxI73m8frqnui0 EGZGonG+UrXmsM9h/Pp7aIQU+C5rDu7u3NqmG+WeeDjgJTD5vorkpKhMwDfeH6sL BSsMWiUvEpwWMx2MqYOZSo3qn8wIb/HfUFAZNNrWhasCygOTWaKOgm5IdfVbSENO wxve8VPF6UG26diVc/1OaDBmMdao7+L9eU+CHD/J/U9xVcRBGPdZ8ATFJfKTOUnT myusPwsdTYNM1AAdZFVev0sOrRhmW6lWh117n8ideJ68w4ldpK9s70Ln+IfTPbUx jKlSiKn0dYOui61oag0EFwJrm3uWJEBggLd7TUC9Gp+tZwpDuFsYJCIULHkJeXC9 sWHkqe4RtGo= =pTf2 -----END PGP SIGNATURE----- --=-=-=-- ------------=_1601918222-23527-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 18 Sep 2020 18:34:57 +0000 Received: from localhost ([127.0.0.1]:44378 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLDl-0003F2-Ef for submit@debbugs.gnu.org; Fri, 18 Sep 2020 14:34:57 -0400 Received: from lists.gnu.org ([209.51.188.17]:46262) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kJLDk-0003Ev-4H for submit@debbugs.gnu.org; Fri, 18 Sep 2020 14:34:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47080) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kJLDi-0004WE-Aa for guix-patches@gnu.org; Fri, 18 Sep 2020 14:34:55 -0400 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:36565) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kJLDg-0006bo-G0 for guix-patches@gnu.org; Fri, 18 Sep 2020 14:34:53 -0400 Received: from localhost (unknown [46.237.174.246]) by mira.cbaines.net (Postfix) with ESMTPSA id C6F7B27BBE7 for ; Fri, 18 Sep 2020 19:34:49 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id cf0e3d57 for ; Fri, 18 Sep 2020 18:34:47 +0000 (UTC) User-agent: mu4e 1.4.13; emacs 26.3 From: Christopher Baines To: guix-patches@gnu.org Subject: [PATCH 0/4] Add package and services for the Guix Build Coordinator Date: Fri, 18 Sep 2020 19:34:47 +0100 Message-ID: <87o8m39bnc.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/18 14:34:50 X-ACL-Warn: Detected OS = ??? X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --=-=-= Content-Type: text/plain This patch series is not particularly well tested, like the Guix Build Coordinator itself, but it should hopefully still work at least in some configurations. Christopher Baines (4): gnu: Add guix-build-coordinator. services: guix: Add guix-build-coordinator-service-type. services: guix: Add guix-build-coordinator-agent-service-type. services: guix: Add guix-build-coordinator-queue-builds-service-type. doc/guix.texi | 143 +++++++++ gnu/packages/package-management.scm | 89 ++++++ gnu/services/guix.scm | 433 +++++++++++++++++++++++++++- gnu/tests/guix.scm | 75 ++++- 4 files changed, 736 insertions(+), 4 deletions(-) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl9k/cdfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9Xfrig//b1nSvUfhfTzlMrAc5stuOAEBdd7hrG82aj8exISjDZa2UfzFN0ox828z ggjWbz69KjKP41OJfQgNGJ9GXqk582CvUaFloQBPQXH+3tBNJR+DlZHYEPhO5KlU K0ZuatnpqCC704DQOsqTPAxKsaKuqSHYy9j7XFMieqilznvv0YMp8qpepTRyza6F Y/H0qG4Ohfk3EhV5c99c4Yyvxx6x16TB+Gr93HBClVh3AYNbiEW2xuN7xSS03ljl yr26rpna8fY4sXH3hUgbyOu7B1LNeiAveLRnEW2E3OtRBhXFB/nBc/L17Uz2tje4 zRD86Bl4nWZ26v9FKaTwQAKaCl0S5Fi5DSPZviO8FySVRXfDHYy3PaZks3HHpDDQ /vgMbLba9BbBJ0Zq7/uoGgk9rWLUCuUDkfIigxlvyMHUAuNCnh3PnvjwJ3zenK6B HGSJ2Xi2JDE0pHrt4Tzs4uBG2JC+JyDZAhgZjJSkFfWP4YdJaLKq7J8BkFJV5c/E x42nxLdf9LaqeR7zfCqS8u0mBocmrvl9bfOY7aafo4ezE6ktDDp1ecwnxzpVlgIE Pn3ItRAIrs/w1O8i0ITHIMArpUdS+Yk+rfyEs400QB4kSXnphaBQdUKL0arRIFXr nqzi+I857L7ZqKWR7AKGSmWAJQay3KzzuENVho+dLJtC4wZxwcg= =uTsG -----END PGP SIGNATURE----- --=-=-=-- ------------=_1601918222-23527-1--