From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 27 18:24:17 2018 Received: (at submit) by debbugs.gnu.org; 27 Aug 2018 22:24:17 +0000 Received: from localhost ([127.0.0.1]:34540 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fuPvl-0002YU-DK for submit@debbugs.gnu.org; Mon, 27 Aug 2018 18:24:17 -0400 Received: from eggs.gnu.org ([208.118.235.92]:35866) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fuPvk-0002YI-2e for submit@debbugs.gnu.org; Mon, 27 Aug 2018 18:24:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fuPvZ-0006MJ-Ph for submit@debbugs.gnu.org; Mon, 27 Aug 2018 18:24:10 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:51907) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fuPvY-0006Lp-Ci for submit@debbugs.gnu.org; Mon, 27 Aug 2018 18:24:05 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56825) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fuPvW-0003zR-Ot for guix-patches@gnu.org; Mon, 27 Aug 2018 18:24:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fuPvO-00065f-HQ for guix-patches@gnu.org; Mon, 27 Aug 2018 18:24:00 -0400 Received: from mail.lassieur.org ([83.152.10.219]:45452) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fuPvK-0005xu-KC for guix-patches@gnu.org; Mon, 27 Aug 2018 18:23:52 -0400 Received: from localhost.localdomain (88.191.118.83 [88.191.118.83]) by mail.lassieur.org (OpenSMTPD) with ESMTPSA id 62840401 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO) for ; Mon, 27 Aug 2018 22:21:55 +0000 (UTC) From: =?UTF-8?q?Cl=C3=A9ment=20Lassieur?= To: guix-patches@gnu.org Subject: [PATCH] hydra: Add support for manifests. Date: Tue, 28 Aug 2018 00:23:33 +0200 Message-Id: <20180827222333.7722-1-clement@lassieur.org> X-Mailer: git-send-email 2.18.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.1 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.1 (-----) * build-aux/hydra/gnu-system.scm (arguments->manifests, manifests->packages): New procedures. (hydra-jobs): Add a "manifests" subset. * doc/guix.texi (Continuous Integration): Update accordingly. --- build-aux/hydra/gnu-system.scm | 34 +++++++++++++++++++++ doc/guix.texi | 56 +++++++++++++++++++++++----------- 2 files changed, 72 insertions(+), 18 deletions(-) diff --git a/build-aux/hydra/gnu-system.scm b/build-aux/hydra/gnu-system.scm index b1554ced4..e8ce8eada 100644 --- a/build-aux/hydra/gnu-system.scm +++ b/build-aux/hydra/gnu-system.scm @@ -56,6 +56,7 @@ (guix packages) (guix derivations) (guix monads) + (guix ui) ((guix licenses) #:select (gpl3+)) ((guix utils) #:select (%current-system)) ((guix scripts system) #:select (read-operating-system)) @@ -311,6 +312,30 @@ valid." packages))) #:select? (const #t))) ;include hidden packages +(define (arguments->manifests arguments) + "Return the list of manifests extracted from ARGUMENTS." + (map (match-lambda + ((input-name . relative-path) + (let* ((checkout (assq-ref arguments (string->symbol input-name))) + (base (assq-ref checkout 'file-name))) + (in-vicinity base relative-path)))) + (assq-ref arguments 'manifests))) + +(define (manifests->packages store manifests) + "Return the list of packages found in MANIFESTS." + (define (load-manifest manifest) + (save-module-excursion + (lambda () + (set-current-module (make-user-module '((guix profiles) (gnu)))) + (primitive-load manifest)))) + + (parameterize ((%graft? #f)) + (delete-duplicates! + (map manifest-entry-item + (append-map (compose manifest-entries + load-manifest) + manifests))))) + ;;; ;;; Hydra entry point. @@ -323,6 +348,7 @@ valid." ("core" 'core) ; only build core packages ("hello" 'hello) ; only build hello (((? string?) (? string?) ...) 'list) ; only build selected list of packages + ("manifests" 'manifests) ; only build packages in the list of manifests (_ 'all))) ; build everything (define systems @@ -419,6 +445,14 @@ valid." package system)) packages)) '())) + ((manifests) + ;; Build packages in the list of manifests. + (let* ((manifests (arguments->manifests arguments)) + (packages (manifests->packages store manifests))) + (map (lambda (package) + (package-job store (job-name package) + package system)) + packages))) (else (error "unknown subset" subset)))) systems))) diff --git a/doc/guix.texi b/doc/guix.texi index d2d278df4..2b2fe6c93 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -17704,23 +17704,43 @@ The type of the Cuirass service. Its value must be a @code{cuirass-configuration} object, as described below. @end defvr -To add build jobs, you have to set the @code{specifications} field of -the configuration. Here is an example of a service defining a build job -based on a specification that can be found in Cuirass source tree. This -service polls the Guix repository and builds a subset of the Guix -packages, as prescribed in the @file{gnu-system.scm} example spec: - -@example -(let ((spec #~((#:name . "guix") - (#:url . "git://git.savannah.gnu.org/guix.git") - (#:load-path . ".") - (#:file . "build-aux/cuirass/gnu-system.scm") - (#:proc . cuirass-jobs) - (#:arguments (subset . "hello")) - (#:branch . "master")))) - (service cuirass-service-type - (cuirass-configuration - (specifications #~(list '#$spec))))) +To add build jobs, you have to set the @code{specifications} field of the +configuration. Here is an example of a service that polls the Guix repository +and builds the packages from a manifest. Some of the packages are defined in +the @code{"custom-packages"} input, which is the equivalent of +@code{GUIX_PACKAGE_PATH}. + +@example +(define %cuirass-specs + #~(list + '((#:name . "my-manifest") + (#:load-path-inputs . ("guix")) + (#:package-path-inputs . ("custom-packages")) + (#:proc-input . "guix") + (#:proc-file . "build-aux/cuirass/gnu-system.scm") + (#:proc . cuirass-jobs) + (#:proc-args . ((subset . "manifests") + (systems . ("x86_64-linux")) + (manifests . (("config" . "guix/manifest.scm"))))) + (#:inputs . (((#:name . "guix") + (#:url . "git://git.savannah.gnu.org/guix.git") + (#:load-path . ".") + (#:branch . "master") + (#:no-compile? . #t)) + ((#:name . "config") + (#:url . "git://git.example.org/config.git") + (#:load-path . ".") + (#:branch . "master") + (#:no-compile? . #t)) + ((#:name . "custom-packages") + (#:url . "git://git.example.org/custom-packages.git") + (#:load-path . ".") + (#:branch . "master") + (#:no-compile? . #t))))))) + +(service cuirass-service-type + (cuirass-configuration + (specifications %cuirass-specs))) @end example While information related to build jobs is located directly in the @@ -17747,7 +17767,7 @@ Owner's group of the @code{cuirass} process. Number of seconds between the poll of the repositories followed by the Cuirass jobs. -@item @code{database} (default: @code{"/var/run/cuirass/cuirass.db"}) +@item @code{database} (default: @code{"/var/lib/cuirass/cuirass.db"}) Location of sqlite database which contains the build results and previously added specifications. -- 2.18.0 From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 13 04:19:43 2018 Received: (at 32547-done) by debbugs.gnu.org; 13 Sep 2018 08:19:43 +0000 Received: from localhost ([127.0.0.1]:38706 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g0Mqj-0001me-Bq for submit@debbugs.gnu.org; Thu, 13 Sep 2018 04:19:42 -0400 Received: from mail.lassieur.org ([83.152.10.219]:54308) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g0Mqh-0001mV-6W for 32547-done@debbugs.gnu.org; Thu, 13 Sep 2018 04:19:40 -0400 Received: from newt (smtp.parrot.biz [62.23.167.188]) by mail.lassieur.org (OpenSMTPD) with ESMTPSA id 324f71fe (TLSv1.2:ECDHE-RSA-CHACHA20-POLY1305:256:NO) for <32547-done@debbugs.gnu.org>; Thu, 13 Sep 2018 08:14:49 +0000 (UTC) References: <20180827222333.7722-1-clement@lassieur.org> User-agent: mu4e 1.0; emacs 26.1 From: =?utf-8?Q?Cl=C3=A9ment?= Lassieur To: 32547-done@debbugs.gnu.org Subject: Re: [bug#32547] [PATCH] hydra: Add support for manifests. In-reply-to: <20180827222333.7722-1-clement@lassieur.org> Date: Thu, 13 Sep 2018 10:19:36 +0200 Message-ID: <8736udbyhz.fsf@lassieur.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 32547-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Cl=C3=A9ment Lassieur writes: > * build-aux/hydra/gnu-system.scm (arguments->manifests, manifests->packag= es): > New procedures. > (hydra-jobs): Add a "manifests" subset. > * doc/guix.texi (Continuous Integration): Update accordingly. Pushed! From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 13 04:34:52 2018 Received: (at 32547) by debbugs.gnu.org; 13 Sep 2018 08:34:52 +0000 Received: from localhost ([127.0.0.1]:38716 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g0N5Q-0002Dy-Cq for submit@debbugs.gnu.org; Thu, 13 Sep 2018 04:34:52 -0400 Received: from eggs.gnu.org ([208.118.235.92]:47766) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g0N5P-0002Dj-64 for 32547@debbugs.gnu.org; Thu, 13 Sep 2018 04:34:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g0N5D-0001J1-Rh for 32547@debbugs.gnu.org; Thu, 13 Sep 2018 04:34:44 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:58255) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g0N5B-00014k-K1; Thu, 13 Sep 2018 04:34:38 -0400 Received: from [193.50.110.53] (port=33826 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1g0N5B-0000bB-BR; Thu, 13 Sep 2018 04:34:37 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: =?utf-8?Q?Cl=C3=A9ment?= Lassieur Subject: Re: [bug#32547] [PATCH] hydra: Add support for manifests. References: <20180827222333.7722-1-clement@lassieur.org> Date: Thu, 13 Sep 2018 10:34:34 +0200 In-Reply-To: <20180827222333.7722-1-clement@lassieur.org> (=?utf-8?Q?=22Cl?= =?utf-8?Q?=C3=A9ment?= Lassieur"'s message of "Tue, 28 Aug 2018 00:23:33 +0200") Message-ID: <8736udre1x.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 32547 Cc: 32547@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Hi Cl=C3=A9ment, Cl=C3=A9ment Lassieur skribis: > * build-aux/hydra/gnu-system.scm (arguments->manifests, manifests->packag= es): > New procedures. > (hydra-jobs): Add a "manifests" subset. > * doc/guix.texi (Continuous Integration): Update accordingly. Thanks for pushing the patch, and sorry for not commenting earlier! I agree this can be very useful. I=E2=80=99m slightly concerned about the complexity of this file. Maybe sometime down the road we could split it in several proper modules, like (guix ci jobs) that would provide procedures to define =E2=80=9Cpackage job= s=E2=80=9D, =E2=80=9Ccross-compilation jobs=E2=80=9D, and =E2=80=9CVM jobs=E2=80=9D. Thoughts? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 13 05:13:51 2018 Received: (at 32547) by debbugs.gnu.org; 13 Sep 2018 09:13:51 +0000 Received: from localhost ([127.0.0.1]:38747 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g0Nh9-0003GY-FU for submit@debbugs.gnu.org; Thu, 13 Sep 2018 05:13:51 -0400 Received: from mail.lassieur.org ([83.152.10.219]:54312) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g0Nh3-0003GG-Tr for 32547@debbugs.gnu.org; Thu, 13 Sep 2018 05:13:48 -0400 Received: from newt (smtp.parrot.biz [62.23.167.188]) by mail.lassieur.org (OpenSMTPD) with ESMTPSA id e466a8c4 (TLSv1.2:ECDHE-RSA-CHACHA20-POLY1305:256:NO); Thu, 13 Sep 2018 09:08:55 +0000 (UTC) References: <20180827222333.7722-1-clement@lassieur.org> <8736udre1x.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: =?utf-8?Q?Cl=C3=A9ment?= Lassieur To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#32547] [PATCH] hydra: Add support for manifests. In-reply-to: <8736udre1x.fsf@gnu.org> Date: Thu, 13 Sep 2018 11:13:43 +0200 Message-ID: <87zhwlahfc.fsf@lassieur.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 32547 Cc: 32547@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludovic, Ludovic Court=C3=A8s writes: > Hi Cl=C3=A9ment, > > Cl=C3=A9ment Lassieur skribis: > >> * build-aux/hydra/gnu-system.scm (arguments->manifests, manifests->packa= ges): >> New procedures. >> (hydra-jobs): Add a "manifests" subset. >> * doc/guix.texi (Continuous Integration): Update accordingly. > > Thanks for pushing the patch, and sorry for not commenting earlier! I > agree this can be very useful. > > I=E2=80=99m slightly concerned about the complexity of this file. Maybe > sometime down the road we could split it in several proper modules, like > (guix ci jobs) that would provide procedures to define =E2=80=9Cpackage j= obs=E2=80=9D, > =E2=80=9Ccross-compilation jobs=E2=80=9D, and =E2=80=9CVM jobs=E2=80=9D. Yes, I agree! And "test jobs" maybe? Cl=C3=A9ment From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 13 12:13:11 2018 Received: (at 32547) by debbugs.gnu.org; 13 Sep 2018 16:13:11 +0000 Received: from localhost ([127.0.0.1]:39416 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g0UEx-0001R5-9o for submit@debbugs.gnu.org; Thu, 13 Sep 2018 12:13:11 -0400 Received: from eggs.gnu.org ([208.118.235.92]:59014) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g0UEt-0001Qi-Ay for 32547@debbugs.gnu.org; Thu, 13 Sep 2018 12:13:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g0UEn-0004WO-H3 for 32547@debbugs.gnu.org; Thu, 13 Sep 2018 12:13:02 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:38635) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g0UEi-0004V0-Kx; Thu, 13 Sep 2018 12:12:58 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=47008 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1g0UEi-0004t3-CC; Thu, 13 Sep 2018 12:12:56 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: =?utf-8?Q?Cl=C3=A9ment?= Lassieur Subject: Re: [bug#32547] [PATCH] hydra: Add support for manifests. References: <20180827222333.7722-1-clement@lassieur.org> <8736udre1x.fsf@gnu.org> <87zhwlahfc.fsf@lassieur.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 27 Fructidor an 226 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 13 Sep 2018 18:12:53 +0200 In-Reply-To: <87zhwlahfc.fsf@lassieur.org> (=?utf-8?Q?=22Cl=C3=A9ment?= Lassieur"'s message of "Thu, 13 Sep 2018 11:13:43 +0200") Message-ID: <874letpe9m.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 32547 Cc: 32547@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Cl=C3=A9ment Lassieur skribis: > Ludovic Court=C3=A8s writes: > >> Hi Cl=C3=A9ment, >> >> Cl=C3=A9ment Lassieur skribis: >> >>> * build-aux/hydra/gnu-system.scm (arguments->manifests, manifests->pack= ages): >>> New procedures. >>> (hydra-jobs): Add a "manifests" subset. >>> * doc/guix.texi (Continuous Integration): Update accordingly. >> >> Thanks for pushing the patch, and sorry for not commenting earlier! I >> agree this can be very useful. >> >> I=E2=80=99m slightly concerned about the complexity of this file. Maybe >> sometime down the road we could split it in several proper modules, like >> (guix ci jobs) that would provide procedures to define =E2=80=9Cpackage = jobs=E2=80=9D, >> =E2=80=9Ccross-compilation jobs=E2=80=9D, and =E2=80=9CVM jobs=E2=80=9D. > > Yes, I agree! And "test jobs" maybe? Indeed. Well, food for thought! Ludo=E2=80=99. From unknown Wed Sep 10 18:35:46 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 12 Oct 2018 11:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator