GNU bug report logs - #32547
[PATCH] hydra: Add support for manifests.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 32547 in the body.
You can then email your comments to 32547 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Clément Lassieur <clement <at> lassieur.org>
To: guix-patches <at> gnu.org
Subject: [PATCH] hydra: Add support for manifests.
Date: Tue, 28 Aug 2018 00:23:33 +0200

* build-aux/hydra/gnu-system.scm (arguments->manifests, manifests->packages):
New procedures.
(hydra-jobs): Add a "manifests" subset.
* doc/guix.texi (Continuous Integration): Update accordingly.
---
 build-aux/hydra/gnu-system.scm | 34 +++++++++++++++++++++
 doc/guix.texi                  | 56 +++++++++++++++++++++++-----------
 2 files changed, 72 insertions(+), 18 deletions(-)

diff --git a/build-aux/hydra/gnu-system.scm b/build-aux/hydra/gnu-system.scm
index b1554ced4..e8ce8eada 100644
--- a/build-aux/hydra/gnu-system.scm
+++ b/build-aux/hydra/gnu-system.scm
@@ -56,6 +56,7 @@
              (guix packages)
              (guix derivations)
              (guix monads)
+             (guix ui)
              ((guix licenses) #:select (gpl3+))
              ((guix utils) #:select (%current-system))
              ((guix scripts system) #:select (read-operating-system))
@@ -311,6 +312,30 @@ valid."
                           packages)))
                  #:select? (const #t)))           ;include hidden packages
 
+(define (arguments->manifests arguments)
+  "Return the list of manifests extracted from ARGUMENTS."
+  (map (match-lambda
+         ((input-name . relative-path)
+          (let* ((checkout (assq-ref arguments (string->symbol input-name)))
+                 (base (assq-ref checkout 'file-name)))
+            (in-vicinity base relative-path))))
+       (assq-ref arguments 'manifests)))
+
+(define (manifests->packages store manifests)
+  "Return the list of packages found in MANIFESTS."
+  (define (load-manifest manifest)
+    (save-module-excursion
+     (lambda ()
+       (set-current-module (make-user-module '((guix profiles) (gnu))))
+       (primitive-load manifest))))
+
+  (parameterize ((%graft? #f))
+    (delete-duplicates!
+     (map manifest-entry-item
+          (append-map (compose manifest-entries
+                               load-manifest)
+                      manifests)))))
+
 
 ;;;
 ;;; Hydra entry point.
@@ -323,6 +348,7 @@ valid."
       ("core" 'core)                              ; only build core packages
       ("hello" 'hello)                            ; only build hello
       (((? string?) (? string?) ...) 'list)       ; only build selected list of packages
+      ("manifests" 'manifests)                    ; only build packages in the list of manifests
       (_ 'all)))                                  ; build everything
 
   (define systems
@@ -419,6 +445,14 @@ valid."
                                                  package system))
                                   packages))
                          '()))
+                    ((manifests)
+                     ;; Build packages in the list of manifests.
+                     (let* ((manifests (arguments->manifests arguments))
+                            (packages (manifests->packages store manifests)))
+                       (map (lambda (package)
+                              (package-job store (job-name package)
+                                           package system))
+                            packages)))
                     (else
                      (error "unknown subset" subset))))
                 systems)))
diff --git a/doc/guix.texi b/doc/guix.texi
index d2d278df4..2b2fe6c93 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -17704,23 +17704,43 @@ The type of the Cuirass service.  Its value must be a
 @code{cuirass-configuration} object, as described below.
 @end defvr
 
-To add build jobs, you have to set the @code{specifications} field of
-the configuration.  Here is an example of a service defining a build job
-based on a specification that can be found in Cuirass source tree.  This
-service polls the Guix repository and builds a subset of the Guix
-packages, as prescribed in the @file{gnu-system.scm} example spec:
-
-@example
-(let ((spec #~((#:name . "guix")
-               (#:url . "git://git.savannah.gnu.org/guix.git")
-               (#:load-path . ".")
-               (#:file . "build-aux/cuirass/gnu-system.scm")
-               (#:proc . cuirass-jobs)
-               (#:arguments (subset . "hello"))
-               (#:branch . "master"))))
-  (service cuirass-service-type
-           (cuirass-configuration
-            (specifications #~(list '#$spec)))))
+To add build jobs, you have to set the @code{specifications} field of the
+configuration.  Here is an example of a service that polls the Guix repository
+and builds the packages from a manifest.  Some of the packages are defined in
+the @code{"custom-packages"} input, which is the equivalent of
+@code{GUIX_PACKAGE_PATH}.
+
+@example
+(define %cuirass-specs
+  #~(list
+     '((#:name . "my-manifest")
+       (#:load-path-inputs . ("guix"))
+       (#:package-path-inputs . ("custom-packages"))
+       (#:proc-input . "guix")
+       (#:proc-file . "build-aux/cuirass/gnu-system.scm")
+       (#:proc . cuirass-jobs)
+       (#:proc-args . ((subset . "manifests")
+                       (systems . ("x86_64-linux"))
+                       (manifests . (("config" . "guix/manifest.scm")))))
+       (#:inputs . (((#:name . "guix")
+                     (#:url . "git://git.savannah.gnu.org/guix.git")
+                     (#:load-path . ".")
+                     (#:branch . "master")
+                     (#:no-compile? . #t))
+                    ((#:name . "config")
+                     (#:url . "git://git.example.org/config.git")
+                     (#:load-path . ".")
+                     (#:branch . "master")
+                     (#:no-compile? . #t))
+                    ((#:name . "custom-packages")
+                     (#:url . "git://git.example.org/custom-packages.git")
+                     (#:load-path . ".")
+                     (#:branch . "master")
+                     (#:no-compile? . #t)))))))
+
+(service cuirass-service-type
+         (cuirass-configuration
+          (specifications %cuirass-specs)))
 @end example
 
 While information related to build jobs is located directly in the
@@ -17747,7 +17767,7 @@ Owner's group of the @code{cuirass} process.
 Number of seconds between the poll of the repositories followed by the
 Cuirass jobs.
 
-@item @code{database} (default: @code{"/var/run/cuirass/cuirass.db"})
+@item @code{database} (default: @code{"/var/lib/cuirass/cuirass.db"})
 Location of sqlite database which contains the build results and previously
 added specifications.
 
-- 
2.18.0

Message #10 received at 32547-done <at> debbugs.gnu.org (full text, mbox):

From: Clément Lassieur <clement <at> lassieur.org>
To: 32547-done <at> debbugs.gnu.org
Subject: Re: [bug#32547] [PATCH] hydra: Add support for manifests.
Date: Thu, 13 Sep 2018 10:19:36 +0200

Clément Lassieur <clement <at> lassieur.org> writes:

> * build-aux/hydra/gnu-system.scm (arguments->manifests, manifests->packages):
> New procedures.
> (hydra-jobs): Add a "manifests" subset.
> * doc/guix.texi (Continuous Integration): Update accordingly.

Pushed!

Message #13 received at 32547 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Clément Lassieur <clement <at> lassieur.org>
Cc: 32547 <at> debbugs.gnu.org
Subject: Re: [bug#32547] [PATCH] hydra: Add support for manifests.
Date: Thu, 13 Sep 2018 10:34:34 +0200

Hi Clément,

Clément Lassieur <clement <at> lassieur.org> skribis:

> * build-aux/hydra/gnu-system.scm (arguments->manifests, manifests->packages):
> New procedures.
> (hydra-jobs): Add a "manifests" subset.
> * doc/guix.texi (Continuous Integration): Update accordingly.

Thanks for pushing the patch, and sorry for not commenting earlier!  I
agree this can be very useful.

I’m slightly concerned about the complexity of this file.  Maybe
sometime down the road we could split it in several proper modules, like
(guix ci jobs) that would provide procedures to define “package jobs”,
“cross-compilation jobs”, and “VM jobs”.

Thoughts?

Ludo’.

Message #16 received at 32547 <at> debbugs.gnu.org (full text, mbox):

From: Clément Lassieur <clement <at> lassieur.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 32547 <at> debbugs.gnu.org
Subject: Re: [bug#32547] [PATCH] hydra: Add support for manifests.
Date: Thu, 13 Sep 2018 11:13:43 +0200

Hi Ludovic,

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi Clément,
>
> Clément Lassieur <clement <at> lassieur.org> skribis:
>
>> * build-aux/hydra/gnu-system.scm (arguments->manifests, manifests->packages):
>> New procedures.
>> (hydra-jobs): Add a "manifests" subset.
>> * doc/guix.texi (Continuous Integration): Update accordingly.
>
> Thanks for pushing the patch, and sorry for not commenting earlier!  I
> agree this can be very useful.
>
> I’m slightly concerned about the complexity of this file.  Maybe
> sometime down the road we could split it in several proper modules, like
> (guix ci jobs) that would provide procedures to define “package jobs”,
> “cross-compilation jobs”, and “VM jobs”.

Yes, I agree!  And "test jobs" maybe?

Clément

Message #19 received at 32547 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Clément Lassieur <clement <at> lassieur.org>
Cc: 32547 <at> debbugs.gnu.org
Subject: Re: [bug#32547] [PATCH] hydra: Add support for manifests.
Date: Thu, 13 Sep 2018 18:12:53 +0200

Clément Lassieur <clement <at> lassieur.org> skribis:

> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>> Hi Clément,
>>
>> Clément Lassieur <clement <at> lassieur.org> skribis:
>>
>>> * build-aux/hydra/gnu-system.scm (arguments->manifests, manifests->packages):
>>> New procedures.
>>> (hydra-jobs): Add a "manifests" subset.
>>> * doc/guix.texi (Continuous Integration): Update accordingly.
>>
>> Thanks for pushing the patch, and sorry for not commenting earlier!  I
>> agree this can be very useful.
>>
>> I’m slightly concerned about the complexity of this file.  Maybe
>> sometime down the road we could split it in several proper modules, like
>> (guix ci jobs) that would provide procedures to define “package jobs”,
>> “cross-compilation jobs”, and “VM jobs”.
>
> Yes, I agree!  And "test jobs" maybe?

Indeed.  Well, food for thought!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.