From unknown Fri Jun 20 07:18:59 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#53140 <53140@debbugs.gnu.org> To: bug#53140 <53140@debbugs.gnu.org> Subject: Status: [PATCH] services: guix: Add nar-herder-service-type. Reply-To: bug#53140 <53140@debbugs.gnu.org> Date: Fri, 20 Jun 2025 14:18:59 +0000 retitle 53140 [PATCH] services: guix: Add nar-herder-service-type. reassign 53140 guix-patches submitter 53140 Christopher Baines severity 53140 normal tag 53140 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 09 07:00:23 2022 Received: (at submit) by debbugs.gnu.org; 9 Jan 2022 12:00:23 +0000 Received: from localhost ([127.0.0.1]:49331 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n6Ws2-0007Pn-EB for submit@debbugs.gnu.org; Sun, 09 Jan 2022 07:00:23 -0500 Received: from lists.gnu.org ([209.51.188.17]:60092) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n6Wry-0007Pe-PB for submit@debbugs.gnu.org; Sun, 09 Jan 2022 07:00:21 -0500 Received: from eggs.gnu.org ([209.51.188.92]:45544) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n6Wry-0005Bv-Gz for guix-patches@gnu.org; Sun, 09 Jan 2022 07:00:18 -0500 Received: from mira.cbaines.net ([212.71.252.8]:36244) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n6Wrv-0001vw-Rl for guix-patches@gnu.org; Sun, 09 Jan 2022 07:00:18 -0500 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id C792F27BBE9 for ; Sun, 9 Jan 2022 12:00:12 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id f5e1342c for ; Sun, 9 Jan 2022 12:00:12 +0000 (UTC) From: Christopher Baines To: guix-patches@gnu.org Subject: [PATCH] services: guix: Add nar-herder-service-type. Date: Sun, 9 Jan 2022 12:00:12 +0000 Message-Id: <20220109120012.18655-1-mail@cbaines.net> X-Mailer: git-send-email 2.34.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) * gnu/services/guix.scm (): New record type. (nar-herder-configuration, nar-herder-configuration?, nar-herder-configuration-package, nar-herder-configuration-user, nar-herder-configuration-group, nar-herder-configuration-mirror nar-herder-configuration-database nar-herder-configuration-database-dump nar-herder-configuration-host nar-herder-configuration-port nar-herder-configuration-storage nar-herder-configuration-storage-limit nar-herder-configuration-storage-nar-removal-criteria nar-herder-shepherd-services, nar-herder-activation, nar-herder-account): New procedures. (nar-herder-service-type): New variable. * gnu/tests/guix.scm (%test-nar-herder): New variable. * doc/guix.texi (Guix Services): Document the new service. --- doc/guix.texi | 72 +++++++++++++++++++++ gnu/services/guix.scm | 147 +++++++++++++++++++++++++++++++++++++++++- gnu/tests/guix.scm | 79 ++++++++++++++++++++++- 3 files changed, 296 insertions(+), 2 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 08e5bfa111..8884052956 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -33287,6 +33287,78 @@ Extra command line options for @code{guix-data-service-process-jobs}. @end table @end deftp +@subsubheading Nar Herder +The @uref{https://git.cbaines.net/guix/nar-herder/about/,Nar Herder} is +a utility for managing a collection of nars. + +@defvar {Scheme Variable} nar-herder-type +Service type for the Guix Data Service. Its value must be a +@code{nar-herder-configuration} object. The service optionally +extends the getmail service, as the guix-commits mailing list is used to +find out about changes in the Guix git repository. +@end defvar + +@deftp {Data Type} nar-herder-configuration +Data type representing the configuration of the Guix Data Service. + +@table @asis +@item @code{package} (default: @code{nar-herder}) +The Nar Herder package to use. + +@item @code{user} (default: @code{"nar-herder"}) +The system user to run the service as. + +@item @code{group} (default: @code{"nar-herder"}) +The system group to run the service as. + +@item @code{port} (default: @code{8734}) +The port to bind the server to. + +@item @code{host} (default: @code{"127.0.0.1"}) +The host to bind the server to. + +@item @code{mirror} (default: @code{#f}) +Optional URL of the other Nar Herder instance which should be mirrored. +This means that this Nar Herder instance will download it's database, +and keep it up to date. + +@item @code{database} (default: @code{"/var/lib/nar-herder/nar_herder.db"}) +Location for the database. If this Nar Herder instance is mirroring +another, the database will be downloaded if it doesn't exist. If this +Nar Herder instance isn't mirroring another, an empty database will be +created. + +@item @code{database-dump} (default: @code{"/var/lib/nar-herder/nar_herder_dump.db"}) +Location of the database dump. This is created and regularly updated by +taking a copy of the database. This is the version of the database that +is available to download. + +@item @code{storage} (default: @code{#f}) +Optional location in which to store nars. + +@item @code{storage-limit} (default: @code{"none"}) +Limit in bytes for the nars stored in the storage location. This can +also be set to ``none'' so that there is no limit. + +When the storage location exceeds this size, nars are removed according +to the nar removal criteria. + +@item @code{storage-nar-removal-criteria} (default: @code{'()}) +Criteria used to remove nars from the storage location. These are used +in conjunction with the storage limit. + +When the storage location exceeds the storage limit size, nars will be +checked against the nar removal criteria and if any of the criteria +match, they will be removed. This will continue until the storage +location is below the storage limit size. + +Each criteria is specified by a string, then an equals sign, then +another string. Currently, only one criteria is supported, checking if a +nar is stored on another Nar Herder instance. + +@end table +@end deftp + @node Linux Services @subsection Linux Services diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm index df5fa13bea..930a78bf3c 100644 --- a/gnu/services/guix.scm +++ b/gnu/services/guix.scm @@ -107,7 +107,22 @@ (define-module (gnu services guix) guix-data-service-getmail-idle-mailboxes guix-data-service-commits-getmail-retriever-configuration - guix-data-service-type)) + guix-data-service-type + + nar-herder-service-type + nar-herder-configuration + nar-herder-configuration? + nar-herder-configuration-package + nar-herder-configuration-user + nar-herder-configuration-group + nar-herder-configuration-mirror + nar-herder-configuration-database + nar-herder-configuration-database-dump + nar-herder-configuration-host + nar-herder-configuration-port + nar-herder-configuration-storage + nar-herder-configuration-storage-limit + nar-herder-configuration-storage-nar-removal-criteria)) ;;;; Commentary: ;;; @@ -728,3 +743,133 @@ (define guix-data-service-type (guix-data-service-configuration)) (description "Run an instance of the Guix Data Service."))) + + +;;; +;;; Nar Herder +;;; + +(define-record-type* + nar-herder-configuration make-nar-herder-configuration + nar-herder-configuration? + (package nar-herder-configuration-package + (default nar-herder)) + (user nar-herder-configuration-user + (default "nar-herder")) + (group nar-herder-configuration-group + (default "nar-herder")) + (mirror nar-herder-configuration-mirror + (default #f)) + (database nar-herder-configuration-database + (default "/var/lib/nar-herder/nar_herder.db")) + (database-dump nar-herder-configuration-database-dump + (default "/var/lib/nar-herder/nar_herder_dump.db")) + (host nar-herder-configuration-host + (default "127.0.0.1")) + (port nar-herder-configuration-port + (default 8734)) + (storage nar-herder-configuration-storage + (default #f)) + (storage-limit nar-herder-configuration-storage-limit + (default "none")) + (storage-nar-removal-criteria + nar-herder-configuration-storage-nar-removal-criteria + (default '()))) + +(define (nar-herder-shepherd-services config) + (match-record config + (package user group + mirror + database database-dump + host port + storage storage-limit storage-nar-removal-criteria) + + (unless (or mirror storage) + (error "nar-herder: mirror or storage must be set")) + + (list + (shepherd-service + (documentation "Nar Herder") + (provision '(nar-herder)) + (requirement '(networking)) + (start #~(make-forkexec-constructor + (list #$(file-append package + "/bin/nar-herder") + "run-server" + "--pid-file=/var/run/nar-herder/pid" + #$(string-append "--port=" (number->string port)) + #$(string-append "--host=" host) + #$@(if mirror + (list (string-append "--mirror=" mirror)) + '()) + #$(string-append "--database=" database) + #$(string-append "--database-dump=" database-dump) + #$@(if storage + (list (string-append "--storage=" storage)) + '()) + #$(string-append "--storage-limit=" + (if (number? storage-limit) + (number->string storage-limit) + storage-limit)) + #$@(map (lambda (criteria) + (string-append + "--storage-nar-removal-criteria=" + (match criteria + ((k . v) (simple-format #f "~A=~A" k v)) + (str str)))) + storage-nar-removal-criteria)) + #:user #$user + #:group #$group + #:pid-file "/var/run/nar-herder/pid" + #:environment-variables + `(,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") + #:log-file "/var/log/nar-herder/server.log")) + (stop #~(make-kill-destructor)))))) + +(define (nar-herder-activation config) + #~(begin + (use-modules (guix build utils)) + + (define %user + (getpw #$(nar-herder-configuration-user + config))) + + (chmod "/var/lib/nar-herder" #o755) + + (mkdir-p "/var/log/nar-herder") + + ;; Allow writing the PID file + (mkdir-p "/var/run/nar-herder") + (chown "/var/run/nar-herder" + (passwd:uid %user) + (passwd:gid %user)))) + +(define (nar-herder-account config) + (match-record config + (user group) + (list (user-group + (name group) + (system? #t)) + (user-account + (name user) + (group group) + (system? #t) + (comment "Nar Herder user") + (home-directory "/var/lib/nar-herder") + (shell (file-append shadow "/sbin/nologin")))))) + +(define nar-herder-service-type + (service-type + (name 'nar-herder) + (extensions + (list + (service-extension shepherd-root-service-type + nar-herder-shepherd-services) + (service-extension activation-service-type + nar-herder-activation) + (service-extension account-service-type + nar-herder-account))) + (description + "Run a Nar Herder server."))) diff --git a/gnu/tests/guix.scm b/gnu/tests/guix.scm index 69cac7c1aa..0209767cd2 100644 --- a/gnu/tests/guix.scm +++ b/gnu/tests/guix.scm @@ -36,7 +36,8 @@ (define-module (gnu tests guix) #:use-module (guix utils) #:use-module (ice-9 match) #:export (%test-guix-build-coordinator - %test-guix-data-service)) + %test-guix-data-service + %test-nar-herder)) ;;; ;;; Guix Build Coordinator @@ -239,3 +240,79 @@ (define %test-guix-data-service (name "guix-data-service") (description "Connect to a running Guix Data Service.") (value (run-guix-data-service-test)))) + + +;;; +;;; Nar Herder +;;; + +(define %nar-herder-os + (simple-operating-system + (service dhcp-client-service-type) + (service nar-herder-service-type + (nar-herder-configuration + (host "0.0.0.0") + ;; Not a realistic value, but works for the test + (storage "/tmp"))))) + +(define (run-nar-herder-test) + (define os + (marionette-operating-system + %nar-herder-os + #:imported-modules '((gnu services herd) + (guix combinators)))) + + (define forwarded-port + (nar-herder-configuration-port + (nar-herder-configuration))) + + (define vm + (virtual-machine + (operating-system os) + (memory-size 1024) + (port-forwardings `((,forwarded-port . ,forwarded-port))))) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-11) (srfi srfi-64) + (gnu build marionette) + (web uri) + (web client) + (web response)) + + (define marionette + (make-marionette (list #$vm))) + + (test-runner-current (system-test-runner #$output)) + (test-begin "nar-herder") + + (test-assert "service running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'nar-herder) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (test-equal "http-get" + 404 + (let-values + (((response text) + (http-get #$(simple-format + #f "http://localhost:~A/" forwarded-port) + #:decode-body? #t))) + (response-code response))) + + (test-end)))) + + (gexp->derivation "nar-herder-test" test)) + +(define %test-nar-herder + (system-test + (name "nar-herder") + (description "Connect to a running Nar Herder server.") + (value (run-nar-herder-test)))) -- 2.34.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 31 13:31:01 2022 Received: (at 53140-done) by debbugs.gnu.org; 31 Jan 2022 18:31:01 +0000 Received: from localhost ([127.0.0.1]:40739 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nEbS9-0005Es-5k for submit@debbugs.gnu.org; Mon, 31 Jan 2022 13:31:01 -0500 Received: from mira.cbaines.net ([212.71.252.8]:41266) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nEbS6-0005Ej-Nf for 53140-done@debbugs.gnu.org; Mon, 31 Jan 2022 13:30:59 -0500 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id C12DC27BBE9 for <53140-done@debbugs.gnu.org>; Mon, 31 Jan 2022 18:30:57 +0000 (GMT) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 1dab9403 for <53140-done@debbugs.gnu.org>; Mon, 31 Jan 2022 18:30:56 +0000 (UTC) References: <20220109120012.18655-1-mail@cbaines.net> User-agent: mu4e 1.6.10; emacs 27.2 From: Christopher Baines To: 53140-done@debbugs.gnu.org Subject: Re: [bug#53140] [PATCH] services: guix: Add nar-herder-service-type. Date: Mon, 31 Jan 2022 18:30:31 +0000 In-reply-to: <20220109120012.18655-1-mail@cbaines.net> Message-ID: <875ypzn4f3.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 53140-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Christopher Baines writes: > * gnu/services/guix.scm (): New record type. > (nar-herder-configuration, nar-herder-configuration?, > nar-herder-configuration-package, > nar-herder-configuration-user, > nar-herder-configuration-group, > nar-herder-configuration-mirror > nar-herder-configuration-database > nar-herder-configuration-database-dump > nar-herder-configuration-host > nar-herder-configuration-port > nar-herder-configuration-storage > nar-herder-configuration-storage-limit > nar-herder-configuration-storage-nar-removal-criteria > nar-herder-shepherd-services, nar-herder-activation, > nar-herder-account): New procedures. > (nar-herder-service-type): New variable. > * gnu/tests/guix.scm (%test-nar-herder): New variable. > * doc/guix.texi (Guix Services): Document the new service. > --- > doc/guix.texi | 72 +++++++++++++++++++++ > gnu/services/guix.scm | 147 +++++++++++++++++++++++++++++++++++++++++- > gnu/tests/guix.scm | 79 ++++++++++++++++++++++- > 3 files changed, 296 insertions(+), 2 deletions(-) Merged to master as 087cdafc9f8ef1d73780ab3e0b4dd340b9e0bce0. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmH4KuBfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XeTjhAAkEPR5Dak5wLgZtBiIU5vU7IgQ0Ps84RP aDz0uRNP26ij0S7rwmbTr7/TLRyyRKl5q38vN8NwwM75Xfk0QRV3Go3wcMQ5KBrG 3Hvp2QUwVsRfBtdCkyYD81lCVng+krPHVMPE5gtFP+HL7sx1Omw4mNe7kJNiPjNP fRU5EKQXNMvyiB1FNFtIdJJLc2sp7VOjnLlIrPikJ9+ygCs/JBPEOYvcwpVGp7y8 vN3N/P1qA4m+SqMDKJzU+h6D/+ISVNtdfwf9xHJfJE044d0PEl1gsBPM6wl2p0/g KsPh11BSKSDYiBq5XOBwatM5A6/lu9i3Teq20K6Aiu94/K9GZfa0sat/btxgseCg Ca23A8UfRbO4SB88udnbP/4l2NPjE1Nda9wBVStD27g7srUivTuYd2IvGp0YDMJo CPd/g4g6ls8SqJgW8fHUnb1zNNbOh4/8eF3+w61p+En59sutahuxhzQNJ012E56h 3nqy5MwwLgduF6Gjah48nlfs7JpMJ9SQMrQP4AfiXEnL/u6Ouf9RgKRkHYQQ3tkR ppGzsSFnSOFTSKqe10k77Ad0ZTac3hKScGr++78uC/3LMo62KMqgdYWC9QmdlGVq RUr9l5B5dlSdy//ezb1cYYXYJvStK03VS1sn/GXYvj/Be22VCzLe8LDv0l2Yb4h5 KBu9ccB/amw= =BRiN -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Jun 20 07:18:59 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 01 Mar 2022 12:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator