From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 03 22:15:51 2023 Received: (at submit) by debbugs.gnu.org; 4 Mar 2023 03:15:51 +0000 Received: from localhost ([127.0.0.1]:35013 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pYINB-0001fN-HK for submit@debbugs.gnu.org; Fri, 03 Mar 2023 22:15:51 -0500 Received: from lists.gnu.org ([209.51.188.17]:37098) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pYIN9-0001fF-Mn for submit@debbugs.gnu.org; Fri, 03 Mar 2023 22:15:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYIN9-0005If-Eh for guix-patches@gnu.org; Fri, 03 Mar 2023 22:15:47 -0500 Received: from mail-qv1-xf31.google.com ([2607:f8b0:4864:20::f31]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pYIN3-0000hy-H1 for guix-patches@gnu.org; Fri, 03 Mar 2023 22:15:46 -0500 Received: by mail-qv1-xf31.google.com with SMTP id g9so1388016qvt.8 for ; Fri, 03 Mar 2023 19:15:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Ca7UgmTmyEZ0Gb1P9L25PYwbuWbEkWkYryWTD6PEB18=; b=P+b0ggMvmQxzA/hdccGEK9x0P/Qv/ucU9mrg7As96xHmgbx2AT9BVj4rlHrbGsCAZp aqKCVqEX//hSe/HMkcAsPASyYtpw5i+84QFpOGXNoJ7AtSGWAkoYXT+Ft0sMxu18ob9f 6+Otuwqc7fFHFeSFBSy5+u7VcnBboq9VznycMOT3E65s+sVG0PPrz/u2c8dwB8mC3Cs8 iCo4zHXyeWM66sRc2U7MaGbjv32omTz9rB5DKLLUbjufWl4jK/bDi3c53SOgSIDxWdjz +wH0uY2QH62U5OaMf9ZcpCeeicrxPCazNXraFoKFOV06Af9Mj/xWrRcERIoVBk2N3Neb 7FfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Ca7UgmTmyEZ0Gb1P9L25PYwbuWbEkWkYryWTD6PEB18=; b=J3dwBvtViskenYoS86J/WhW+u25cmqer4JMz3Rs4FjGPKsZm5OZ9HNXGQ1ke4KLV// v3mF0pmItYpop59K7BLV1TyHlbM7Ca4q9KEIcwmA8QzGSBdfRgtsJ4A2724OKtEHvFmj jI56S16Y5QV9BBmxPH/LgxQRPEUt4v71JUxZA7iYF/aSp5aY9IS+qPB14jSi8QoqzqsW ZR99oiEJUGFrK0Tl2icDEhMRgyXbJwmALXFb8Tge5duqLrQiVZsKBNG/hfD+akNgyxiN 6AbCVeL7FjXQWWU7whTBoGqzivzUHds7Ll0MQOW07Nu3a5jACKqhaXQYg3tf/69ln+o3 koSQ== X-Gm-Message-State: AO0yUKXFdl4QicXsga6uhgFV42XfVLbFWsixaE8Hw6fX6k39rkc9Bhq1 bU0P37hgYa1zQz1q3qfWMKOa3HPzXysqRQ== X-Google-Smtp-Source: AK7set+kpxknzXED64uwY9MM447GH7aUA1sb+lkXM/VTocd355+Qs/qQBe+L77zHIwbgIZ+rZ4VBYA== X-Received: by 2002:a05:6214:4109:b0:56f:378:954 with SMTP id kc9-20020a056214410900b0056f03780954mr13524122qvb.3.1677899738922; Fri, 03 Mar 2023 19:15:38 -0800 (PST) Received: from localhost.localdomain (dsl-10-129-180.b2b2c.ca. [72.10.129.180]) by smtp.gmail.com with ESMTPSA id t33-20020a05622a182100b003b646123691sm3057169qtc.31.2023.03.03.19.15.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Mar 2023 19:15:38 -0800 (PST) From: Maxim Cournoyer To: guix-patches@gnu.org Subject: [PATCH] pack: Move common build code to (guix build pack). Date: Fri, 3 Mar 2023 22:15:23 -0500 Message-Id: <20230304031523.24102-1-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.39.1 MIME-Version: 1.0 X-Debbugs-Cc: zimon.toutoune@gmail.com X-Debbugs-Cc: dev@jpoiret.xyz X-Debbugs-Cc: mail@cbaines.net X-Debbugs-Cc: rekado@elephly.net X-Debbugs-Cc: othacehe@gnu.org X-Debbugs-Cc: me@tobias.gr X-Debbugs-Cc: ludo@gnu.org Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=2607:f8b0:4864:20::f31; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qv1-xf31.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: Maxim Cournoyer X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) The rationale is to reduce the number of derivations built per pack to ideally one, to minimize storage requirements. The number of derivations had gone up with 68380db4 ("pack: Extract populate-profile-root from self-contained-tarball/builder.") as a side effect to improving code reuse. * guix/scripts/pack.scm (guix): Add commentary comment. (populate-profile-root, self-contained-tarball/builder): Extract to... * guix/build/pack.scm (populate-profile-root!): ... this, and... (build-self-contained-tarball): ... that, adjusting for use on the build side. (assert-utf8-locale): New procedure. (self-contained-tarball, debian-archive, rpm-archive): Adjust accordingly. --- guix/build/pack.scm | 115 +++++++++++++- guix/scripts/pack.scm | 341 +++++++++++++++--------------------------- tests/pack.scm | 104 ++++++------- 3 files changed, 284 insertions(+), 276 deletions(-) diff --git a/guix/build/pack.scm b/guix/build/pack.scm index 3b73d1b227..fa9a5f5905 100644 --- a/guix/build/pack.scm +++ b/guix/build/pack.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021, 2023 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -16,9 +16,26 @@ ;;; You should have received a copy of the GNU General Public License ;;; along with GNU Guix. If not, see . +;;; Commentary: + +;;; This module contains build-side common procedures used by the host-side +;;; (guix scripts pack) module, mostly to allow for code reuse. Due to making +;;; use of the (guix build store-copy) module, it transitively requires the +;;; sqlite and gcrypt extensions to be available. + +;;; Code: + (define-module (guix build pack) + #:use-module (gnu build install) #:use-module (guix build utils) - #:export (tar-base-options)) + #:use-module (guix build store-copy) + #:use-module ((guix build union) #:select (relative-file-name)) + #:use-module (ice-9 match) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:export (tar-base-options + populate-profile-root! + build-self-contained-tarball)) (define* (tar-base-options #:key tar compressor) "Return the base GNU tar options required to produce deterministic archives @@ -52,3 +69,97 @@ (define (tar-supports-sort? tar) ;; process. Use '--hard-dereference' to eliminate it. "--hard-dereference" "--check-links")) + +(define (assert-utf8-locale) + "Verify the current process is using the en_US.utf8 locale." + (unless (string=? "unset for tests" (getenv "GUIX_LOCPATH")) + (unless (false-if-exception (setlocale LC_ALL "en_US.utf8")) + (error "environment not configured for en_US.utf8 locale")))) + +(define* (populate-profile-root! profile + #:key (profile-name "guix-profile") + localstatedir? + store-database + deduplicate? + (symlinks '())) + "Populate the root profile directory with SYMLINKS and a Guix database, when +LOCALSTATEDIR? is set, and a pre-computed STORE-DATABASE is provided. The +directory is created as \"root\" in the current working directory. When +DEDUPLICATE? is true, deduplicate the store items, which relies on hard +links. It needs to run in an environment where " + (when localstatedir? + (unless store-database + (error "missing STORE-DATABASE argument"))) + + (define symlink->directives + ;; Return "populate directives" to make the given symlink and its + ;; parent directories. + (match-lambda + ((source '-> target) + (let ((target (string-append profile "/" target)) + (parent (dirname source))) + ;; Never add a 'directory' directive for "/" so as to + ;; preserve its ownership when extracting the archive (see + ;; below), and also because this would lead to adding the + ;; same entries twice in the tarball. + `(,@(if (string=? parent "/") + '() + `((directory ,parent))) + ;; Use a relative file name for compatibility with + ;; relocatable packs. + (,source -> ,(relative-file-name parent target))))))) + + (define directives + ;; Fully-qualified symlinks. + (append-map symlink->directives symlinks)) + + (define %root "root") + + (assert-utf8-locale) + + ;; Note: there is not much to gain here with deduplication and there + ;; is the overhead of the '.links' directory, so turn it off by + ;; default. Furthermore GNU tar < 1.30 sometimes fails to extract + ;; tarballs with hard links: + ;; . + (populate-store (list "profile") %root #:deduplicate? deduplicate?) + + (when localstatedir? + (install-database-and-gc-roots %root store-database + profile #:profile-name profile-name)) + + ;; Create SYMLINKS. + (for-each (cut evaluate-populate-directive <> %root) directives)) + +(define* (build-self-contained-tarball profile + tarball-file-name + #:key (profile-name "guix-profile") + target + localstatedir? + store-database + deduplicate? + symlinks + compressor-command + archiver) + "Create a self-contained tarball TARBALL-FILE-NAME from PROFILE, optionally +compressing it with COMPRESSOR-COMMAND, the complete command-line string to +use for the compressor." + (assert-utf8-locale) + + (populate-profile-root! profile + #:profile-name profile-name + #:localstatedir? localstatedir? + #:store-database store-database + #:deduplicate? deduplicate? + #:symlinks symlinks) + + (define tar (string-append archiver "/bin/tar")) + + ;; GNU Tar recurses directories by default. Simply add the whole root + ;; directory, which contains all the files to be archived. This avoids + ;; creating duplicate files in the archives that would be stored as hard + ;; links by GNU Tar. + (apply invoke tar "-cvf" tarball-file-name "-C" "root" "." + (tar-base-options + #:tar tar + #:compressor compressor-command))) diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index eb41eb5563..984622bd16 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -24,6 +24,14 @@ ;;; You should have received a copy of the GNU General Public License ;;; along with GNU Guix. If not, see . +;;; Commentary: + +;;; This module implements the 'guix pack' command and the various supported +;;; formats. Where feasible, the builders of the packs should be implemented +;;; as single derivations to minimize storage requirements. + +;;; Code: + (define-module (guix scripts pack) #:use-module (guix scripts) #:use-module (guix ui) @@ -199,153 +207,18 @@ (define (set-utf8-locale profile) "Configure the environment to use the \"en_US.utf8\" locale provided by the GLIBC-UT8-LOCALES package." ;; Arrange to not depend on 'glibc-utf8-locales' when using '--bootstrap'. - (and (or (not (profile? profile)) - (profile-locales? profile)) - #~(begin - (setenv "GUIX_LOCPATH" - #+(file-append glibc-utf8-locales "/lib/locale")) - (setlocale LC_ALL "en_US.utf8")))) - -(define* (populate-profile-root profile - #:key (profile-name "guix-profile") - target - localstatedir? - deduplicate? - (symlinks '())) - "Populate the root profile directory with SYMLINKS and a Guix database, when -LOCALSTATEDIR? is set. When DEDUPLICATE? is true, deduplicate the store -items, which relies on hard links." - (define database - (and localstatedir? - (file-append (store-database (list profile)) - "/db/db.sqlite"))) - - (define bootstrap? - ;; Whether a '--bootstrap' environment is needed, for testing purposes. - ;; XXX: Infer that from available info. - (and (not database) (not (profile-locales? profile)))) - - (define (import-module? module) - ;; Since we don't use deduplication support in 'populate-store', don't - ;; import (guix store deduplication) and its dependencies, which includes - ;; Guile-Gcrypt, unless DEDUPLICATE? is #t. This makes it possible to run - ;; tests with '--bootstrap'. - (and (not-config? module) - (or deduplicate? (not (equal? '(guix store deduplication) module))))) - - (computed-file "profile-directory" - (with-imported-modules (source-module-closure - `((guix build pack) - (guix build store-copy) - (guix build utils) - (guix build union) - (gnu build install)) - #:select? import-module?) + (if (or (not (profile? profile)) + (profile-locales? profile)) #~(begin - (use-modules (guix build pack) - (guix build store-copy) - (guix build utils) - ((guix build union) #:select (relative-file-name)) - (gnu build install) - (srfi srfi-1) - (srfi srfi-26) - (ice-9 match)) - - (define symlink->directives - ;; Return "populate directives" to make the given symlink and its - ;; parent directories. - (match-lambda - ((source '-> target) - (let ((target (string-append #$profile "/" target)) - (parent (dirname source))) - ;; Never add a 'directory' directive for "/" so as to - ;; preserve its ownership when extracting the archive (see - ;; below), and also because this would lead to adding the - ;; same entries twice in the tarball. - `(,@(if (string=? parent "/") - '() - `((directory ,parent))) - ;; Use a relative file name for compatibility with - ;; relocatable packs. - (,source -> ,(relative-file-name parent target))))))) - - (define directives - ;; Fully-qualified symlinks. - (append-map symlink->directives '#$symlinks)) - - ;; Make sure non-ASCII file names are properly handled. - #+(set-utf8-locale profile) - - ;; Note: there is not much to gain here with deduplication and there - ;; is the overhead of the '.links' directory, so turn it off by - ;; default. Furthermore GNU tar < 1.30 sometimes fails to extract - ;; tarballs with hard links: - ;; . - (populate-store (list "profile") #$output - #:deduplicate? #$deduplicate?) - - (when #+localstatedir? - (install-database-and-gc-roots #$output #+database #$profile - #:profile-name #$profile-name)) - - ;; Create SYMLINKS. - (for-each (cut evaluate-populate-directive <> #$output) - directives))) - #:local-build? #f - #:guile (if bootstrap? %bootstrap-guile (default-guile)) - #:options (list #:references-graphs `(("profile" ,profile)) - #:target target))) + (setenv "GUIX_LOCPATH" + #+(file-append glibc-utf8-locales "/lib/locale")) + (setlocale LC_ALL "en_US.utf8")) + #~(setenv "GUIX_LOCPATH" "unset for tests"))) ;;; ;;; Tarball format. ;;; -(define* (self-contained-tarball/builder profile - #:key (profile-name "guix-profile") - target - localstatedir? - deduplicate? - symlinks - compressor - archiver) - "Return a GEXP that can build a self-contained tarball." - - (define root (populate-profile-root profile - #:profile-name profile-name - #:target target - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks)) - - (with-imported-modules (source-module-closure '((guix build pack) - (guix build utils))) - #~(begin - (use-modules (guix build pack) - (guix build utils)) - - ;; Make sure non-ASCII file names are properly handled. - #+(set-utf8-locale profile) - - (define tar #+(file-append archiver "/bin/tar")) - - (define %root (if #$localstatedir? "." #$root)) - - (when #$localstatedir? - ;; Fix the permission of the Guix database file, which was made - ;; read-only when copied to the store in populate-profile-root. - (copy-recursively #$root %root) - (chmod (string-append %root "/var/guix/db/db.sqlite") #o644)) - - (with-directory-excursion %root - ;; GNU Tar recurses directories by default. Simply add the whole - ;; current directory, which contains all the files to be archived. - ;; This avoids creating duplicate files in the archives that would - ;; be stored as hard links by GNU Tar. - (apply invoke tar "-cvf" #$output "." - (tar-base-options - #:tar tar - #:compressor #+(and=> compressor compressor-command))))))) - (define* (self-contained-tarball name profile #:key target (profile-name "guix-profile") @@ -367,16 +240,39 @@ (define* (self-contained-tarball name profile (warning (G_ "entry point not supported in the '~a' format~%") 'tarball)) + (define database + (and localstatedir? + (file-append (store-database (list profile)) + "/db/db.sqlite"))) + (gexp->derivation (string-append name ".tar" (compressor-extension compressor)) - (self-contained-tarball/builder profile - #:profile-name profile-name - #:target target - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks - #:compressor compressor - #:archiver archiver))) + (with-extensions (list guile-gcrypt) + (with-imported-modules `(((guix config) => ,(make-config.scm)) + ,@(source-module-closure + `((guix build pack) + (guix build utils)) + #:select? not-config?)) + #~(begin + (use-modules (guix build pack) + (guix build utils)) + + ;; Make sure non-ASCII file names are properly handled. + #+(set-utf8-locale profile) + + (build-self-contained-tarball #$profile + #$output + #:profile-name #$profile-name + #:target #$target + #:localstatedir? #$localstatedir? + #:store-database #+database + #:deduplicate? #$deduplicate? + #:symlinks '#$symlinks + #:compressor-command + #+(and=> compressor compressor-command) + #:archiver #+archiver)))) + #:target target + #:references-graphs `(("profile" ,profile)))) ;;; @@ -721,20 +617,10 @@ (define %valid-compressors '("gzip" "xz" "none")) (warning (G_ "entry point not supported in the '~a' format~%") 'deb)) - (define data-tarball - (computed-file (string-append "data.tar" (compressor-extension - compressor)) - (self-contained-tarball/builder profile - #:target target - #:profile-name profile-name - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks - #:compressor compressor - #:archiver archiver) - #:local-build? #f ;allow offloading - #:options (list #:references-graphs `(("profile" ,profile)) - #:target target))) + (define database + (and localstatedir? + (file-append (store-database (list profile)) + "/db/db.sqlite"))) (define build (with-extensions (list guile-gcrypt) @@ -752,6 +638,9 @@ (define build (ice-9 optargs) (srfi srfi-1)) + ;; Make sure non-ASCII file names are properly handled. + #+(set-utf8-locale profile) + (define machine-type ;; Extract the machine type from the specified target, else from the ;; current system. @@ -805,10 +694,25 @@ (define debian-format-version "2.0") (lambda (port) (format port "~a~%" debian-format-version))) - (define data-tarball-file-name (strip-store-file-name - #+data-tarball)) + (define compressor-command + #+(and=> compressor compressor-command)) - (copy-file #+data-tarball data-tarball-file-name) + (define compressor-extension + #+(compressor-extension compressor)) + + (define data-tarball-file-name + (string-append "data.tar" compressor-extension)) + + (build-self-contained-tarball #$profile + data-tarball-file-name + #:profile-name #$profile-name + #:localstatedir? #$localstatedir? + #:store-database #+database + #:deduplicate? #$deduplicate? + #:symlinks '#$symlinks + #:compressor-command + compressor-command + #:archiver #+archiver) ;; Generate the control archive. (let-keywords '#$extra-options #f @@ -817,8 +721,7 @@ (define data-tarball-file-name (strip-store-file-name (triggers-file #f)) (define control-tarball-file-name - (string-append "control.tar" - #$(compressor-extension compressor))) + (string-append "control.tar" compressor-extension)) ;; Write the compressed control tarball. Only the control file is ;; mandatory (see: 'man deb' and 'man deb-control'). @@ -848,7 +751,7 @@ (define tar (string-append #+archiver "/bin/tar")) (apply invoke tar `(,@(tar-base-options #:tar tar - #:compressor #+(and=> compressor compressor-command)) + #:compressor compressor-command) "-cvf" ,control-tarball-file-name "control" ,@(if postinst-file '("postinst") '()) @@ -859,7 +762,9 @@ (define tar (string-append #+archiver "/bin/tar")) "debian-binary" control-tarball-file-name data-tarball-file-name)))))) - (gexp->derivation (string-append name ".deb") build)) + (gexp->derivation (string-append name ".deb") build + #:target target + #:references-graphs `(("profile" ,profile)))) ;;; @@ -883,66 +788,27 @@ (define* (rpm-archive name profile (when entry-point (warning (G_ "entry point not supported in the '~a' format~%") 'rpm)) - (define root (populate-profile-root profile - #:profile-name profile-name - #:target target - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks)) - - (define payload - (let* ((raw-cpio-file-name "payload.cpio") - (compressed-cpio-file-name (string-append raw-cpio-file-name - (compressor-extension - compressor)))) - (computed-file compressed-cpio-file-name - (with-imported-modules (source-module-closure - '((guix build utils) - (guix cpio) - (guix rpm))) - #~(begin - (use-modules (guix build utils) - (guix cpio) - (guix rpm) - (srfi srfi-1)) - - ;; Make sure non-ASCII file names are properly handled. - #+(set-utf8-locale profile) - - (define %root (if #$localstatedir? "." #$root)) - - (when #$localstatedir? - ;; Fix the permission of the Guix database file, which was made - ;; read-only when copied to the store in populate-profile-root. - (copy-recursively #$root %root) - (chmod (string-append %root "/var/guix/db/db.sqlite") #o644)) - - (call-with-output-file #$raw-cpio-file-name - (lambda (port) - (with-directory-excursion %root - ;; The first "." entry is discarded. - (write-cpio-archive - (remove fhs-directory? - (cdr (find-files "." #:directories? #t))) - port)))) - (when #+(compressor-command compressor) - (apply invoke (append #+(compressor-command compressor) - (list #$raw-cpio-file-name)))) - (copy-file #$compressed-cpio-file-name #$output))) - #:local-build? #f))) ;allow offloading + (define database + (and localstatedir? + (file-append (store-database (list profile)) + "/db/db.sqlite"))) (define build (with-extensions (list guile-gcrypt) (with-imported-modules `(((guix config) => ,(make-config.scm)) ,@(source-module-closure `((gcrypt hash) + (guix build pack) (guix build utils) + (guix cpio) (guix profiles) (guix rpm)) #:select? not-config?)) #~(begin (use-modules (gcrypt hash) + (guix build pack) (guix build utils) + (guix cpio) (guix profiles) (guix rpm) (ice-9 binary-ports) @@ -954,6 +820,35 @@ (define build ;; Make sure non-ASCII file names are properly handled. #+(set-utf8-locale profile) + (define %root "root") + + (populate-profile-root! #$profile + #:profile-name #$profile-name + #:localstatedir? #$localstatedir? + #:store-database #+database + #:deduplicate? #$deduplicate? + #:symlinks '#$symlinks) + + (define raw-cpio-file-name "payload.cpio") + + ;; Generate CPIO payload. + (call-with-output-file raw-cpio-file-name + (lambda (port) + (with-directory-excursion %root + ;; The first "." entry is discarded. + (write-cpio-archive + (remove fhs-directory? + (cdr (find-files "." #:directories? #t))) + port)))) + + (when #+(compressor-command compressor) + (apply invoke (append #+(compressor-command compressor) + (list raw-cpio-file-name)))) + + (define cpio-file-name + (string-append "payload.cpio" + #$(compressor-extension compressor))) + (define machine-type (and=> (or #$target %host-type) (lambda (triplet) @@ -981,7 +876,7 @@ (define lead #:target (or #$target %host-type))) (define payload-digest - (bytevector->hex-string (file-sha256 #$payload))) + (bytevector->hex-string (file-sha256 cpio-file-name))) (let-keywords '#$extra-options #f ((relocatable? #f) (prein-file #f) @@ -991,7 +886,7 @@ (define payload-digest (let ((header (generate-header name version payload-digest - #$root + %root #$(compressor-name compressor) #:target (or #$target %host-type) #:relocatable? relocatable? @@ -1003,7 +898,7 @@ (define payload-digest (define header-sha256 (bytevector->hex-string (sha256 (u8-list->bytevector header)))) - (define payload-size (stat:size (stat #$payload))) + (define payload-size (stat:size (stat cpio-file-name))) (define header+compressed-payload-size (+ (length header) payload-size)) @@ -1013,7 +908,7 @@ (define signature header+compressed-payload-size)) ;; Serialize the archive components to a file. - (call-with-input-file #$payload + (call-with-input-file cpio-file-name (lambda (in) (call-with-output-file #$output (lambda (out) @@ -1022,7 +917,9 @@ (define signature header)) (sendfile out in payload-size))))))))))) - (gexp->derivation (string-append name ".rpm") build)) + (gexp->derivation (string-append name ".rpm") build + #:target target + #:references-graphs `(("profile" ,profile)))) ;;; diff --git a/tests/pack.scm b/tests/pack.scm index 87187bb62c..397fb37b12 100644 --- a/tests/pack.scm +++ b/tests/pack.scm @@ -76,64 +76,64 @@ (define rpm-for-tests (test-begin "pack") -(unless (network-reachable?) (test-skip 1)) -(test-assertm "self-contained-tarball" %store - (mlet* %store-monad - ((profile -> (profile - (content (packages->manifest (list %bootstrap-guile))) - (hooks '()) - (locales? #f))) - (tarball (self-contained-tarball "pack" profile - #:symlinks '(("/bin/Guile" - -> "bin/guile")) - #:compressor %gzip-compressor - #:archiver %tar-bootstrap)) - (check (gexp->derivation "check-tarball" - (with-imported-modules '((guix build utils)) - #~(begin - (use-modules (guix build utils) - (srfi srfi-1)) - - (define store - ;; The unpacked store. - (string-append "." (%store-directory) "/")) - - (define (canonical? file) - ;; Return #t if FILE is read-only and its mtime is 1. - (let ((st (lstat file))) - (or (not (string-prefix? store file)) - (eq? 'symlink (stat:type st)) - (and (= 1 (stat:mtime st)) - (zero? (logand #o222 - (stat:mode st))))))) - - (define bin - (string-append "." #$profile "/bin")) - - (setenv "PATH" - (string-append #$%tar-bootstrap "/bin")) - (system* "tar" "xvf" #$tarball) - (mkdir #$output) - (exit - (and (file-exists? (string-append bin "/guile")) - (file-exists? store) - (every canonical? - (find-files "." (const #t) - #:directories? #t)) - (string=? (string-append #$%bootstrap-guile "/bin") - (readlink bin)) - (string=? (string-append ".." #$profile - "/bin/guile") - (readlink "bin/Guile"))))))))) - (built-derivations (list check)))) - ;; The following test needs guile-sqlite3, libgcrypt, etc. as a consequence of ;; commit c45477d2a1a651485feede20fe0f3d15aec48b39 and related changes. Thus, ;; run it on the user's store, if it's available, on the grounds that these ;; dependencies may be already there, or we can get substitutes or build them ;; quite inexpensively; see . - (with-external-store store + (unless store (test-skip 1)) + (test-assertm "self-contained-tarball" store + (mlet* %store-monad + ((guile (set-guile-for-build (default-guile))) + (profile -> (profile + (content (packages->manifest (list %bootstrap-guile))) + (hooks '()) + (locales? #f))) + (tarball (self-contained-tarball "pack" profile + #:symlinks '(("/bin/Guile" + -> "bin/guile")) + #:compressor %gzip-compressor + #:archiver %tar-bootstrap)) + (check (gexp->derivation "check-tarball" + (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils) + (srfi srfi-1)) + + (define store + ;; The unpacked store. + (string-append "." (%store-directory) "/")) + + (define (canonical? file) + ;; Return #t if FILE is read-only and its mtime is 1. + (let ((st (lstat file))) + (or (not (string-prefix? store file)) + (eq? 'symlink (stat:type st)) + (and (= 1 (stat:mtime st)) + (zero? (logand #o222 + (stat:mode st))))))) + + (define bin + (string-append "." #$profile "/bin")) + + (setenv "PATH" + (string-append #$%tar-bootstrap "/bin")) + (system* "tar" "xvf" #$tarball) + (mkdir #$output) + (exit + (and (file-exists? (string-append bin "/guile")) + (file-exists? store) + (every canonical? + (find-files "." (const #t) + #:directories? #t)) + (string=? (string-append #$%bootstrap-guile "/bin") + (readlink bin)) + (string=? (string-append ".." #$profile + "/bin/guile") + (readlink "bin/Guile"))))))))) + (built-derivations (list check)))) + (unless store (test-skip 1)) (test-assertm "self-contained-tarball + localstatedir" store (mlet* %store-monad base-commit: 89e5f3f3847b3bfd507ea9f0874a73f99a53cbf9 -- 2.39.1 From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 06 10:47:39 2023 Received: (at 61949) by debbugs.gnu.org; 6 Mar 2023 15:47:39 +0000 Received: from localhost ([127.0.0.1]:43382 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pZD3q-00040U-SX for submit@debbugs.gnu.org; Mon, 06 Mar 2023 10:47:39 -0500 Received: from eggs.gnu.org ([209.51.188.92]:49904) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pZD3p-00040I-Hp for 61949@debbugs.gnu.org; Mon, 06 Mar 2023 10:47:38 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZD3k-0004k2-Aa; Mon, 06 Mar 2023 10:47:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=4kI8vykYLKbGOgffmC6bFZfVADVdHiQiitIygaVLndg=; b=l/APIsMOa54bOizf78Y3 8Hbx7zcQHx2jsXwjL9ask9PQCqgcdiYayyrRok+e/Sq5Qp/CSFqn7dIfJUhlnAdfrvPLdJEVBmF8t 9aM00uu50SqZYMlM79NQjej4KlX8O/uhdHGgCR2Tc43nXordsjeFp0qx0vzYkoSP0mSj120y3XFN9 DkoXg3NImJTMRbJjaqWA3q3YZ/sd+fVUOdNdIVM7l1r19qK+eEtGi8ukRK4jf7ddPy8Y/FB9vJw+Q 3XSxH2RVerILRp+eaXM9nwOh8wfDMXMGe3geAM+4Jw5F2ZsFEsM9koSXPljubQhmbRALAc6nE9514 sT3CPgh/ThnEKQ==; Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pZD3j-00020z-Ti; Mon, 06 Mar 2023 10:47:32 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Maxim Cournoyer Subject: Re: [bug#61949] [PATCH] pack: Move common build code to (guix build pack). References: <20230304031523.24102-1-maxim.cournoyer@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Sextidi 16 =?utf-8?Q?Vent=C3=B4se?= an 231 de la =?utf-8?Q?R=C3=A9volution=2C?= jour de =?utf-8?Q?l'=C3=89pinard?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 06 Mar 2023 16:47:28 +0100 In-Reply-To: <20230304031523.24102-1-maxim.cournoyer@gmail.com> (Maxim Cournoyer's message of "Fri, 3 Mar 2023 22:15:23 -0500") Message-ID: <87sfehanlb.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 61949 Cc: 61949@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Maxim Cournoyer skribis: > The rationale is to reduce the number of derivations built per pack to id= eally > one, to minimize storage requirements. The number of derivations had gon= e up > with 68380db4 ("pack: Extract populate-profile-root from > self-contained-tarball/builder.") as a side effect to improving code reus= e. > > * guix/scripts/pack.scm (guix): Add commentary comment. > (populate-profile-root, self-contained-tarball/builder): Extract to... > * guix/build/pack.scm (populate-profile-root!): ... this, and... > (build-self-contained-tarball): ... that, adjusting for use on the build = side. > (assert-utf8-locale): New procedure. > (self-contained-tarball, debian-archive, rpm-archive): Adjust accordingly. Thanks for working on it! [...] > +;;; Copyright =C2=A9 2021, 2023 Maxim Cournoyer This may be inaccurate given that some of the code here predates this file. > ;;; along with GNU Guix. If not, see . >=20=20 > +;;; Commentary: > + > +;;; This module contains build-side common procedures used by the host-s= ide > +;;; (guix scripts pack) module, mostly to allow for code reuse. Due to = making > +;;; use of the (guix build store-copy) module, it transitively requires = the > +;;; sqlite and gcrypt extensions to be available. > + > +;;; Code: > + > (define-module (guix build pack) Commentary/code should come after =E2=80=98define-module=E2=80=99. > +(define (assert-utf8-locale) > + "Verify the current process is using the en_US.utf8 locale." > + (unless (string=3D? "unset for tests" (getenv "GUIX_LOCPATH")) > + (unless (false-if-exception (setlocale LC_ALL "en_US.utf8")) > + (error "environment not configured for en_US.utf8 locale")))) > + > +(define* (populate-profile-root! profile > + #:key (profile-name "guix-profile") > + localstatedir? > + store-database > + deduplicate? > + (symlinks '())) Please leave out the bang from the name. The convention in Scheme is to suffix a name with bang when it modifies the object(s) it=E2=80=99s given; that=E2=80=99s not the case here (see also =E2=80=98mkdir=E2=80=99, =E2=80= =98open-output-file=E2=80=99, etc.). > + "Populate the root profile directory with SYMLINKS and a Guix database= , when > +LOCALSTATEDIR? is set, and a pre-computed STORE-DATABASE is provided. T= he > +directory is created as \"root\" in the current working directory. When > +DEDUPLICATE? is true, deduplicate the store items, which relies on hard > +links. It needs to run in an environment where " > + (when localstatedir? > + (unless store-database > + (error "missing STORE-DATABASE argument"))) > + > + (define symlink->directives Please move the =E2=80=98when=E2=80=99 expression after all defines so that= this code can be interpreted by Guile 2.0, which in turn will allow us to run tests on =E2=80=98guile-bootstrap=E2=80=99. > +(define* (build-self-contained-tarball profile > + tarball-file-name > + #:key (profile-name "guix-profile= ") > + target > + localstatedir? > + store-database > + deduplicate? > + symlinks > + compressor-command > + archiver) > + "Create a self-contained tarball TARBALL-FILE-NAME from PROFILE, optio= nally > +compressing it with COMPRESSOR-COMMAND, the complete command-line string= to > +use for the compressor." > + (assert-utf8-locale) > + > + (populate-profile-root! profile > + #:profile-name profile-name > + #:localstatedir? localstatedir? > + #:store-database store-database > + #:deduplicate? deduplicate? > + #:symlinks symlinks) > + > + (define tar (string-append archiver "/bin/tar")) Likewise, move defines before statements. Also, I would just assume =E2=80=9Ctar=E2=80=9D is in $PATH. That=E2=80=99= s the assumption generally made for things that need to shell out to various commands, such as (gnu build file-systems), (guix docker), etc. > ;;; along with GNU Guix. If not, see . >=20=20 > +;;; Commentary: > + > +;;; This module implements the 'guix pack' command and the various suppo= rted > +;;; formats. Where feasible, the builders of the packs should be implem= ented > +;;; as single derivations to minimize storage requirements. > + > +;;; Code: Likewise needs to be moved down. :-) > -(test-assertm "self-contained-tarball" %store > - (mlet* %store-monad > - ((profile -> (profile > - (content (packages->manifest (list %bootstrap-guile)= )) > - (hooks '()) > - (locales? #f))) > - (tarball (self-contained-tarball "pack" profile > - #:symlinks '(("/bin/Guile" > - -> "bin/guile")) > - #:compressor %gzip-compressor > - #:archiver %tar-bootstrap)) [...] > ;; The following test needs guile-sqlite3, libgcrypt, etc. as a conseque= nce of > ;; commit c45477d2a1a651485feede20fe0f3d15aec48b39 and related changes. = Thus, > ;; run it on the user's store, if it's available, on the grounds that th= ese > ;; dependencies may be already there, or we can get substitutes or build= them > ;; quite inexpensively; see . > - > (with-external-store store > + (unless store (test-skip 1)) > + (test-assertm "self-contained-tarball" store We should avoid moving this tests here. The goal is to keep as many tests as possible under the =E2=80=9Cnormal mode=E2=80=9D (outside =E2=80=98with-external-store=E2=80=99) because they are exercised more freq= uently. I went to great lengths to make it possible here, so we should strive to preserve that property. (Note that I haven=E2=80=99t tried running the code and tests yet.) Could you send a v2? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 06 14:13:21 2023 Received: (at 61949) by debbugs.gnu.org; 6 Mar 2023 19:13:21 +0000 Received: from localhost ([127.0.0.1]:43642 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pZGGu-00010E-V1 for submit@debbugs.gnu.org; Mon, 06 Mar 2023 14:13:21 -0500 Received: from mail-qv1-f54.google.com ([209.85.219.54]:45683) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pZGGt-000100-1u for 61949@debbugs.gnu.org; Mon, 06 Mar 2023 14:13:19 -0500 Received: by mail-qv1-f54.google.com with SMTP id bo10so7336919qvb.12 for <61949@debbugs.gnu.org>; Mon, 06 Mar 2023 11:13:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678129993; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=Dm9U0S6UKrGR28o87r+VPUrx0faCfdOap0VtLk5zwQ4=; b=C99Ll1MvoIwdEzUTe0MwwL5JD5TTe9LTubisoGUQeNhdMv6ClynHZ85JDPuf1BCB8k 2MtYo36pG1QLxE8vgWHf7gHb96XJCf/LZeGYbAekbLY2z6Bz7Oc4O74cnKg8c4SWhx97 Nwlf4sYsT6RuOFQJGpWMAX1kJSvzEEEQ/KXarFyif2pttXHfTsMZgpDsDINLT7pgbEHz SyNqEryHdGEovlCXIQLg4uHBdJEG98NmK1wlLiZm8zLKpwEPGMzSfrZ4yY5xx4ogaOd4 x+7LrjvENIOzHSTKJ9sr6mNOLwwI5lOgSlAsJeGRk1R6/oh9qPop2ZEjCQqL3okSYO6V uMvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678129993; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Dm9U0S6UKrGR28o87r+VPUrx0faCfdOap0VtLk5zwQ4=; b=6PSqRFZWA5vkk0ZG9+I8029smwBv0t7HILJ2CzmZIV4T0UkHMhzk+xoumU0EDM19wO plAQruBh5OXT/0TiwbcszRmRIXnPAr6VUQo3LDsgjj6kb89KpOeTPdFP5QP1Gpxev+o3 SjXC+zK8FjasM/hbwHcWaIkNBTyAOQMIs7TkYHhGDYT0hcIrooMG4fAvCy70ZC9OqlK9 OKp5dBWDVPKojPpTfQ/MlCcdtNyslmZf0kgtRLCzgcxsFQVmJyf8vEcAWAcmLMfPz+oZ ogjpuRJ/cDIXnw5MhiPpO8xMw6eKD3eMxH7tMGI9S7tnOojxDRaYvA09UX0jR+CpDcbu VUrQ== X-Gm-Message-State: AO0yUKVtamnh4LKLy0axnRwb2grAiclzSYRJhMkU9EJMQmKl0W5/kiZ/ 4+JYv3CXRVExz01Fu9Y5H6/XfKza+AU= X-Google-Smtp-Source: AK7set8g1ERs8/ZB+oMxG+aNXgLfRxsZO7/lcQO9w4XwhNaJKxDofdj7Yy2s86um61BGQ8ukSsQC1Q== X-Received: by 2002:ad4:5ca4:0:b0:4c7:595c:9940 with SMTP id q4-20020ad45ca4000000b004c7595c9940mr20879661qvh.51.1678129993113; Mon, 06 Mar 2023 11:13:13 -0800 (PST) Received: from hurd (dsl-149-144.b2b2c.ca. [66.158.149.144]) by smtp.gmail.com with ESMTPSA id t190-20020a3746c7000000b0074235745fdasm7925907qka.58.2023.03.06.11.13.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Mar 2023 11:13:12 -0800 (PST) From: Maxim Cournoyer To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#61949] [PATCH] pack: Move common build code to (guix build pack). References: <20230304031523.24102-1-maxim.cournoyer@gmail.com> <87sfehanlb.fsf@gnu.org> Date: Mon, 06 Mar 2023 14:13:11 -0500 In-Reply-To: <87sfehanlb.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Mon, 06 Mar 2023 16:47:28 +0100") Message-ID: <87ttyxzoag.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61949 Cc: 61949@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludovic, Ludovic Court=C3=A8s writes: > Hi, > > Maxim Cournoyer skribis: > >> The rationale is to reduce the number of derivations built per pack to i= deally >> one, to minimize storage requirements. The number of derivations had go= ne up >> with 68380db4 ("pack: Extract populate-profile-root from >> self-contained-tarball/builder.") as a side effect to improving code reu= se. >> >> * guix/scripts/pack.scm (guix): Add commentary comment. >> (populate-profile-root, self-contained-tarball/builder): Extract to... >> * guix/build/pack.scm (populate-profile-root!): ... this, and... >> (build-self-contained-tarball): ... that, adjusting for use on the build= side. >> (assert-utf8-locale): New procedure. >> (self-contained-tarball, debian-archive, rpm-archive): Adjust accordingl= y. > > Thanks for working on it! > [...] > >> +;;; Copyright =C2=A9 2021, 2023 Maxim Cournoyer > > This may be inaccurate given that some of the code here predates this > file. > >> ;;; along with GNU Guix. If not, see . >> >> +;;; Commentary: >> + >> +;;; This module contains build-side common procedures used by the host-= side >> +;;; (guix scripts pack) module, mostly to allow for code reuse. Due to= making >> +;;; use of the (guix build store-copy) module, it transitively requires= the >> +;;; sqlite and gcrypt extensions to be available. >> + >> +;;; Code: >> + >> (define-module (guix build pack) > > Commentary/code should come after =E2=80=98define-module=E2=80=99. Eh. I remembered getting it wrong last time, and tried finding the information in the Guile Reference manual; I ended up looking at the "module/scripts/display-commentary.scm" source of the Guile tree, which has: --8<---------------cut here---------------start------------->8--- [...] ;;; Commentary: ;; Usage: display-commentary REF1 REF2 ... ;; ;; Display Commentary section from REF1, REF2 and so on. ;; Each REF may be a filename or module name (list of symbols). ;; In the latter case, a filename is computed by searching `%load-path'. ;;; Code: (define-module (scripts display-commentary) :use-module (ice-9 documentation) :export (display-commentary)) --8<---------------cut here---------------end--------------->8--- Is this wrong? It seems the module implementing the functionality should have gotten that right, ha! Fixed. >> +(define (assert-utf8-locale) >> + "Verify the current process is using the en_US.utf8 locale." >> + (unless (string=3D? "unset for tests" (getenv "GUIX_LOCPATH")) >> + (unless (false-if-exception (setlocale LC_ALL "en_US.utf8")) >> + (error "environment not configured for en_US.utf8 locale")))) >> + >> +(define* (populate-profile-root! profile >> + #:key (profile-name "guix-profile") >> + localstatedir? >> + store-database >> + deduplicate? >> + (symlinks '())) > > Please leave out the bang from the name. The convention in Scheme is to > suffix a name with bang when it modifies the object(s) it=E2=80=99s given; > that=E2=80=99s not the case here (see also =E2=80=98mkdir=E2=80=99, =E2= =80=98open-output-file=E2=80=99, etc.). I see. I wasn't sure, thanks. Fixed. >> + "Populate the root profile directory with SYMLINKS and a Guix databas= e, when >> +LOCALSTATEDIR? is set, and a pre-computed STORE-DATABASE is provided. = The >> +directory is created as \"root\" in the current working directory. When >> +DEDUPLICATE? is true, deduplicate the store items, which relies on hard >> +links. It needs to run in an environment where " >> + (when localstatedir? >> + (unless store-database >> + (error "missing STORE-DATABASE argument"))) >> + >> + (define symlink->directives > > Please move the =E2=80=98when=E2=80=99 expression after all defines so th= at this code > can be interpreted by Guile 2.0, which in turn will allow us to run > tests on =E2=80=98guile-bootstrap=E2=80=99. Done, but there were more complications to get the correct Guile running (because of the new gcrypt extension dependency introduced with the move of 'populate-profile-root' to inside the build module). >> +(define* (build-self-contained-tarball profile >> + tarball-file-name >> + #:key (profile-name "guix-profil= e") >> + target >> + localstatedir? >> + store-database >> + deduplicate? >> + symlinks >> + compressor-command >> + archiver) >> + "Create a self-contained tarball TARBALL-FILE-NAME from PROFILE, opti= onally >> +compressing it with COMPRESSOR-COMMAND, the complete command-line strin= g to >> +use for the compressor." >> + (assert-utf8-locale) >> + >> + (populate-profile-root! profile >> + #:profile-name profile-name >> + #:localstatedir? localstatedir? >> + #:store-database store-database >> + #:deduplicate? deduplicate? >> + #:symlinks symlinks) >> + >> + (define tar (string-append archiver "/bin/tar")) > > Likewise, move defines before statements. Done. > Also, I would just assume =E2=80=9Ctar=E2=80=9D is in $PATH. That=E2=80= =99s the assumption > generally made for things that need to shell out to various commands, > such as (gnu build file-systems), (guix docker), etc. Done. I also dropped the extraneous #:target argument of the 'build-self-contained-tarball' procedure. >> ;;; along with GNU Guix. If not, see . >> >> +;;; Commentary: >> + >> +;;; This module implements the 'guix pack' command and the various supp= orted >> +;;; formats. Where feasible, the builders of the packs should be imple= mented >> +;;; as single derivations to minimize storage requirements. >> + >> +;;; Code: > > Likewise needs to be moved down. :-) Done. >> -(test-assertm "self-contained-tarball" %store >> - (mlet* %store-monad >> - ((profile -> (profile >> - (content (packages->manifest (list %bootstrap-guile= ))) >> - (hooks '()) >> - (locales? #f))) >> - (tarball (self-contained-tarball "pack" profile >> - #:symlinks '(("/bin/Guile" >> - -> "bin/guile")) >> - #:compressor %gzip-compressor >> - #:archiver %tar-bootstrap)) > > [...] > >> ;; The following test needs guile-sqlite3, libgcrypt, etc. as a consequ= ence of >> ;; commit c45477d2a1a651485feede20fe0f3d15aec48b39 and related changes.= Thus, >> ;; run it on the user's store, if it's available, on the grounds that t= hese >> ;; dependencies may be already there, or we can get substitutes or buil= d them >> ;; quite inexpensively; see . >> - >> (with-external-store store >> + (unless store (test-skip 1)) >> + (test-assertm "self-contained-tarball" store > > We should avoid moving this tests here. The goal is to keep as many > tests as possible under the =E2=80=9Cnormal mode=E2=80=9D (outside > =E2=80=98with-external-store=E2=80=99) because they are exercised more fr= equently. I tried avoiding it, but I think it's because of the new gcrypt 'with-extensions' requirement that is now needed for the populate-profile-root that runs on the build side, as explained above. It would attempt to build guile-default and others, like the earlier problem we've had. > I went to great lengths to make it possible here, so we should strive to > preserve that property. I also appreciate the value of being able to run things without a true store/daemon. > (Note that I haven=E2=80=99t tried running the code and tests yet.) > > Could you send a v2? It will follow shortly. By the way, any clue why this happens? --8<---------------cut here---------------start------------->8--- $ make check TESTS=3Dtests/pack.sh [...] PASS: tests/pack.scm --8<---------------cut here---------------end--------------->8--- I'd have expected PASS: tests/pack.sh Thanks! Maxim From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 06 14:14:37 2023 Received: (at 61949) by debbugs.gnu.org; 6 Mar 2023 19:14:37 +0000 Received: from localhost ([127.0.0.1]:43646 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pZGI7-00012B-Mx for submit@debbugs.gnu.org; Mon, 06 Mar 2023 14:14:37 -0500 Received: from mail-qt1-f175.google.com ([209.85.160.175]:38524) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pZGI4-00011w-ND for 61949@debbugs.gnu.org; Mon, 06 Mar 2023 14:14:34 -0500 Received: by mail-qt1-f175.google.com with SMTP id c18so11786790qte.5 for <61949@debbugs.gnu.org>; Mon, 06 Mar 2023 11:14:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678130067; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=/Ffjg1/fz2bt38N6KoXcrJx6KN/EbxhtLIUYOnDrLJw=; b=WXw3FWFjvIAobSa1OU0jY8InIhWdzHWmfXJ/kmpaBDt6EehGTC5jtBUqIQtr37Jr9t AaYgL1AnDr6g57Gpcsq5SD3jwZaKrUtMqidxyMbpLqa2EWmYwsXAfm3haPcnslJ5jIIg dm5C7hpJsOphieMgQa3ZTWWmt8DK8sUGFjDxep6LhCqwxWoV/MQf2PKDb/zXM8hZg3NT 87qr075txWG/rLVs6rx2mIu3/MsFzkPbPDkB30SylPc9B0nBmxdRh6HtFtyeudER9R3E UwXexv81gPVo7EskcTJOFuyJYbH8t046/vDSvDmiozGQ+lkXm8VRSysKyp05KjpiQq/q XnLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678130067; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=/Ffjg1/fz2bt38N6KoXcrJx6KN/EbxhtLIUYOnDrLJw=; b=b8nr9fRJegwt9HIBh92v0JlEK0E9KPVTnz5yQp/a5GNyXxZon5hk9NvobQDkIR0asx LYarlpBlt5qCP+xKRV4JbLZUno2wTOkjYitSe714Q1wDWO3VTA7S6O2hCM/VEETJR5uO qKwgtmzO6NmjbMvRSLx/DditiqODNW3nxpMo4vOBZ4bJ5+BGLvSZ4IKgwXx/+qkHMktY UQCD+RfgakHxURlVfTXHljLmpqYv1sT1kFG0uPGD6XAuz8oU7k1QjnqHpx9Q/pCtN3o5 tToC+DA0gSNfN8M7CSze+Ih28yeJItbubi9fMv7JTJBq0/UA4KxvLxZu4afS3/GJgiqw p2Kw== X-Gm-Message-State: AO0yUKXligWr46fbOqVNPm0d2cJeTXNJWvoWyXgnDBXjuD7jKp2RPGmN qwrAFTmh5cpkZIBIlvgypIEZl36FeHKhOw== X-Google-Smtp-Source: AK7set/jyFkgorNw/KThTsu3A7O5OmUSB9Mfeow6fuw7ikiXIYB8cM8bKyLGEM508JtajqbnsRIaaw== X-Received: by 2002:a05:622a:2cc:b0:3bf:de2e:6165 with SMTP id a12-20020a05622a02cc00b003bfde2e6165mr18202437qtx.56.1678130066628; Mon, 06 Mar 2023 11:14:26 -0800 (PST) Received: from localhost.localdomain (dsl-149-144.b2b2c.ca. [66.158.149.144]) by smtp.gmail.com with ESMTPSA id i20-20020ac860d4000000b003b34650039bsm8095797qtm.76.2023.03.06.11.14.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Mar 2023 11:14:26 -0800 (PST) From: Maxim Cournoyer To: 61949@debbugs.gnu.org Subject: [PATCH v2] pack: Move common build code to (guix build pack). Date: Mon, 6 Mar 2023 14:14:21 -0500 Message-Id: <20230306191421.3726-1-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.39.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61949 Cc: Josselin Poiret , Tobias Geerinckx-Rice , Maxim Cournoyer , Simon Tournier , Mathieu Othacehe , ludo@gnu.org, Christopher Baines , Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) The rationale is to reduce the number of derivations built per pack to ideally one, to minimize storage requirements. The number of derivations had gone up with 68380db4 ("pack: Extract populate-profile-root from self-contained-tarball/builder.") as a side effect to improving code reuse. * guix/scripts/pack.scm (guix): Add commentary comment. (populate-profile-root, self-contained-tarball/builder): Extract to... * guix/build/pack.scm (populate-profile-root): ... this, and... (build-self-contained-tarball): ... that, adjusting for use on the build side. (assert-utf8-locale): New procedure. (self-contained-tarball, debian-archive, rpm-archive): Adjust accordingly. --- Changes in v2: - Drop '!' from populate-profile-root! - Move top commentary comment below define-module block - Move expressions after definitions for Guile 2.0 compatibility - Remove #:target and #:archiver from build-self-contained-tarball guix/build/pack.scm | 111 +++++++++++++- guix/scripts/pack.scm | 343 +++++++++++++++--------------------------- tests/pack.scm | 104 ++++++------- 3 files changed, 282 insertions(+), 276 deletions(-) diff --git a/guix/build/pack.scm b/guix/build/pack.scm index 3b73d1b227..fcb1da2a6c 100644 --- a/guix/build/pack.scm +++ b/guix/build/pack.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021, 2023 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -17,8 +17,25 @@ ;;; along with GNU Guix. If not, see . (define-module (guix build pack) + #:use-module (gnu build install) #:use-module (guix build utils) - #:export (tar-base-options)) + #:use-module (guix build store-copy) + #:use-module ((guix build union) #:select (relative-file-name)) + #:use-module (ice-9 match) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:export (tar-base-options + populate-profile-root + build-self-contained-tarball)) + +;;; Commentary: + +;;; This module contains build-side common procedures used by the host-side +;;; (guix scripts pack) module, mostly to allow for code reuse. Due to making +;;; use of the (guix build store-copy) module, it transitively requires the +;;; sqlite and gcrypt extensions to be available. + +;;; Code: (define* (tar-base-options #:key tar compressor) "Return the base GNU tar options required to produce deterministic archives @@ -52,3 +69,93 @@ (define (tar-supports-sort? tar) ;; process. Use '--hard-dereference' to eliminate it. "--hard-dereference" "--check-links")) + +(define (assert-utf8-locale) + "Verify the current process is using the en_US.utf8 locale." + (unless (string=? "unset for tests" (getenv "GUIX_LOCPATH")) + (unless (false-if-exception (setlocale LC_ALL "en_US.utf8")) + (error "environment not configured for en_US.utf8 locale")))) + +(define* (populate-profile-root profile + #:key (profile-name "guix-profile") + localstatedir? + store-database + deduplicate? + (symlinks '())) + "Populate the root profile directory with SYMLINKS and a Guix database, when +LOCALSTATEDIR? is set, and a pre-computed STORE-DATABASE is provided. The +directory is created as \"root\" in the current working directory. When +DEDUPLICATE? is true, deduplicate the store items, which relies on hard +links. It needs to run in an environment where " + (define symlink->directives + ;; Return "populate directives" to make the given symlink and its + ;; parent directories. + (match-lambda + ((source '-> target) + (let ((target (string-append profile "/" target)) + (parent (dirname source))) + ;; Never add a 'directory' directive for "/" so as to + ;; preserve its ownership when extracting the archive (see + ;; below), and also because this would lead to adding the + ;; same entries twice in the tarball. + `(,@(if (string=? parent "/") + '() + `((directory ,parent))) + ;; Use a relative file name for compatibility with + ;; relocatable packs. + (,source -> ,(relative-file-name parent target))))))) + + (define directives + ;; Fully-qualified symlinks. + (append-map symlink->directives symlinks)) + + (define %root "root") + + (when localstatedir? + (unless store-database + (error "missing STORE-DATABASE argument"))) + + (assert-utf8-locale) + + ;; Note: there is not much to gain here with deduplication and there + ;; is the overhead of the '.links' directory, so turn it off by + ;; default. Furthermore GNU tar < 1.30 sometimes fails to extract + ;; tarballs with hard links: + ;; . + (populate-store (list "profile") %root #:deduplicate? deduplicate?) + + (when localstatedir? + (install-database-and-gc-roots %root store-database + profile #:profile-name profile-name)) + + ;; Create SYMLINKS. + (for-each (cut evaluate-populate-directive <> %root) directives)) + +(define* (build-self-contained-tarball profile + tarball-file-name + #:key (profile-name "guix-profile") + localstatedir? + store-database + deduplicate? + symlinks + compressor-command) + "Create a self-contained tarball TARBALL-FILE-NAME from PROFILE, optionally +compressing it with COMPRESSOR-COMMAND, the complete command-line string to +use for the compressor." + (populate-profile-root profile + #:profile-name profile-name + #:localstatedir? localstatedir? + #:store-database store-database + #:deduplicate? deduplicate? + #:symlinks symlinks) + + (assert-utf8-locale) + + ;; GNU Tar recurses directories by default. Simply add the whole root + ;; directory, which contains all the files to be archived. This avoids + ;; creating duplicate files in the archives that would be stored as hard + ;; links by GNU Tar. + (apply invoke "tar" "-cvf" tarball-file-name "-C" "root" "." + (tar-base-options + #:tar "tar" + #:compressor compressor-command))) diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index eb41eb5563..eeb729b931 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -74,6 +74,14 @@ (define-module (guix scripts pack) %formats guix-pack)) +;;; Commentary: + +;;; This module implements the 'guix pack' command and the various supported +;;; formats. Where feasible, the builders of the packs should be implemented +;;; as single derivations to minimize storage requirements. + +;;; Code: + ;; This one is only for use in this module, so don't put it in %compressors. (define bootstrap-xz (compressor "bootstrap-xz" ".xz" @@ -199,153 +207,18 @@ (define (set-utf8-locale profile) "Configure the environment to use the \"en_US.utf8\" locale provided by the GLIBC-UT8-LOCALES package." ;; Arrange to not depend on 'glibc-utf8-locales' when using '--bootstrap'. - (and (or (not (profile? profile)) - (profile-locales? profile)) - #~(begin - (setenv "GUIX_LOCPATH" - #+(file-append glibc-utf8-locales "/lib/locale")) - (setlocale LC_ALL "en_US.utf8")))) - -(define* (populate-profile-root profile - #:key (profile-name "guix-profile") - target - localstatedir? - deduplicate? - (symlinks '())) - "Populate the root profile directory with SYMLINKS and a Guix database, when -LOCALSTATEDIR? is set. When DEDUPLICATE? is true, deduplicate the store -items, which relies on hard links." - (define database - (and localstatedir? - (file-append (store-database (list profile)) - "/db/db.sqlite"))) - - (define bootstrap? - ;; Whether a '--bootstrap' environment is needed, for testing purposes. - ;; XXX: Infer that from available info. - (and (not database) (not (profile-locales? profile)))) - - (define (import-module? module) - ;; Since we don't use deduplication support in 'populate-store', don't - ;; import (guix store deduplication) and its dependencies, which includes - ;; Guile-Gcrypt, unless DEDUPLICATE? is #t. This makes it possible to run - ;; tests with '--bootstrap'. - (and (not-config? module) - (or deduplicate? (not (equal? '(guix store deduplication) module))))) - - (computed-file "profile-directory" - (with-imported-modules (source-module-closure - `((guix build pack) - (guix build store-copy) - (guix build utils) - (guix build union) - (gnu build install)) - #:select? import-module?) + (if (or (not (profile? profile)) + (profile-locales? profile)) #~(begin - (use-modules (guix build pack) - (guix build store-copy) - (guix build utils) - ((guix build union) #:select (relative-file-name)) - (gnu build install) - (srfi srfi-1) - (srfi srfi-26) - (ice-9 match)) - - (define symlink->directives - ;; Return "populate directives" to make the given symlink and its - ;; parent directories. - (match-lambda - ((source '-> target) - (let ((target (string-append #$profile "/" target)) - (parent (dirname source))) - ;; Never add a 'directory' directive for "/" so as to - ;; preserve its ownership when extracting the archive (see - ;; below), and also because this would lead to adding the - ;; same entries twice in the tarball. - `(,@(if (string=? parent "/") - '() - `((directory ,parent))) - ;; Use a relative file name for compatibility with - ;; relocatable packs. - (,source -> ,(relative-file-name parent target))))))) - - (define directives - ;; Fully-qualified symlinks. - (append-map symlink->directives '#$symlinks)) - - ;; Make sure non-ASCII file names are properly handled. - #+(set-utf8-locale profile) - - ;; Note: there is not much to gain here with deduplication and there - ;; is the overhead of the '.links' directory, so turn it off by - ;; default. Furthermore GNU tar < 1.30 sometimes fails to extract - ;; tarballs with hard links: - ;; . - (populate-store (list "profile") #$output - #:deduplicate? #$deduplicate?) - - (when #+localstatedir? - (install-database-and-gc-roots #$output #+database #$profile - #:profile-name #$profile-name)) - - ;; Create SYMLINKS. - (for-each (cut evaluate-populate-directive <> #$output) - directives))) - #:local-build? #f - #:guile (if bootstrap? %bootstrap-guile (default-guile)) - #:options (list #:references-graphs `(("profile" ,profile)) - #:target target))) + (setenv "GUIX_LOCPATH" + #+(file-append glibc-utf8-locales "/lib/locale")) + (setlocale LC_ALL "en_US.utf8")) + #~(setenv "GUIX_LOCPATH" "unset for tests"))) ;;; ;;; Tarball format. ;;; -(define* (self-contained-tarball/builder profile - #:key (profile-name "guix-profile") - target - localstatedir? - deduplicate? - symlinks - compressor - archiver) - "Return a GEXP that can build a self-contained tarball." - - (define root (populate-profile-root profile - #:profile-name profile-name - #:target target - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks)) - - (with-imported-modules (source-module-closure '((guix build pack) - (guix build utils))) - #~(begin - (use-modules (guix build pack) - (guix build utils)) - - ;; Make sure non-ASCII file names are properly handled. - #+(set-utf8-locale profile) - - (define tar #+(file-append archiver "/bin/tar")) - - (define %root (if #$localstatedir? "." #$root)) - - (when #$localstatedir? - ;; Fix the permission of the Guix database file, which was made - ;; read-only when copied to the store in populate-profile-root. - (copy-recursively #$root %root) - (chmod (string-append %root "/var/guix/db/db.sqlite") #o644)) - - (with-directory-excursion %root - ;; GNU Tar recurses directories by default. Simply add the whole - ;; current directory, which contains all the files to be archived. - ;; This avoids creating duplicate files in the archives that would - ;; be stored as hard links by GNU Tar. - (apply invoke tar "-cvf" #$output "." - (tar-base-options - #:tar tar - #:compressor #+(and=> compressor compressor-command))))))) - (define* (self-contained-tarball name profile #:key target (profile-name "guix-profile") @@ -367,16 +240,40 @@ (define* (self-contained-tarball name profile (warning (G_ "entry point not supported in the '~a' format~%") 'tarball)) + (define database + (and localstatedir? + (file-append (store-database (list profile)) + "/db/db.sqlite"))) + (gexp->derivation (string-append name ".tar" (compressor-extension compressor)) - (self-contained-tarball/builder profile - #:profile-name profile-name - #:target target - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks - #:compressor compressor - #:archiver archiver))) + (with-extensions (list guile-gcrypt) + (with-imported-modules `(((guix config) => ,(make-config.scm)) + ,@(source-module-closure + `((guix build pack) + (guix build utils)) + #:select? not-config?)) + #~(begin + (use-modules (guix build pack) + (guix build utils)) + + ;; Make sure non-ASCII file names are properly handled. + #+(set-utf8-locale profile) + + (setenv "PATH" #+(file-append archiver "/bin")) + + (build-self-contained-tarball #$profile + #$output + #:profile-name #$profile-name + #:localstatedir? #$localstatedir? + #:store-database #+database + #:deduplicate? #$deduplicate? + #:symlinks '#$symlinks + #:compressor-command + #+(and=> compressor + compressor-command))))) + #:target target + #:references-graphs `(("profile" ,profile)))) ;;; @@ -721,20 +618,10 @@ (define %valid-compressors '("gzip" "xz" "none")) (warning (G_ "entry point not supported in the '~a' format~%") 'deb)) - (define data-tarball - (computed-file (string-append "data.tar" (compressor-extension - compressor)) - (self-contained-tarball/builder profile - #:target target - #:profile-name profile-name - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks - #:compressor compressor - #:archiver archiver) - #:local-build? #f ;allow offloading - #:options (list #:references-graphs `(("profile" ,profile)) - #:target target))) + (define database + (and localstatedir? + (file-append (store-database (list profile)) + "/db/db.sqlite"))) (define build (with-extensions (list guile-gcrypt) @@ -752,6 +639,9 @@ (define build (ice-9 optargs) (srfi srfi-1)) + ;; Make sure non-ASCII file names are properly handled. + #+(set-utf8-locale profile) + (define machine-type ;; Extract the machine type from the specified target, else from the ;; current system. @@ -805,10 +695,26 @@ (define debian-format-version "2.0") (lambda (port) (format port "~a~%" debian-format-version))) - (define data-tarball-file-name (strip-store-file-name - #+data-tarball)) + (define compressor-command + #+(and=> compressor compressor-command)) + + (define compressor-extension + #+(compressor-extension compressor)) + + (define data-tarball-file-name + (string-append "data.tar" compressor-extension)) + + (setenv "PATH" #+(file-append archiver "/bin")) - (copy-file #+data-tarball data-tarball-file-name) + (build-self-contained-tarball #$profile + data-tarball-file-name + #:profile-name #$profile-name + #:localstatedir? #$localstatedir? + #:store-database #+database + #:deduplicate? #$deduplicate? + #:symlinks '#$symlinks + #:compressor-command + compressor-command) ;; Generate the control archive. (let-keywords '#$extra-options #f @@ -817,8 +723,7 @@ (define data-tarball-file-name (strip-store-file-name (triggers-file #f)) (define control-tarball-file-name - (string-append "control.tar" - #$(compressor-extension compressor))) + (string-append "control.tar" compressor-extension)) ;; Write the compressed control tarball. Only the control file is ;; mandatory (see: 'man deb' and 'man deb-control'). @@ -848,7 +753,7 @@ (define tar (string-append #+archiver "/bin/tar")) (apply invoke tar `(,@(tar-base-options #:tar tar - #:compressor #+(and=> compressor compressor-command)) + #:compressor compressor-command) "-cvf" ,control-tarball-file-name "control" ,@(if postinst-file '("postinst") '()) @@ -859,7 +764,9 @@ (define tar (string-append #+archiver "/bin/tar")) "debian-binary" control-tarball-file-name data-tarball-file-name)))))) - (gexp->derivation (string-append name ".deb") build)) + (gexp->derivation (string-append name ".deb") build + #:target target + #:references-graphs `(("profile" ,profile)))) ;;; @@ -883,66 +790,27 @@ (define* (rpm-archive name profile (when entry-point (warning (G_ "entry point not supported in the '~a' format~%") 'rpm)) - (define root (populate-profile-root profile - #:profile-name profile-name - #:target target - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks)) - - (define payload - (let* ((raw-cpio-file-name "payload.cpio") - (compressed-cpio-file-name (string-append raw-cpio-file-name - (compressor-extension - compressor)))) - (computed-file compressed-cpio-file-name - (with-imported-modules (source-module-closure - '((guix build utils) - (guix cpio) - (guix rpm))) - #~(begin - (use-modules (guix build utils) - (guix cpio) - (guix rpm) - (srfi srfi-1)) - - ;; Make sure non-ASCII file names are properly handled. - #+(set-utf8-locale profile) - - (define %root (if #$localstatedir? "." #$root)) - - (when #$localstatedir? - ;; Fix the permission of the Guix database file, which was made - ;; read-only when copied to the store in populate-profile-root. - (copy-recursively #$root %root) - (chmod (string-append %root "/var/guix/db/db.sqlite") #o644)) - - (call-with-output-file #$raw-cpio-file-name - (lambda (port) - (with-directory-excursion %root - ;; The first "." entry is discarded. - (write-cpio-archive - (remove fhs-directory? - (cdr (find-files "." #:directories? #t))) - port)))) - (when #+(compressor-command compressor) - (apply invoke (append #+(compressor-command compressor) - (list #$raw-cpio-file-name)))) - (copy-file #$compressed-cpio-file-name #$output))) - #:local-build? #f))) ;allow offloading + (define database + (and localstatedir? + (file-append (store-database (list profile)) + "/db/db.sqlite"))) (define build (with-extensions (list guile-gcrypt) (with-imported-modules `(((guix config) => ,(make-config.scm)) ,@(source-module-closure `((gcrypt hash) + (guix build pack) (guix build utils) + (guix cpio) (guix profiles) (guix rpm)) #:select? not-config?)) #~(begin (use-modules (gcrypt hash) + (guix build pack) (guix build utils) + (guix cpio) (guix profiles) (guix rpm) (ice-9 binary-ports) @@ -954,6 +822,35 @@ (define build ;; Make sure non-ASCII file names are properly handled. #+(set-utf8-locale profile) + (define %root "root") + + (populate-profile-root #$profile + #:profile-name #$profile-name + #:localstatedir? #$localstatedir? + #:store-database #+database + #:deduplicate? #$deduplicate? + #:symlinks '#$symlinks) + + (define raw-cpio-file-name "payload.cpio") + + ;; Generate CPIO payload. + (call-with-output-file raw-cpio-file-name + (lambda (port) + (with-directory-excursion %root + ;; The first "." entry is discarded. + (write-cpio-archive + (remove fhs-directory? + (cdr (find-files "." #:directories? #t))) + port)))) + + (when #+(compressor-command compressor) + (apply invoke (append #+(compressor-command compressor) + (list raw-cpio-file-name)))) + + (define cpio-file-name + (string-append "payload.cpio" + #$(compressor-extension compressor))) + (define machine-type (and=> (or #$target %host-type) (lambda (triplet) @@ -981,7 +878,7 @@ (define lead #:target (or #$target %host-type))) (define payload-digest - (bytevector->hex-string (file-sha256 #$payload))) + (bytevector->hex-string (file-sha256 cpio-file-name))) (let-keywords '#$extra-options #f ((relocatable? #f) (prein-file #f) @@ -991,7 +888,7 @@ (define payload-digest (let ((header (generate-header name version payload-digest - #$root + %root #$(compressor-name compressor) #:target (or #$target %host-type) #:relocatable? relocatable? @@ -1003,7 +900,7 @@ (define payload-digest (define header-sha256 (bytevector->hex-string (sha256 (u8-list->bytevector header)))) - (define payload-size (stat:size (stat #$payload))) + (define payload-size (stat:size (stat cpio-file-name))) (define header+compressed-payload-size (+ (length header) payload-size)) @@ -1013,7 +910,7 @@ (define signature header+compressed-payload-size)) ;; Serialize the archive components to a file. - (call-with-input-file #$payload + (call-with-input-file cpio-file-name (lambda (in) (call-with-output-file #$output (lambda (out) @@ -1022,7 +919,9 @@ (define signature header)) (sendfile out in payload-size))))))))))) - (gexp->derivation (string-append name ".rpm") build)) + (gexp->derivation (string-append name ".rpm") build + #:target target + #:references-graphs `(("profile" ,profile)))) ;;; diff --git a/tests/pack.scm b/tests/pack.scm index 87187bb62c..397fb37b12 100644 --- a/tests/pack.scm +++ b/tests/pack.scm @@ -76,64 +76,64 @@ (define rpm-for-tests (test-begin "pack") -(unless (network-reachable?) (test-skip 1)) -(test-assertm "self-contained-tarball" %store - (mlet* %store-monad - ((profile -> (profile - (content (packages->manifest (list %bootstrap-guile))) - (hooks '()) - (locales? #f))) - (tarball (self-contained-tarball "pack" profile - #:symlinks '(("/bin/Guile" - -> "bin/guile")) - #:compressor %gzip-compressor - #:archiver %tar-bootstrap)) - (check (gexp->derivation "check-tarball" - (with-imported-modules '((guix build utils)) - #~(begin - (use-modules (guix build utils) - (srfi srfi-1)) - - (define store - ;; The unpacked store. - (string-append "." (%store-directory) "/")) - - (define (canonical? file) - ;; Return #t if FILE is read-only and its mtime is 1. - (let ((st (lstat file))) - (or (not (string-prefix? store file)) - (eq? 'symlink (stat:type st)) - (and (= 1 (stat:mtime st)) - (zero? (logand #o222 - (stat:mode st))))))) - - (define bin - (string-append "." #$profile "/bin")) - - (setenv "PATH" - (string-append #$%tar-bootstrap "/bin")) - (system* "tar" "xvf" #$tarball) - (mkdir #$output) - (exit - (and (file-exists? (string-append bin "/guile")) - (file-exists? store) - (every canonical? - (find-files "." (const #t) - #:directories? #t)) - (string=? (string-append #$%bootstrap-guile "/bin") - (readlink bin)) - (string=? (string-append ".." #$profile - "/bin/guile") - (readlink "bin/Guile"))))))))) - (built-derivations (list check)))) - ;; The following test needs guile-sqlite3, libgcrypt, etc. as a consequence of ;; commit c45477d2a1a651485feede20fe0f3d15aec48b39 and related changes. Thus, ;; run it on the user's store, if it's available, on the grounds that these ;; dependencies may be already there, or we can get substitutes or build them ;; quite inexpensively; see . - (with-external-store store + (unless store (test-skip 1)) + (test-assertm "self-contained-tarball" store + (mlet* %store-monad + ((guile (set-guile-for-build (default-guile))) + (profile -> (profile + (content (packages->manifest (list %bootstrap-guile))) + (hooks '()) + (locales? #f))) + (tarball (self-contained-tarball "pack" profile + #:symlinks '(("/bin/Guile" + -> "bin/guile")) + #:compressor %gzip-compressor + #:archiver %tar-bootstrap)) + (check (gexp->derivation "check-tarball" + (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils) + (srfi srfi-1)) + + (define store + ;; The unpacked store. + (string-append "." (%store-directory) "/")) + + (define (canonical? file) + ;; Return #t if FILE is read-only and its mtime is 1. + (let ((st (lstat file))) + (or (not (string-prefix? store file)) + (eq? 'symlink (stat:type st)) + (and (= 1 (stat:mtime st)) + (zero? (logand #o222 + (stat:mode st))))))) + + (define bin + (string-append "." #$profile "/bin")) + + (setenv "PATH" + (string-append #$%tar-bootstrap "/bin")) + (system* "tar" "xvf" #$tarball) + (mkdir #$output) + (exit + (and (file-exists? (string-append bin "/guile")) + (file-exists? store) + (every canonical? + (find-files "." (const #t) + #:directories? #t)) + (string=? (string-append #$%bootstrap-guile "/bin") + (readlink bin)) + (string=? (string-append ".." #$profile + "/bin/guile") + (readlink "bin/Guile"))))))))) + (built-derivations (list check)))) + (unless store (test-skip 1)) (test-assertm "self-contained-tarball + localstatedir" store (mlet* %store-monad base-commit: a0d22c41989e529859c813fb64a78250bde76991 -- 2.39.1 From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 03 05:11:13 2023 Received: (at 61949) by debbugs.gnu.org; 3 Jul 2023 09:11:13 +0000 Received: from localhost ([127.0.0.1]:33125 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qGFaS-0003NI-Dd for submit@debbugs.gnu.org; Mon, 03 Jul 2023 05:11:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:51348) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qGFaN-0003Mz-09 for 61949@debbugs.gnu.org; Mon, 03 Jul 2023 05:11:10 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qGFaH-0002YB-DT; Mon, 03 Jul 2023 05:11:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=6VCdE6WVninkT4c4fIEBMetEk7rHDEE8XFucjD2VIuM=; b=WyBQ15Uty+1rFYy9xK3s 9nr0YairQqnQMD4lPSy7m6hiNHoi+rQM5qsqAyuxMWljK3+cfkNp6To4jGfrYdI9U8uWShyNDTdog kiTiIbO+RGbYYrUlcDHesviNVCKVzQqihVN/H+GkRMry4lnmXhdTo9BPu9D8csysp+XO8ojl9MM9n 6ZYduL98PMOMLcb1CdIlkHrinjlO5jVcs6vzSwIZQjauwrU/K/9gSH0My8+WHOftippxKehqalsFX QbvLiDanQM8p1uZQJsRtQfwDhjnlexfQvEDMVrkgbLSz+bXqAFc0xlpzHGYJYK548LIogamDN/Z8k pI29sWO8TRMDnw==; Received: from [89.207.171.77] (helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qGFaE-0002rX-O5; Mon, 03 Jul 2023 05:11:01 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Maxim Cournoyer Subject: Re: bug#61949: [PATCH] pack: Move common build code to (guix build pack). References: <20230304031523.24102-1-maxim.cournoyer@gmail.com> <20230306191421.3726-1-maxim.cournoyer@gmail.com> Date: Mon, 03 Jul 2023 11:10:51 +0200 In-Reply-To: <20230306191421.3726-1-maxim.cournoyer@gmail.com> (Maxim Cournoyer's message of "Mon, 6 Mar 2023 14:14:21 -0500") Message-ID: <87y1jxqs10.fsf_-_@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 61949 Cc: 61949@debbugs.gnu.org, Christopher Baines , Simon Tournier , Mathieu Othacehe , Tobias Geerinckx-Rice , Josselin Poiret , Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Maxim and all, Maxim Cournoyer skribis: > The rationale is to reduce the number of derivations built per pack to id= eally > one, to minimize storage requirements. The number of derivations had gon= e up > with 68380db4 ("pack: Extract populate-profile-root from > self-contained-tarball/builder.") as a side effect to improving code reus= e. > > * guix/scripts/pack.scm (guix): Add commentary comment. > (populate-profile-root, self-contained-tarball/builder): Extract to... > * guix/build/pack.scm (populate-profile-root): ... this, and... > (build-self-contained-tarball): ... that, adjusting for use on the build = side. > (assert-utf8-locale): New procedure. > (self-contained-tarball, debian-archive, rpm-archive): Adjust accordingly. Sorry for dropping the ball! > Changes in v2: > - Drop '!' from populate-profile-root! > - Move top commentary comment below define-module block > - Move expressions after definitions for Guile 2.0 compatibility > - Remove #:target and #:archiver from build-self-contained-tarball I believe the only outstanding issue for me is the ability to run tests without relying on an external store. > (define* (self-contained-tarball name profile > #:key target > (profile-name "guix-profile") > @@ -367,16 +240,40 @@ (define* (self-contained-tarball name profile > (warning (G_ "entry point not supported in the '~a' format~%") > 'tarball)) >=20=20 > + (define database > + (and localstatedir? > + (file-append (store-database (list profile)) > + "/db/db.sqlite"))) > + > (gexp->derivation (string-append name ".tar" > (compressor-extension compressor)) > - (self-contained-tarball/builder profile > - #:profile-name profile-name > - #:target target > - #:localstatedir? localstatedir? > - #:deduplicate? deduplicate? > - #:symlinks symlinks > - #:compressor compressor > - #:archiver archiver))) > + (with-extensions (list guile-gcrypt) > + (with-imported-modules `(((guix config) =3D> ,(make-config.scm)) You can remove (guix config) here since it=E2=80=99s unused. > + ,@(source-module-closure > + `((guix build pack) > + (guix build utils)) > + #:select? not-config?)) I figured that, to allow tests to run without an external store, we need to change it like this: (gexp->derivation (string-append name ".tar" (compressor-extension compressor)) (with-extensions (if deduplicate? (list guile-gcrypt) '()) (with-imported-modules (let ((lst (source-module-closure `((guix build pack) (guix build utils)) #:select? not-config?))) (if deduplicate? lst (delete '(guix store deduplication) lst)= )) #~(begin (use-modules (guix build pack) (guix build utils)) =E2=80=A6)))) Quite some boilerplate given that #:deduplicate? is always #f in practice. So maybe it=E2=80=99s not worth the trouble, after all. I=E2=80=99ll let you rebase and adjust as you see fit, but it LGTM! Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 18 17:13:56 2023 Received: (at 61949-done) by debbugs.gnu.org; 18 Jul 2023 21:13:56 +0000 Received: from localhost ([127.0.0.1]:54443 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qLs16-0005Xn-4M for submit@debbugs.gnu.org; Tue, 18 Jul 2023 17:13:56 -0400 Received: from mail-ua1-x932.google.com ([2607:f8b0:4864:20::932]:46586) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qLs12-0005XX-8M for 61949-done@debbugs.gnu.org; Tue, 18 Jul 2023 17:13:54 -0400 Received: by mail-ua1-x932.google.com with SMTP id a1e0cc1a2514c-794af22ed66so2297275241.1 for <61949-done@debbugs.gnu.org>; Tue, 18 Jul 2023 14:13:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689714826; x=1692306826; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=HsIpixdFG9DYqMEgqX933gIGcwC43v0NY0Qi9YtSLbo=; b=cf3onIfVymS06+J4D9dO56oQcI/XwHDsQkXk66yOOnmvuNNSSYRn6gXMzFqggR2Bok bXU1BjI5OrmNTj4plNYE4dlkLkQS6jBO0h/YVjndHeJovz68PCpqasryAC0HSGa3/Hvi v7Z68Slrn0pD2sHo+MBJggMd2C4bn09f8u5L+L5i9EraJpncvblsWzRIqkFNyGS8WQRD 93tJuvtq9cGe5h7b6MLccv9lNlZ9Tp9L5YtbsKEBYN36JRrchu4ulKvw9hE+2LudT7O+ moeryi0j70NpdqgO+J247qeT5o0dB8ELPdJnltplb1PRYQAL/p+yiqEzP/ZiGhssMbW5 Yr3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689714826; x=1692306826; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=HsIpixdFG9DYqMEgqX933gIGcwC43v0NY0Qi9YtSLbo=; b=EdoOzNGzwJrzjLPg4YvUnealftD4RLRpDpty/TW4owhHLmRISIeqiyOG0rg+GhZjBQ 5zo7pbg8kpuTVOI5HkWIV65A3OXszCjcII1uQc8qQI0fce8+moQfW3H+9uIrqvZ4lso9 CzWBRBJm9GDOkHtJePDoehECwBP4dB4nl7bghpJNZvEaZ2Mi6LHWm3Lxi5mNi7VD5rsM 3l6dItaRxe50ZNK5gEvSriZB8SfR2xc4aIZUulN3biEVlOZyoWGU3/mabFY4k6Vh18eX krPLENq6vOYk/409XqxduezCHymiykj77fupSp0XP2YWQYNtkZe8DiNmaRGIh9LhoGPG WKiA== X-Gm-Message-State: ABy/qLZZp6BQWwjjcbvhObXCEAvgIogabBfctbExceURgHHyywCtnEqQ 02S6i/QtA36ThEDLnh4y6NE= X-Google-Smtp-Source: APBJJlEtGkoxuVj/DGfe3B9PTupega+ZApBPlAX4hM3gCe+sX+IgKj/9ClayQUdS+DcnUqHRWLCZRg== X-Received: by 2002:a67:fa47:0:b0:443:3e1f:7e00 with SMTP id j7-20020a67fa47000000b004433e1f7e00mr520089vsq.6.1689714826392; Tue, 18 Jul 2023 14:13:46 -0700 (PDT) Received: from raisin ([2607:fad8:4:6::984a]) by smtp.gmail.com with ESMTPSA id u17-20020a0ca711000000b00623813aa1d5sm1001875qva.89.2023.07.18.14.13.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jul 2023 14:13:45 -0700 (PDT) From: Maxim Cournoyer To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#61949: [PATCH] pack: Move common build code to (guix build pack). References: <20230304031523.24102-1-maxim.cournoyer@gmail.com> <20230306191421.3726-1-maxim.cournoyer@gmail.com> <87y1jxqs10.fsf_-_@gnu.org> Date: Tue, 18 Jul 2023 17:13:44 -0400 In-Reply-To: <87y1jxqs10.fsf_-_@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s?= =?utf-8?Q?=22's?= message of "Mon, 03 Jul 2023 11:10:51 +0200") Message-ID: <87bkg8udmf.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 61949-done Cc: 61949-done@debbugs.gnu.org, Christopher Baines , Simon Tournier , Mathieu Othacehe , Tobias Geerinckx-Rice , Josselin Poiret , Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludo! Ludovic Court=C3=A8s writes: > Hi Maxim and all, > > Maxim Cournoyer skribis: > >> The rationale is to reduce the number of derivations built per pack to i= deally >> one, to minimize storage requirements. The number of derivations had go= ne up >> with 68380db4 ("pack: Extract populate-profile-root from >> self-contained-tarball/builder.") as a side effect to improving code reu= se. >> >> * guix/scripts/pack.scm (guix): Add commentary comment. >> (populate-profile-root, self-contained-tarball/builder): Extract to... >> * guix/build/pack.scm (populate-profile-root): ... this, and... >> (build-self-contained-tarball): ... that, adjusting for use on the build= side. >> (assert-utf8-locale): New procedure. >> (self-contained-tarball, debian-archive, rpm-archive): Adjust accordingl= y. > > Sorry for dropping the ball! Thanks for picking it up again :-) [...] >> (gexp->derivation (string-append name ".tar" >> (compressor-extension compressor)) >> - (self-contained-tarball/builder profile >> - #:profile-name profile-name >> - #:target target >> - #:localstatedir? localstatedir? >> - #:deduplicate? deduplicate? >> - #:symlinks symlinks >> - #:compressor compressor >> - #:archiver archiver))) >> + (with-extensions (list guile-gcrypt) >> + (with-imported-modules `(((guix config) =3D> ,(make-config.scm)) > > You can remove (guix config) here since it=E2=80=99s unused. Done. >> + ,@(source-module-closure >> + `((guix build pack) >> + (guix build utils)) >> + #:select? not-config?)) > > I figured that, to allow tests to run without an external store, we need > to change it like this: > > (gexp->derivation (string-append name ".tar" > (compressor-extension compressor)) > (with-extensions (if deduplicate? (list guile-gcrypt) '()) > (with-imported-modules (let ((lst (source-module-closure > `((guix build pack) > (guix build utils)) > #:select? not-config?))) > (if deduplicate? > lst > (delete '(guix store deduplication) ls= t))) > #~(begin > (use-modules (guix build pack) > (guix build utils)) > =E2=80=A6)))) > > Quite some boilerplate given that #:deduplicate? is always #f in practice. > So maybe it=E2=80=99s not worth the trouble, after all. > > I=E2=80=99ll let you rebase and adjust as you see fit, but it LGTM! Thanks for this! I had to use it for tests/guix-pack.sh to pass, but it still would work for the first test in tests/pack.scm (after modifying it to use the test store as it used to), so I've left tests/pack.scm The change is now installed, thanks for the review! --=20 Thanks, Maxim From unknown Mon Jun 23 09:39:03 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Wed, 16 Aug 2023 11:24:09 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator