From unknown Wed Jun 18 23:11:40 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#45891 <45891@debbugs.gnu.org> To: bug#45891 <45891@debbugs.gnu.org> Subject: Status: [PATCH] packages: 'patch-and-repack' returns a directory when given a directory. Reply-To: bug#45891 <45891@debbugs.gnu.org> Date: Thu, 19 Jun 2025 06:11:40 +0000 retitle 45891 [PATCH] packages: 'patch-and-repack' returns a directory when= given a directory. reassign 45891 guix-patches submitter 45891 Ludovic Court=C3=A8s severity 45891 normal tag 45891 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 15 08:16:05 2021 Received: (at submit) by debbugs.gnu.org; 15 Jan 2021 13:16:05 +0000 Received: from localhost ([127.0.0.1]:40119 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l0OxR-00016l-7I for submit@debbugs.gnu.org; Fri, 15 Jan 2021 08:16:05 -0500 Received: from lists.gnu.org ([209.51.188.17]:57658) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l0OxO-00016B-HN for submit@debbugs.gnu.org; Fri, 15 Jan 2021 08:16:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39146) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0OxN-0003Gu-FI for guix-patches@gnu.org; Fri, 15 Jan 2021 08:16:02 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:56790) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0OxM-0001KD-JT; Fri, 15 Jan 2021 08:16:00 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=38924 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l0OxH-0005Ou-1A; Fri, 15 Jan 2021 08:15:57 -0500 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: guix-patches@gnu.org Subject: [PATCH] packages: 'patch-and-repack' returns a directory when given a directory. Date: Fri, 15 Jan 2021 14:15:48 +0100 Message-Id: <20210115131548.8792-1-ludo@gnu.org> X-Mailer: git-send-email 2.30.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Previously, 'patch-and-repack' would always create a tar.xz archive as a result, even if the input was a directory (a checkout). This change reduces gratuitous CPU and storage overhead. * guix/packages.scm (patch-and-repack)[tarxz-name]: Remove 'checkout?' case. [build](repack): New procedure, with "tar" invocation formerly at the top level. If SOURCE is a directory, call 'copy-recursively'; otherwise, call 'repack'. Change NAME to ORIGINAL-FILE-NAME when it matches 'checkout?'. --- guix/packages.scm | 65 ++++++++++++++++++++++++++--------------------- 1 file changed, 36 insertions(+), 29 deletions(-) Hi! This change is a followup to a recent IRC discussion: it makes ‘patch-and-repack’ preserve the “directoriness” of its input. It conflicts with other changes Maxim posted at , though that could be addressed. Thoughts? Ludo’. diff --git a/guix/packages.scm b/guix/packages.scm index 4caaa9cb79..cd2cded9ee 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès +;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021 Ludovic Courtès ;;; Copyright © 2014, 2015, 2017, 2018 Mark H Weaver ;;; Copyright © 2015 Eric Bavier ;;; Copyright © 2016 Alex Kost @@ -635,11 +635,9 @@ specifies modules in scope when evaluating SNIPPET." (define (tarxz-name file-name) ;; Return a '.tar.xz' file name based on FILE-NAME. - (let ((base (cond ((numeric-extension? file-name) - original-file-name) - ((checkout? file-name) - (string-drop-right file-name 9)) - (else (file-sans-extension file-name))))) + (let ((base (if (numeric-extension? file-name) + original-file-name + (file-sans-extension file-name)))) (string-append base (if (equal? (file-extension base) "tar") ".xz" @@ -689,6 +687,29 @@ specifies modules in scope when evaluating SNIPPET." (lambda (name) (not (member name '("." ".."))))))) + (define (repack directory output) + ;; Write to OUTPUT a compressed tarball containing DIRECTORY. + (unless tar-supports-sort? + (call-with-output-file ".file_list" + (lambda (port) + (for-each (lambda (name) + (format port "~a~%" name)) + (find-files directory + #:directories? #t + #:fail-on-error? #t))))) + + (apply invoke #+(file-append tar "/bin/tar") + "cvfa" output + ;; Avoid non-determinism in the archive. Set the mtime + ;; to 1 as is the case in the store (software like gzip + ;; behaves differently when it stumbles upon mtime = 0). + "--mtime=@1" + "--owner=root:0" "--group=root:0" + (if tar-supports-sort? + `("--sort=name" ,directory) + '("--no-recursion" + "--files-from=.file_list")))) + ;; Encoding/decoding errors shouldn't be silent. (fluid-set! %default-port-conversion-strategy 'error) @@ -742,30 +763,16 @@ specifies modules in scope when evaluating SNIPPET." (chdir "..") - (unless tar-supports-sort? - (call-with-output-file ".file_list" - (lambda (port) - (for-each (lambda (name) - (format port "~a~%" name)) - (find-files directory - #:directories? #t - #:fail-on-error? #t))))) - (apply invoke - (string-append #+tar "/bin/tar") - "cvfa" #$output - ;; Avoid non-determinism in the archive. Set the mtime - ;; to 1 as is the case in the store (software like gzip - ;; behaves differently when it stumbles upon mtime = 0). - "--mtime=@1" - "--owner=root:0" - "--group=root:0" - (if tar-supports-sort? - `("--sort=name" - ,directory) - '("--no-recursion" - "--files-from=.file_list"))))))) + ;; If SOURCE is a directory (such as a checkout), return a + ;; directory. Otherwise create a tarball. + (if (file-is-directory? #+source) + (copy-recursively directory #$output + #:log (%make-void-port "w")) + (repack directory #$output)))))) - (let ((name (tarxz-name original-file-name))) + (let ((name (if (checkout? original-file-name) + original-file-name + (tarxz-name original-file-name)))) (gexp->derivation name build #:graft? #f #:system system -- 2.30.0 From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 15 17:14:41 2021 Received: (at 45891) by debbugs.gnu.org; 15 Jan 2021 22:14:41 +0000 Received: from localhost ([127.0.0.1]:41913 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l0XMf-0001qr-6P for submit@debbugs.gnu.org; Fri, 15 Jan 2021 17:14:41 -0500 Received: from mail-qk1-f169.google.com ([209.85.222.169]:44192) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l0XMd-0001qe-SZ for 45891@debbugs.gnu.org; Fri, 15 Jan 2021 17:14:40 -0500 Received: by mail-qk1-f169.google.com with SMTP id v126so13224866qkd.11 for <45891@debbugs.gnu.org>; Fri, 15 Jan 2021 14:14:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=eLnVEG1Qsced4dYQ+Ap9iMwDjBK5tWRuQM1IS/tKagU=; b=Br9+uoeVQBLGHruBQjcx1ZiitRzsSUHVq0ykQRqEtUlfkGSV5hCQxH8xRdn2b8i9LJ XQZxghdMjxe+MPQHkCtT2h0KKrbj5F+FNN5eMYEm2+t9iC+vePMUkqRRK0RZrDDPOT/v 286OF1WfLzp8v+3R9UZmOCEYNo1KOz0/29BCoFhu2jNYow06brUbEDspL96Cnvmk6vIo 99l1RHPM9v03oAQw73AZNTcuJG4TW+NwOFQdK/5VdjVSFGYZnBswfMJsVfNAIGs6DY6X x3z49PVg5S5nS4tmWY4V5aV+gFlr1+jnS2aJ+Y6Yf9a7jzH4H79PNJpdwcQfmkBn4s0p rzFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=eLnVEG1Qsced4dYQ+Ap9iMwDjBK5tWRuQM1IS/tKagU=; b=KSmqCp+QFuNQh2EKL6Vj73rGuxb5fQ/5W/4hpUqYjcljnS5FzXlWpTSsiJQdXb/LcX tePGBThV3EpiHw+v6byIystRyag5VQBkbKzeE/MU2M/L7oh+7wWv86gkJtaAs9QUEhRv zD0bSlBpVFCpKwLL5KAU0htN8pRCeZzZ+q7+g3DeK6LR/KFGZ2gIc8jcPFEV6UalWlOC Wo8zC1RK8BqXw6sezZ3GZWJSWTzHhfAm8NS3mcp3D7J+ScbJwpk0Z1AtFfhu6qe9kjRc st72n5h4b7colTzsEVfCetQL7y01ZEuLnKS7FV8QKLR11GpomfaEFEBxoatc7KbA9YnG QPrQ== X-Gm-Message-State: AOAM533cwa+p3gnYrFi0xhBvT4W9qAXYWQfqXzr7e5EoKG9CQJh9gAYY 1QZpfU39n7C0dsF5K/hg0BD1xygeh5Y= X-Google-Smtp-Source: ABdhPJyIBCX+T1ghSudwMAoD0bEjlAnrHsyXyTqAiV9raim9YAIjfMYvT2iRgiQIfmkH+LlW9RabaQ== X-Received: by 2002:a37:a984:: with SMTP id s126mr14550626qke.407.1610748874100; Fri, 15 Jan 2021 14:14:34 -0800 (PST) Received: from hurd (dsl-205-233-125-239.b2b2c.ca. [205.233.125.239]) by smtp.gmail.com with ESMTPSA id d1sm5970346qkf.102.2021.01.15.14.14.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Jan 2021 14:14:33 -0800 (PST) From: Maxim Cournoyer To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#45891: [PATCH] packages: 'patch-and-repack' returns a directory when given a directory. References: <20210115131548.8792-1-ludo@gnu.org> Date: Fri, 15 Jan 2021 17:14:32 -0500 In-Reply-To: <20210115131548.8792-1-ludo@gnu.org> ("Ludovic =?utf-8?Q?Cour?= =?utf-8?Q?t=C3=A8s=22's?= message of "Fri, 15 Jan 2021 14:15:48 +0100") Message-ID: <878s8tluev.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 45891 Cc: 45891@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello! Ludovic Court=C3=A8s writes: > Previously, 'patch-and-repack' would always create a tar.xz archive as a > result, even if the input was a directory (a checkout). This change > reduces gratuitous CPU and storage overhead. I like it! Note that on core-updates, xz compression is relatively fast on modern machines as it can do multi-threading. About space the savings; could the 'temporary' pristine source be cleared from the store always? This would prevent keeping nonfree cruft under /gnu/store until the next garbage collection run, for those sources that are cleaned up. > * guix/packages.scm (patch-and-repack)[tarxz-name]: Remove 'checkout?' ca= se. > [build](repack): New procedure, with "tar" invocation formerly at the > top level. > If SOURCE is a directory, call 'copy-recursively'; otherwise, call > 'repack'. > Change NAME to ORIGINAL-FILE-NAME when it matches 'checkout?'. > --- > guix/packages.scm | 65 ++++++++++++++++++++++++++--------------------- > 1 file changed, 36 insertions(+), 29 deletions(-) > > Hi! > > This change is a followup to a recent IRC discussion: it makes > =E2=80=98patch-and-repack=E2=80=99 preserve the =E2=80=9Cdirectoriness=E2= =80=9D of its input. > > It conflicts with other changes Maxim posted at > , though that could be addressed. > > Thoughts? > > Ludo=E2=80=99. > > diff --git a/guix/packages.scm b/guix/packages.scm > index 4caaa9cb79..cd2cded9ee 100644 > --- a/guix/packages.scm > +++ b/guix/packages.scm > @@ -1,5 +1,5 @@ > ;;; GNU Guix --- Functional package management for GNU > -;;; Copyright =C2=A9 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 202= 0 Ludovic Court=C3=A8s > +;;; Copyright =C2=A9 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 202= 0, 2021 Ludovic Court=C3=A8s > ;;; Copyright =C2=A9 2014, 2015, 2017, 2018 Mark H Weaver > ;;; Copyright =C2=A9 2015 Eric Bavier > ;;; Copyright =C2=A9 2016 Alex Kost > @@ -635,11 +635,9 @@ specifies modules in scope when evaluating SNIPPET." >=20=20 > (define (tarxz-name file-name) > ;; Return a '.tar.xz' file name based on FILE-NAME. > - (let ((base (cond ((numeric-extension? file-name) > - original-file-name) > - ((checkout? file-name) > - (string-drop-right file-name 9)) > - (else (file-sans-extension file-name))))) > + (let ((base (if (numeric-extension? file-name) > + original-file-name > + (file-sans-extension file-name)))) This is not new code, but I'm wondering what's the purpose of numeric-extension? What kind of files does it expect to catch? Also, what happened to stripping the '-checkout' suffix that used to be done? It doesn't seem like it will happen anymore. > (string-append base > (if (equal? (file-extension base) "tar") > ".xz" > @@ -689,6 +687,29 @@ specifies modules in scope when evaluating SNIPPET." > (lambda (name) > (not (member name '("." ".."))))))) >=20=20 > + (define (repack directory output) > + ;; Write to OUTPUT a compressed tarball containing DIRECTO= RY. > + (unless tar-supports-sort? > + (call-with-output-file ".file_list" > + (lambda (port) > + (for-each (lambda (name) > + (format port "~a~%" name)) > + (find-files directory > + #:directories? #t > + #:fail-on-error? #t))))) > + > + (apply invoke #+(file-append tar "/bin/tar") > + "cvfa" output > + ;; Avoid non-determinism in the archive. Set the m= time > + ;; to 1 as is the case in the store (software like = gzip > + ;; behaves differently when it stumbles upon mtime = =3D 0). > + "--mtime=3D@1" > + "--owner=3Droot:0" "--group=3Droot:0" > + (if tar-supports-sort? > + `("--sort=3Dname" ,directory) > + '("--no-recursion" > + "--files-from=3D.file_list")))) > + > ;; Encoding/decoding errors shouldn't be silent. > (fluid-set! %default-port-conversion-strategy 'error) >=20=20 > @@ -742,30 +763,16 @@ specifies modules in scope when evaluating SNIPPET." >=20=20 > (chdir "..") >=20=20 > - (unless tar-supports-sort? > - (call-with-output-file ".file_list" > - (lambda (port) > - (for-each (lambda (name) > - (format port "~a~%" name)) > - (find-files directory > - #:directories? #t > - #:fail-on-error? #t))))) > - (apply invoke > - (string-append #+tar "/bin/tar") > - "cvfa" #$output > - ;; Avoid non-determinism in the archive. Set the m= time > - ;; to 1 as is the case in the store (software like = gzip > - ;; behaves differently when it stumbles upon mtime = =3D 0). > - "--mtime=3D@1" > - "--owner=3Droot:0" > - "--group=3Droot:0" > - (if tar-supports-sort? > - `("--sort=3Dname" > - ,directory) > - '("--no-recursion" > - "--files-from=3D.file_list"))))))) > + ;; If SOURCE is a directory (such as a checkout), return a > + ;; directory. Otherwise create a tarball. > + (if (file-is-directory? #+source) > + (copy-recursively directory #$output > + #:log (%make-void-port "w")) > + (repack directory #$output)))))) >=20=20 > - (let ((name (tarxz-name original-file-name))) > + (let ((name (if (checkout? original-file-name) > + original-file-name > + (tarxz-name original-file-name)))) > (gexp->derivation name build > #:graft? #f > #:system system Was these cases (tar archive source derivation, directory source derivation) already covered by tests under tests/packages.cm? How did you otherwise test it? World rebuilding changes are not fun to test without unit tests. I've run that test module like this: $ make check TESTS=3Dtests/packages.scm SCM_LOG_DRIVER_FLAGS=3D' --brief=3D= no' VERBOSE=3D1 with your change and it passed. Thanks, Maxim From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 16 16:29:06 2021 Received: (at 45891) by debbugs.gnu.org; 16 Jan 2021 21:29:06 +0000 Received: from localhost ([127.0.0.1]:44146 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l0t86-0000Pk-IK for submit@debbugs.gnu.org; Sat, 16 Jan 2021 16:29:06 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54912) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l0t83-0000PF-RX for 45891@debbugs.gnu.org; Sat, 16 Jan 2021 16:29:04 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58591) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0t7y-0000fI-LS; Sat, 16 Jan 2021 16:28:58 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=37586 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l0t7w-000377-VE; Sat, 16 Jan 2021 16:28:58 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Maxim Cournoyer Subject: Re: bug#45891: [PATCH] packages: 'patch-and-repack' returns a directory when given a directory. References: <20210115131548.8792-1-ludo@gnu.org> <878s8tluev.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 27 =?utf-8?Q?Niv=C3=B4se?= an 229 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 16 Jan 2021 22:28:54 +0100 In-Reply-To: <878s8tluev.fsf@gmail.com> (Maxim Cournoyer's message of "Fri, 15 Jan 2021 17:14:32 -0500") Message-ID: <87lfcsy3jd.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45891 Cc: 45891@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Maxim, Maxim Cournoyer skribis: > Ludovic Court=C3=A8s writes: > >> Previously, 'patch-and-repack' would always create a tar.xz archive as a >> result, even if the input was a directory (a checkout). This change >> reduces gratuitous CPU and storage overhead. > > I like it! > > Note that on core-updates, xz compression is relatively fast on modern > machines as it can do multi-threading. About space the savings; could > the 'temporary' pristine source be cleared from the store always? No, it=E2=80=99s not possible=E2=80=94the GC will remove what=E2=80=99s unr= eachable when it eventually runs. >> (define (tarxz-name file-name) >> ;; Return a '.tar.xz' file name based on FILE-NAME. >> - (let ((base (cond ((numeric-extension? file-name) >> - original-file-name) >> - ((checkout? file-name) >> - (string-drop-right file-name 9)) >> - (else (file-sans-extension file-name))))) >> + (let ((base (if (numeric-extension? file-name) >> + original-file-name >> + (file-sans-extension file-name)))) > > This is not new code, but I'm wondering what's the purpose of > numeric-extension? It=E2=80=99s for file names like =E2=80=9Chello-2.10=E2=80=9D, where you wo= uldn=E2=80=99t want to strip =E2=80=9C.10=E2=80=9D. (Such file names should be rare, but it=E2=80=99s n= ot impossible.) > What kind of files does it expect to catch? Also, what happened to > stripping the '-checkout' suffix that used to be done? It doesn't > seem like it will happen anymore. Stripping =E2=80=9C-checkout=E2=80=9D is no longer necessary because for th= ese we keep the original name. >> - (let ((name (tarxz-name original-file-name))) >> + (let ((name (if (checkout? original-file-name) >> + original-file-name >> + (tarxz-name original-file-name)))) >> (gexp->derivation name build >> #:graft? #f >> #:system system > > Was these cases (tar archive source derivation, directory source > derivation) already covered by tests under tests/packages.cm? How did > you otherwise test it? World rebuilding changes are not fun to test > without unit tests. In this case I rebuilt the world and tested =E2=80=98guix build -S=E2=80=99= on a git-fetch package with a snippet, but I agree that=E2=80=99s super expensiv= e (I tested the handful of commits I recently pushed to =E2=80=98core-updates=E2= =80=99 at the same time.) There are no unit tests specifically for this procedure, but I think we=E2=80=99ll quickly find out if it doesn=E2=80=99t behave as intended. WDYT? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 18 01:24:14 2021 Received: (at 45891) by debbugs.gnu.org; 18 Jan 2021 06:24:14 +0000 Received: from localhost ([127.0.0.1]:46138 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l1NxU-0003pL-Oa for submit@debbugs.gnu.org; Mon, 18 Jan 2021 01:24:13 -0500 Received: from mail-qt1-f182.google.com ([209.85.160.182]:45974) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l1NxT-0003p8-0y for 45891@debbugs.gnu.org; Mon, 18 Jan 2021 01:24:11 -0500 Received: by mail-qt1-f182.google.com with SMTP id d15so5155549qtw.12 for <45891@debbugs.gnu.org>; Sun, 17 Jan 2021 22:24:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=wIXlcT90EziNuBfKN97sj7RL4V+R7+fNXoP8gQ/9tIY=; b=sgaDGYi8QrsCbDdv9YpWBPwwIUTJTt6OxhGM7wYBxQRuBvGKfzGV3kPjlbmO7Tzdhp 6fAxsMpwaRmVTigSmp/kIWprpd3tnJB7OKyqmMYr8sfgBV49inc4/305YQE6wvCTdith 6o/86kVCMJit3GNo6ZL0sSETDWGRWZx9R4jNPSRlP1Nj15uj46c2Mzzte8yXrWJF4MnL THI/Ign1WxyiPHyFrhdD/PjWYLSV9lBPSno7e9xRl/b3zjLscmfPeO9LNLmgyjQuILwA PSQS8MLAQRuFTqQBnr2Ro99WRSifwNt9Ml1fr7qzMxbj0rGj3IWFyqTeRL/fVsZpBEtj kTBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=wIXlcT90EziNuBfKN97sj7RL4V+R7+fNXoP8gQ/9tIY=; b=DbCFX6hzaIL1pvY7RmWtoOxruJ0xAFeFOA7IoKOe6+cyFDSK5i/LGCCraAIsj0925b gb3mV98mrQn+MfcZ75TKGrngX9DewUG7aKsNmCrL6/H/8zPT9pxmNokVfQW/rc7RUQsg 87r7614gAypcaFfjJgRMJXl8sT0CQNG2vE6iO6Zhs5qSQTZEqaQYRsxDv1ybqbm4p+/z 2cPqEcx/qykBpZtlIAgeyKxAPV++Dcvl+h3ZgIpHYmVxRMBzuR4CmHBgkAbd1H8TS9Fl ZciutmL1Xdn0OILVCltlnuos26FKOMF0/DYEOUkZv5wEpfAjMsakaQh7boU9uoXAcmr1 levw== X-Gm-Message-State: AOAM533MQhOCe9wqxwFDj7fSABlynX1kWUOjrx5fQhXG9pQI0BGQkDd1 kNvJAdGeOsU079fyf99M+zaFwE8ZPsE= X-Google-Smtp-Source: ABdhPJyeFiin4n1eYfcI7Zneeroo61Cg06WBa9m9ta/6FPFyCJyPnYdLK0tSh3IudCohclDaQihh7w== X-Received: by 2002:ac8:6c50:: with SMTP id z16mr3459234qtu.112.1610951043774; Sun, 17 Jan 2021 22:24:03 -0800 (PST) Received: from hurd (dsl-149-228.b2b2c.ca. [66.158.149.228]) by smtp.gmail.com with ESMTPSA id 70sm10192162qkk.10.2021.01.17.22.24.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 Jan 2021 22:24:03 -0800 (PST) From: Maxim Cournoyer To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#45891: [PATCH] packages: 'patch-and-repack' returns a directory when given a directory. References: <20210115131548.8792-1-ludo@gnu.org> <878s8tluev.fsf@gmail.com> <87lfcsy3jd.fsf@gnu.org> Date: Mon, 18 Jan 2021 01:24:02 -0500 In-Reply-To: <87lfcsy3jd.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Sat, 16 Jan 2021 22:28:54 +0100") Message-ID: <87v9bukbjx.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 45891 Cc: 45891@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludo, Ludovic Court=C3=A8s writes: > Hi Maxim, > > Maxim Cournoyer skribis: > >> Ludovic Court=C3=A8s writes: >> >>> Previously, 'patch-and-repack' would always create a tar.xz archive as a >>> result, even if the input was a directory (a checkout). This change >>> reduces gratuitous CPU and storage overhead. >> >> I like it! >> >> Note that on core-updates, xz compression is relatively fast on modern >> machines as it can do multi-threading. About space the savings; could >> the 'temporary' pristine source be cleared from the store always? > > No, it=E2=80=99s not possible=E2=80=94the GC will remove what=E2=80=99s u= nreachable when it > eventually runs. OK. > >>> (define (tarxz-name file-name) >>> ;; Return a '.tar.xz' file name based on FILE-NAME. >>> - (let ((base (cond ((numeric-extension? file-name) >>> - original-file-name) >>> - ((checkout? file-name) >>> - (string-drop-right file-name 9)) >>> - (else (file-sans-extension file-name))))) >>> + (let ((base (if (numeric-extension? file-name) >>> + original-file-name >>> + (file-sans-extension file-name)))) >> >> This is not new code, but I'm wondering what's the purpose of >> numeric-extension? > > It=E2=80=99s for file names like =E2=80=9Chello-2.10=E2=80=9D, where you = wouldn=E2=80=99t want to strip > =E2=80=9C.10=E2=80=9D. (Such file names should be rare, but it=E2=80=99s= not impossible.) Ah! Thanks for explaining. >> What kind of files does it expect to catch? Also, what happened to >> stripping the '-checkout' suffix that used to be done? It doesn't >> seem like it will happen anymore. > > Stripping =E2=80=9C-checkout=E2=80=9D is no longer necessary because for = these we keep > the original name. > >>> - (let ((name (tarxz-name original-file-name))) >>> + (let ((name (if (checkout? original-file-name) >>> + original-file-name >>> + (tarxz-name original-file-name)))) >>> (gexp->derivation name build >>> #:graft? #f >>> #:system system >> >> Was these cases (tar archive source derivation, directory source >> derivation) already covered by tests under tests/packages.cm? How did >> you otherwise test it? World rebuilding changes are not fun to test >> without unit tests. > > In this case I rebuilt the world and tested =E2=80=98guix build -S=E2=80= =99 on a > git-fetch package with a snippet, but I agree that=E2=80=99s super expens= ive (I > tested the handful of commits I recently pushed to =E2=80=98core-updates= =E2=80=99 at the > same time.) > > There are no unit tests specifically for this procedure, but I think > we=E2=80=99ll quickly find out if it doesn=E2=80=99t behave as intended. > > WDYT? LGTM. Feel free to push! About my related change that we thought was conflicting with this one; it at least applies on top of your change, and only one test fails currently, I'm working on a fix. Feel free to push to core-updates! Thank you, Maxim From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 18 09:56:59 2021 Received: (at 45891-done) by debbugs.gnu.org; 18 Jan 2021 14:56:59 +0000 Received: from localhost ([127.0.0.1]:47897 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l1Vxj-0006Db-23 for submit@debbugs.gnu.org; Mon, 18 Jan 2021 09:56:59 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40408) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l1Vxg-0006DN-OT for 45891-done@debbugs.gnu.org; Mon, 18 Jan 2021 09:56:57 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57316) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l1Vxb-0000uV-EW; Mon, 18 Jan 2021 09:56:51 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=55870 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l1Vxa-00018n-6b; Mon, 18 Jan 2021 09:56:50 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Maxim Cournoyer Subject: Re: bug#45891: [PATCH] packages: 'patch-and-repack' returns a directory when given a directory. References: <20210115131548.8792-1-ludo@gnu.org> <878s8tluev.fsf@gmail.com> <87lfcsy3jd.fsf@gnu.org> <87v9bukbjx.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 29 =?utf-8?Q?Niv=C3=B4se?= an 229 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 18 Jan 2021 15:56:49 +0100 In-Reply-To: <87v9bukbjx.fsf@gmail.com> (Maxim Cournoyer's message of "Mon, 18 Jan 2021 01:24:02 -0500") Message-ID: <87o8hmjnta.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 45891-done Cc: 45891-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Maxim Cournoyer skribis: > LGTM. Feel free to push! About my related change that we thought was > conflicting with this one; it at least applies on top of your change, > and only one test fails currently, I'm working on a fix. Alright. > Feel free to push to core-updates! Done, thanks! Ludo=E2=80=99. From unknown Wed Jun 18 23:11:40 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 16 Feb 2021 12:24:09 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator