From unknown Fri Jun 20 07:17:17 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#41382 <41382@debbugs.gnu.org> To: bug#41382 <41382@debbugs.gnu.org> Subject: Status: [PATCH 0/6] Allow for a cryptographic hash function migration Reply-To: bug#41382 <41382@debbugs.gnu.org> Date: Fri, 20 Jun 2025 14:17:17 +0000 retitle 41382 [PATCH 0/6] Allow for a cryptographic hash function migration reassign 41382 guix-patches submitter 41382 Ludovic Court=C3=A8s severity 41382 normal tag 41382 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Mon May 18 17:31:30 2020 Received: (at submit) by debbugs.gnu.org; 18 May 2020 21:31:30 +0000 Received: from localhost ([127.0.0.1]:48448 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janMA-0001V5-7y for submit@debbugs.gnu.org; Mon, 18 May 2020 17:31:30 -0400 Received: from lists.gnu.org ([209.51.188.17]:51708) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janM8-0001Uw-19 for submit@debbugs.gnu.org; Mon, 18 May 2020 17:31:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38640) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1janM7-00076f-QQ for guix-patches@gnu.org; Mon, 18 May 2020 17:31:27 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37535) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1janM7-0000HA-0d; Mon, 18 May 2020 17:31:27 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=50956 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1janM5-0003IX-WC; Mon, 18 May 2020 17:31:26 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: guix-patches@gnu.org Subject: [PATCH 0/6] Allow for a cryptographic hash function migration Date: Mon, 18 May 2020 23:31:16 +0200 Message-Id: <20200518213116.23978-1-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello Guix! This is a first stab at preparing for a possible migration to hash algorithms other than SHA256 (there’s no rush AFAIK, but it’s good to be prepared). The first bits just improve tests and adjust the tools, which doesn’t hurt. The last bit adds a ‘sha512’ field to , which we could eventually provide instead of or in addition to ‘sha256’, possibly as a base64-encoded string so that it’s not too large. Another option would be to create a data type that specifies its algorithm and its value. We’d replace the ‘sha256’ field with a ‘hash’ field of that type (in a backward-compatible way). Thinking about it, this is perhaps the better option. Other bits that would need to be adjusted include importer, updaters, and ‘guix publish’. Thoughts? Ludo’. Ludovic Courtès (6): tests: Test 'add-to-store' with several hash algorithms. tests: Test fixed-output derivations with several hash algorithms. guix hash, guix download: Add '--hash'. guix hash, guix download: Support base64 format. packages: Add 'sha512' optional field to . packages: Add 'base64' macro. doc/guix.texi | 25 +++++++++++++++-- guix/packages.scm | 56 +++++++++++++++++++++++---------------- guix/scripts/download.scm | 26 +++++++++++++----- guix/scripts/hash.scm | 35 +++++++++++++++++------- tests/derivations.scm | 39 ++++++++++++++++----------- tests/guix-hash.sh | 7 ++++- tests/packages.scm | 26 ++++++++++++++++++ tests/store.scm | 12 +++++++++ 8 files changed, 170 insertions(+), 56 deletions(-) -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Mon May 18 17:32:59 2020 Received: (at 41382) by debbugs.gnu.org; 18 May 2020 21:32:59 +0000 Received: from localhost ([127.0.0.1]:48453 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNb-0001XT-Ia for submit@debbugs.gnu.org; Mon, 18 May 2020 17:32:59 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59552) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNa-0001XE-C6 for 41382@debbugs.gnu.org; Mon, 18 May 2020 17:32:58 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37551) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1janNU-0000Po-N7; Mon, 18 May 2020 17:32:52 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=50964 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1janNU-0007J8-68; Mon, 18 May 2020 17:32:52 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41382@debbugs.gnu.org Subject: [PATCH 1/6] tests: Test 'add-to-store' with several hash algorithms. Date: Mon, 18 May 2020 23:32:39 +0200 Message-Id: <20200518213244.24165-1-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41382 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * tests/store.scm ("add-to-store"): New test. --- tests/store.scm | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/tests/store.scm b/tests/store.scm index 0af099c1ad..f007846dc1 100644 --- a/tests/store.scm +++ b/tests/store.scm @@ -115,6 +115,18 @@ (passwd:name (getpwuid (getuid))))))) (list (stat:uid s) (stat:perms s)))) +(test-equal "add-to-store" + '("sha1" "sha256" "sha512") + (let* ((file (search-path %load-path "guix.scm")) + (content (call-with-input-file file get-bytevector-all))) + (map (lambda (hash-algo) + (let ((file (add-to-store %store "guix.scm" #f hash-algo file))) + (and (direct-store-path? file) + (bytevector=? (call-with-input-file file get-bytevector-all) + content) + hash-algo))) + '("sha1" "sha256" "sha512")))) + (test-equal "add-data-to-store" #vu8(1 2 3 4 5) (call-with-input-file (add-data-to-store %store "data" #vu8(1 2 3 4 5)) -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Mon May 18 17:33:04 2020 Received: (at 41382) by debbugs.gnu.org; 18 May 2020 21:33:04 +0000 Received: from localhost ([127.0.0.1]:48459 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNf-0001YJ-RN for submit@debbugs.gnu.org; Mon, 18 May 2020 17:33:04 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59556) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNb-0001XH-QJ for 41382@debbugs.gnu.org; Mon, 18 May 2020 17:33:00 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37555) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1janNW-0000Q2-Iw; Mon, 18 May 2020 17:32:54 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=50964 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1janNV-0007J8-B2; Mon, 18 May 2020 17:32:53 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41382@debbugs.gnu.org Subject: [PATCH 2/6] tests: Test fixed-output derivations with several hash algorithms. Date: Mon, 18 May 2020 23:32:40 +0200 Message-Id: <20200518213244.24165-2-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200518213244.24165-1-ludo@gnu.org> References: <20200518213244.24165-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41382 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * tests/derivations.scm ("fixed-output derivation"): Test several hash algorithms. --- tests/derivations.scm | 39 ++++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 15 deletions(-) diff --git a/tests/derivations.scm b/tests/derivations.scm index ef6cec6c76..a409fa99f0 100644 --- a/tests/derivations.scm +++ b/tests/derivations.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès +;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -324,20 +324,29 @@ #:hash hash #:hash-algo 'sha256))) (fixed-output-derivation? drv))) -(test-assert "fixed-output derivation" - (let* ((builder (add-text-to-store %store "my-fixed-builder.sh" - "echo -n hello > $out" '())) - (hash (sha256 (string->utf8 "hello"))) - (drv (derivation %store "fixed" - %bash `(,builder) - #:sources `(,builder) ;optional - #:hash hash #:hash-algo 'sha256)) - (succeeded? (build-derivations %store (list drv)))) - (and succeeded? - (let ((p (derivation->output-path drv))) - (and (equal? (string->utf8 "hello") - (call-with-input-file p get-bytevector-all)) - (bytevector? (query-path-hash %store p))))))) +(test-equal "fixed-output derivation" + '(sha1 sha256 sha512) + (map (lambda (hash-algorithm) + (let* ((builder (add-text-to-store %store "my-fixed-builder.sh" + "echo -n hello > $out" '())) + (sha256 (sha256 (string->utf8 "hello"))) + (hash (bytevector-hash + (string->utf8 "hello") + (lookup-hash-algorithm hash-algorithm))) + (drv (derivation %store + (string-append + "fixed-" (symbol->string hash-algorithm)) + %bash `(,builder) + #:sources `(,builder) ;optional + #:hash hash + #:hash-algo hash-algorithm))) + (build-derivations %store (list drv)) + (let ((p (derivation->output-path drv))) + (and (bytevector=? (string->utf8 "hello") + (call-with-input-file p get-bytevector-all)) + (bytevector? (query-path-hash %store p)) + hash-algorithm)))) + '(sha1 sha256 sha512))) (test-assert "fixed-output derivation: output paths are equal" (let* ((builder1 (add-text-to-store %store "fixed-builder1.sh" -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Mon May 18 17:33:04 2020 Received: (at 41382) by debbugs.gnu.org; 18 May 2020 21:33:04 +0000 Received: from localhost ([127.0.0.1]:48461 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNg-0001YL-4b for submit@debbugs.gnu.org; Mon, 18 May 2020 17:33:04 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59566) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNd-0001XK-Vs for 41382@debbugs.gnu.org; Mon, 18 May 2020 17:33:02 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37557) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1janNY-0000QD-Nz; Mon, 18 May 2020 17:32:56 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=50964 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1janNX-0007J8-ND; Mon, 18 May 2020 17:32:56 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41382@debbugs.gnu.org Subject: [PATCH 3/6] guix hash, guix download: Add '--hash'. Date: Mon, 18 May 2020 23:32:41 +0200 Message-Id: <20200518213244.24165-3-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200518213244.24165-1-ludo@gnu.org> References: <20200518213244.24165-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41382 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/scripts/download.scm (%default-options): Add 'hash-algorithm'. (show-help, %options): Add "--hash". (guix-download): Honor it. * guix/scripts/hash.scm (%default-options): Add 'hash-algorithm'. (show-help, %options): Add "--hash". (guix-hash): Honor it. * tests/guix-hash.sh: Test '-H sha512'. * doc/guix.texi (Invoking guix download): Document it. (Invoking guix hash): Document it. --- doc/guix.texi | 15 +++++++++++++++ guix/scripts/download.scm | 14 ++++++++++++-- guix/scripts/hash.scm | 21 +++++++++++++++++---- tests/guix-hash.sh | 6 +++++- 4 files changed, 49 insertions(+), 7 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index eef5b703fe..0cf006770e 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -9018,6 +9018,11 @@ Certificates}), unless @option{--no-check-certificate} is used. The following options are available: @table @code +@item --hash=@var{algorithm} +@itemx -H @var{algorithm} +Compute a hash using the specified @var{algorithm}. @xref{Invoking guix +hash}, for more information. + @item --format=@var{fmt} @itemx -f @var{fmt} Write the hash in the format specified by @var{fmt}. For more @@ -9057,6 +9062,16 @@ following options: @table @code +@item --hash=@var{algorithm} +@itemx -H @var{algorithm} +Compute a hash using the specified @var{algorithm}, @code{sha256} by +default. + +@var{algorithm} must the name of a cryptographic hash algorithm +supported by Libgcrypt @i{via} Guile-Gcrypt---e.g., @code{sha512} or +@code{sha3-256} (@pxref{Hash Functions,,, guile-gcrypt, Guile-Gcrypt +Reference Manual}). + @item --format=@var{fmt} @itemx -f @var{fmt} Write the hash in the format specified by @var{fmt}. diff --git a/guix/scripts/download.scm b/guix/scripts/download.scm index 22cd75ea0b..b4446c2e2f 100644 --- a/guix/scripts/download.scm +++ b/guix/scripts/download.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012, 2013, 2015, 2016, 2017 Ludovic Courtès +;;; Copyright © 2012, 2013, 2015, 2016, 2017, 2020 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -77,6 +77,7 @@ (define %default-options ;; Alist of default option values. `((format . ,bytevector->nix-base32-string) + (hash-algorithm . ,(hash-algorithm sha256)) (verify-certificate? . #t) (download-proc . ,download-to-store*))) @@ -89,6 +90,8 @@ Supported formats: 'nix-base32' (default), 'base32', and 'base16' ('hex' and 'hexadecimal' can be used as well).\n")) (format #t (G_ " -f, --format=FMT write the hash in the given format")) + (format #t (G_ " + -H, --hash=ALGORITHM use the given hash ALGORITHM")) (format #t (G_ " --no-check-certificate do not validate the certificate of HTTPS servers ")) @@ -119,6 +122,13 @@ Supported formats: 'nix-base32' (default), 'base32', and 'base16' (alist-cons 'format fmt-proc (alist-delete 'format result)))) + (option '(#\H "hash") #t #f + (lambda (opt name arg result) + (match (lookup-hash-algorithm (string->symbol arg)) + (#f + (leave (G_ "~a: unknown hash algorithm~%") arg)) + (algo + (alist-cons 'hash-algorithm algo result))))) (option '("no-check-certificate") #f #f (lambda (opt name arg result) (alist-cons 'verify-certificate? #f result))) @@ -175,7 +185,7 @@ Supported formats: 'nix-base32' (default), 'base32', and 'base16' (or path (leave (G_ "~a: download failed~%") arg)) - port-sha256)) + (cute port-hash (assoc-ref opts 'hash-algorithm) <>))) (fmt (assq-ref opts 'format))) (format #t "~a~%~a~%" path (fmt hash)) #t))) diff --git a/guix/scripts/hash.scm b/guix/scripts/hash.scm index b8b2158195..cfc4420260 100644 --- a/guix/scripts/hash.scm +++ b/guix/scripts/hash.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012, 2013, 2014, 2016, 2017 Ludovic Courtès +;;; Copyright © 2012, 2013, 2014, 2016, 2017, 2020 Ludovic Courtès ;;; Copyright © 2013 Nikita Karetnikov ;;; Copyright © 2016 Jan Nieuwenhuizen ;;; Copyright © 2018 Tim Gesthuizen @@ -42,7 +42,8 @@ (define %default-options ;; Alist of default option values. - `((format . ,bytevector->nix-base32-string))) + `((format . ,bytevector->nix-base32-string) + (hash-algorithm . ,(hash-algorithm sha256)))) (define (show-help) (display (G_ "Usage: guix hash [OPTION] FILE @@ -53,6 +54,8 @@ and 'hexadecimal' can be used as well).\n")) (format #t (G_ " -x, --exclude-vcs exclude version control directories")) (format #t (G_ " + -H, --hash=ALGORITHM use the given hash ALGORITHM")) + (format #t (G_ " -f, --format=FMT write the hash in the given format")) (format #t (G_ " -r, --recursive compute the hash on FILE recursively")) @@ -69,6 +72,13 @@ and 'hexadecimal' can be used as well).\n")) (list (option '(#\x "exclude-vcs") #f #f (lambda (opt name arg result) (alist-cons 'exclude-vcs? #t result))) + (option '(#\H "hash") #t #f + (lambda (opt name arg result) + (match (lookup-hash-algorithm (string->symbol arg)) + (#f + (leave (G_ "~a: unknown hash algorithm~%") arg)) + (algo + (alist-cons 'hash-algorithm algo result))))) (option '(#\f "format") #t #f (lambda (opt name arg result) (define fmt-proc @@ -139,8 +149,11 @@ and 'hexadecimal' can be used as well).\n")) (force-output port) (get-hash)) (match file - ("-" (port-sha256 (current-input-port))) - (_ (call-with-input-file file port-sha256)))))) + ("-" (port-hash (assoc-ref opts 'hash-algorithm) + (current-input-port))) + (_ (call-with-input-file file + (cute port-hash (assoc-ref opts 'hash-algorithm) + <>))))))) (match args ((file) diff --git a/tests/guix-hash.sh b/tests/guix-hash.sh index 190c9e7f8a..1c595b49ed 100644 --- a/tests/guix-hash.sh +++ b/tests/guix-hash.sh @@ -1,5 +1,5 @@ # GNU Guix --- Functional package management for GNU -# Copyright © 2013, 2014, 2016 Ludovic Courtès +# Copyright © 2013, 2014, 2016, 2020 Ludovic Courtès # Copyright © 2016 Jan Nieuwenhuizen # # This file is part of GNU Guix. @@ -31,6 +31,10 @@ test `echo -n | guix hash -` = 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9 test `guix hash -f nix-base32 /dev/null` = 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9c73 test `guix hash -f hex /dev/null` = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 test `guix hash -f base32 /dev/null` = 4oymiquy7qobjgx36tejs35zeqt24qpemsnzgtfeswmrw6csxbkq +test `guix hash -H sha512 -f hex /dev/null` = cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e + +if guix hash -H abcd1234 /dev/null; +then false; else true; fi mkdir "$tmpdir" echo -n executable > "$tmpdir/exe" -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Mon May 18 17:33:09 2020 Received: (at 41382) by debbugs.gnu.org; 18 May 2020 21:33:09 +0000 Received: from localhost ([127.0.0.1]:48465 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNk-0001Ym-LR for submit@debbugs.gnu.org; Mon, 18 May 2020 17:33:08 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59570) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNf-0001XM-2r for 41382@debbugs.gnu.org; Mon, 18 May 2020 17:33:03 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37558) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1janNZ-0000QI-RM; Mon, 18 May 2020 17:32:57 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=50964 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1janNZ-0007J8-20; Mon, 18 May 2020 17:32:57 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41382@debbugs.gnu.org Subject: [PATCH 4/6] guix hash, guix download: Support base64 format. Date: Mon, 18 May 2020 23:32:42 +0200 Message-Id: <20200518213244.24165-4-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200518213244.24165-1-ludo@gnu.org> References: <20200518213244.24165-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41382 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/scripts/download.scm (show-help, %options): Support "base64" format. * guix/scripts/hash.scm (show-help, %options): Likewise. * tests/guix-hash.sh: Test it. * doc/guix.texi (Invoking guix hash): Document it. --- doc/guix.texi | 2 +- guix/scripts/download.scm | 12 ++++++++---- guix/scripts/hash.scm | 14 +++++++++----- tests/guix-hash.sh | 1 + 4 files changed, 19 insertions(+), 10 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 0cf006770e..fdd9622211 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -9076,7 +9076,7 @@ Reference Manual}). @itemx -f @var{fmt} Write the hash in the format specified by @var{fmt}. -Supported formats: @code{nix-base32}, @code{base32}, @code{base16} +Supported formats: @code{base64}, @code{nix-base32}, @code{base32}, @code{base16} (@code{hex} and @code{hexadecimal} can be used as well). If the @option{--format} option is not specified, @command{guix hash} diff --git a/guix/scripts/download.scm b/guix/scripts/download.scm index b4446c2e2f..589f62da9d 100644 --- a/guix/scripts/download.scm +++ b/guix/scripts/download.scm @@ -23,6 +23,7 @@ #:use-module (gcrypt hash) #:use-module (guix base16) #:use-module (guix base32) + #:autoload (guix base64) (base64-encode) #:use-module ((guix download) #:hide (url-fetch)) #:use-module ((guix build download) #:select (url-fetch)) @@ -84,10 +85,11 @@ (define (show-help) (display (G_ "Usage: guix download [OPTION] URL Download the file at URL to the store or to the given file, and print its -file name and the hash of its contents. - -Supported formats: 'nix-base32' (default), 'base32', and 'base16' -('hex' and 'hexadecimal' can be used as well).\n")) +file name and the hash of its contents.\n")) + (newline) + (display (G_ "\ +Supported formats: 'base64', 'nix-base32' (default), 'base32', +and 'base16' ('hex' and 'hexadecimal' can be used as well).\n")) (format #t (G_ " -f, --format=FMT write the hash in the given format")) (format #t (G_ " @@ -111,6 +113,8 @@ Supported formats: 'nix-base32' (default), 'base32', and 'base16' (lambda (opt name arg result) (define fmt-proc (match arg + ("base64" + base64-encode) ("nix-base32" bytevector->nix-base32-string) ("base32" diff --git a/guix/scripts/hash.scm b/guix/scripts/hash.scm index cfc4420260..9b4f419a24 100644 --- a/guix/scripts/hash.scm +++ b/guix/scripts/hash.scm @@ -20,12 +20,13 @@ ;;; along with GNU Guix. If not, see . (define-module (guix scripts hash) - #:use-module (guix base32) #:use-module (gcrypt hash) #:use-module (guix serialization) #:use-module (guix ui) #:use-module (guix scripts) #:use-module (guix base16) + #:use-module (guix base32) + #:autoload (guix base64) (base64-encode) #:use-module (ice-9 binary-ports) #:use-module (rnrs files) #:use-module (ice-9 match) @@ -47,10 +48,11 @@ (define (show-help) (display (G_ "Usage: guix hash [OPTION] FILE -Return the cryptographic hash of FILE. - -Supported formats: 'nix-base32' (default), 'base32', and 'base16' ('hex' -and 'hexadecimal' can be used as well).\n")) +Return the cryptographic hash of FILE.\n")) + (newline) + (display (G_ "\ +Supported formats: 'base64', 'nix-base32' (default), 'base32', +and 'base16' ('hex' and 'hexadecimal' can be used as well).\n")) (format #t (G_ " -x, --exclude-vcs exclude version control directories")) (format #t (G_ " @@ -83,6 +85,8 @@ and 'hexadecimal' can be used as well).\n")) (lambda (opt name arg result) (define fmt-proc (match arg + ("base64" + base64-encode) ("nix-base32" bytevector->nix-base32-string) ("base32" diff --git a/tests/guix-hash.sh b/tests/guix-hash.sh index 1c595b49ed..3538b9aeda 100644 --- a/tests/guix-hash.sh +++ b/tests/guix-hash.sh @@ -32,6 +32,7 @@ test `guix hash -f nix-base32 /dev/null` = 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lz test `guix hash -f hex /dev/null` = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 test `guix hash -f base32 /dev/null` = 4oymiquy7qobjgx36tejs35zeqt24qpemsnzgtfeswmrw6csxbkq test `guix hash -H sha512 -f hex /dev/null` = cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e +test `guix hash -H sha1 -f base64 /dev/null` = "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" if guix hash -H abcd1234 /dev/null; then false; else true; fi -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Mon May 18 17:33:21 2020 Received: (at 41382) by debbugs.gnu.org; 18 May 2020 21:33:21 +0000 Received: from localhost ([127.0.0.1]:48467 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNl-0001Yp-0f for submit@debbugs.gnu.org; Mon, 18 May 2020 17:33:21 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59574) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNg-0001Xb-UN for 41382@debbugs.gnu.org; Mon, 18 May 2020 17:33:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37559) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1janNb-0000QN-Mu; Mon, 18 May 2020 17:32:59 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=50964 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1janNa-0007J8-PW; Mon, 18 May 2020 17:32:59 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41382@debbugs.gnu.org Subject: [PATCH 5/6] packages: Add 'sha512' optional field to . Date: Mon, 18 May 2020 23:32:43 +0200 Message-Id: <20200518213244.24165-5-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200518213244.24165-1-ludo@gnu.org> References: <20200518213244.24165-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41382 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/packages.scm ()[sha512]: New field. (print-origin): Honor it. (origin->derivation): Likewise. * tests/packages.scm ("package-source-derivation, origin, sha512"): New test. * doc/guix.texi (origin Reference): Document 'sha512'. --- doc/guix.texi | 8 +++++++- guix/packages.scm | 25 ++++++++++++++----------- tests/packages.scm | 26 ++++++++++++++++++++++++++ 3 files changed, 47 insertions(+), 12 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index fdd9622211..50d7eb7a43 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5966,10 +5966,16 @@ specified in the @code{uri} field as a @code{git-reference} object; a @end table @item @code{sha256} -A bytevector containing the SHA-256 hash of the source. Typically the +@itemx @code{sha512} +A bytevector containing the SHA-256 (respectively SHA-512) hash of the source. Typically the @code{base32} form is used here to generate the bytevector from a base-32 string. +One of these fields must be a bytevector while the others can be +@code{#f}. When several hashes are provided, the ``strongest'' is used +when computing the underlying fixed-output derivation +(@pxref{Derivations}). + You can obtain this information using @code{guix download} (@pxref{Invoking guix download}) or @code{guix hash} (@pxref{Invoking guix hash}). diff --git a/guix/packages.scm b/guix/packages.scm index 3fff50a6e8..7cf4c9c3e6 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -164,6 +164,7 @@ (uri origin-uri) ; string (method origin-method) ; procedure (sha256 origin-sha256) ; bytevector + (sha512 origin-sha512 (default #f)) ; bytevector | #f (file-name origin-file-name (default #f)) ; optional file name ;; Patches are delayed so that the 'search-patch' calls are made lazily, @@ -189,9 +190,9 @@ (define (print-origin origin port) "Write a concise representation of ORIGIN to PORT." (match origin - (($ uri method sha256 file-name patches) + (($ uri method sha256 sha512 file-name patches) (simple-format port "#" - uri (bytevector->base32-string sha256) + uri (bytevector->base32-string (or sha512 sha256)) (force patches) (number->string (object-address origin) 16))))) @@ -1381,18 +1382,20 @@ unless you know what you are doing." #:optional (system (%current-system))) "Return the derivation corresponding to ORIGIN." (match origin - (($ uri method sha256 name (= force ()) #f) + (($ uri method sha256 sha512 name (= force ()) #f) ;; No patches, no snippet: this is a fixed-output derivation. - (method uri 'sha256 sha256 name #:system system)) - (($ uri method sha256 name (= force (patches ...)) snippet + (let ((algorithm (if sha512 'sha512 'sha256))) + (method uri algorithm (or sha512 sha256) name #:system system))) + (($ uri method sha256 sha512 name (= force (patches ...)) snippet (flags ...) inputs (modules ...) guile-for-build) ;; Patches and/or a snippet. - (mlet %store-monad ((source (method uri 'sha256 sha256 name - #:system system)) - (guile (package->derivation (or guile-for-build - (default-guile)) - system - #:graft? #f))) + (mlet* %store-monad ((algorithm -> (if sha512 'sha512 'sha256)) + (source (method uri algorithm (or sha512 sha256) + name #:system system)) + (guile (package->derivation (or guile-for-build + (default-guile)) + system + #:graft? #f))) (patch-and-repack source patches #:inputs inputs #:snippet snippet diff --git a/tests/packages.scm b/tests/packages.scm index c528d2080c..06d41b5ce7 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -51,6 +51,7 @@ #:use-module (srfi srfi-34) #:use-module (srfi srfi-35) #:use-module (srfi srfi-64) + #:use-module (rnrs bytevectors) #:use-module (rnrs io ports) #:use-module (ice-9 vlist) #:use-module (ice-9 regex) @@ -497,6 +498,31 @@ (search-path %load-path "guix/base32.scm") get-bytevector-all))))) +(test-equal "package-source-derivation, origin, sha512" + "hello" + (let* ((bash (search-bootstrap-binary "bash" (%current-system))) + (builder (add-text-to-store %store "my-fixed-builder.sh" + "echo -n hello > $out" '())) + (method (lambda* (url hash-algo hash #:optional name + #:rest rest) + (and (eq? hash-algo 'sha512) + (raw-derivation name bash (list builder) + #:sources (list builder) + #:hash hash + #:hash-algo hash-algo)))) + (source (origin + (method method) + (uri "unused://") + (file-name "origin-sha512") + (sha256 (bytevector-hash (string->utf8 "hello") + (hash-algorithm sha256))) + (sha512 (bytevector-hash (string->utf8 "hello") + (hash-algorithm sha512))))) + (drv (package-source-derivation %store source)) + (output (derivation->output-path drv))) + (build-derivations %store (list drv)) + (call-with-input-file output get-string-all))) + (unless (network-reachable?) (test-skip 1)) (test-equal "package-source-derivation, snippet" "OK" -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Mon May 18 17:33:21 2020 Received: (at 41382) by debbugs.gnu.org; 18 May 2020 21:33:21 +0000 Received: from localhost ([127.0.0.1]:48469 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNx-0001ZG-C5 for submit@debbugs.gnu.org; Mon, 18 May 2020 17:33:21 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59578) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1janNh-0001Xh-T1 for 41382@debbugs.gnu.org; Mon, 18 May 2020 17:33:06 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37560) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1janNc-0000QU-LM; Mon, 18 May 2020 17:33:00 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=50964 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1janNc-0007J8-0x; Mon, 18 May 2020 17:33:00 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: 41382@debbugs.gnu.org Subject: [PATCH 6/6] packages: Add 'base64' macro. Date: Mon, 18 May 2020 23:32:44 +0200 Message-Id: <20200518213244.24165-6-ludo@gnu.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200518213244.24165-1-ludo@gnu.org> References: <20200518213244.24165-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41382 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/packages.scm (define-compile-time-decoder): New macro. (base32): Redefine in terms of it. (base64): New macro. --- guix/packages.scm | 31 +++++++++++++++++++------------ 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/guix/packages.scm b/guix/packages.scm index 7cf4c9c3e6..724d7693c7 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -28,6 +28,7 @@ #:use-module (guix monads) #:use-module (guix gexp) #:use-module (guix base32) + #:autoload (guix base64) (base64-decode) #:use-module (guix grafts) #:use-module (guix derivations) #:use-module (guix memoization) @@ -62,6 +63,7 @@ origin-snippet origin-modules base32 + base64 package package? @@ -198,19 +200,24 @@ (set-record-type-printer! print-origin) -(define-syntax base32 - (lambda (s) - "Return the bytevector corresponding to the given Nix-base32 +(define-syntax-rule (define-compile-time-decoder name string->bytevector) + "Define NAME as a macro that runs STRING->BYTEVECTOR at macro expansion time +if possible." + (define-syntax name + (lambda (s) + "Return the bytevector corresponding to the given textual representation." - (syntax-case s () - ((_ str) - (string? (syntax->datum #'str)) - ;; A literal string: do the conversion at expansion time. - (with-syntax ((bv (nix-base32-string->bytevector - (syntax->datum #'str)))) - #''bv)) - ((_ str) - #'(nix-base32-string->bytevector str))))) + (syntax-case s () + ((_ str) + (string? (syntax->datum #'str)) + ;; A literal string: do the conversion at expansion time. + (with-syntax ((bv (string->bytevector (syntax->datum #'str)))) + #''bv)) + ((_ str) + #'(string->bytevector str)))))) + +(define-compile-time-decoder base32 nix-base32-string->bytevector) +(define-compile-time-decoder base64 base64-decode) (define (origin-actual-file-name origin) "Return the file name of ORIGIN, either its 'file-name' field or the file -- 2.26.2 From debbugs-submit-bounces@debbugs.gnu.org Tue May 19 10:43:19 2020 Received: (at 41382) by debbugs.gnu.org; 19 May 2020 14:43:19 +0000 Received: from localhost ([127.0.0.1]:50725 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jb3Sa-0004uA-7N for submit@debbugs.gnu.org; Tue, 19 May 2020 10:43:19 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35622) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jb3SU-0004tf-Jb for 41382@debbugs.gnu.org; Tue, 19 May 2020 10:43:11 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52485) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jb3SP-00005g-B8 for 41382@debbugs.gnu.org; Tue, 19 May 2020 10:43:01 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=53110 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jb3SO-00020K-E0 for 41382@debbugs.gnu.org; Tue, 19 May 2020 10:43:00 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 41382@debbugs.gnu.org Subject: Re: [bug#41382] [PATCH 0/6] Allow for a cryptographic hash function migration References: <20200518213116.23978-1-ludo@gnu.org> Date: Tue, 19 May 2020 16:42:58 +0200 In-Reply-To: <20200518213116.23978-1-ludo@gnu.org> ("Ludovic \=\?utf-8\?Q\?Cou\?\= \=\?utf-8\?Q\?rt\=C3\=A8s\=22's\?\= message of "Mon, 18 May 2020 23:31:16 +0200") Message-ID: <871rnggf4d.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 41382 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello, Ludovic Court=C3=A8s skribis: > Another option would be to create a data type that specifies > its algorithm and its value. We=E2=80=99d replace the =E2=80=98sha256=E2= =80=99 field with > a =E2=80=98hash=E2=80=99 field of that type (in a backward-compatible way= ). Thinking > about it, this is perhaps the better option. Here=E2=80=99s a v2 that does that: instead of adding a =E2=80=98sha512=E2= =80=99 field to , it replaces the =E2=80=98sha256=E2=80=99 field with =E2=80=98hash= =E2=80=99 and introduces a data type (similar to the data type we have). One can now write things like: (origin ;; =E2=80=A6 (hash (content-hash (base64 "=E2=80=A6") sha512))) Since it=E2=80=99s a bit verbose, one can also pass a literal string direct= ly, in which case it=E2=80=99s base32-decoded: (origin ;; =E2=80=A6 (hash (content-hash "=E2=80=A6"))) =E2=80=98content-hash=E2=80=99 uses macrology to validate as much as possib= le at macro-expansion time. There=E2=80=99s a compatibility =E2=80=98origin=E2=80=99 macro intended to = allow people to keep writing: (origin (url =E2=80=A6) (method =E2=80=A6) (sha256 =E2=80=A6)) and to automatically =E2=80=9Cconvert=E2=80=9D the =E2=80=98sha256=E2=80=99= field specification to a =E2=80=98content-hash=E2=80=99. Due to the way identifiers are matched, th= ere are cases where we can=E2=80=99t preserve the illusion of compatibility, as can be se= en with the patch below. Perhaps that=E2=80=99s acceptable, though. Thoughts? Thanks, Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-packages-Introduce-content-hash-and-use-it-in-origin.patch Content-Transfer-Encoding: quoted-printable Content-Description: the patch >From 0736d19071cc898e30b0bf06b445e7434848c825 Mon Sep 17 00:00:00 2001 From: =3D?UTF-8?q?Ludovic=3D20Court=3DC3=3DA8s?=3D Date: Tue, 19 May 2020 15:55:08 +0200 Subject: [PATCH] packages: Introduce and use it in . * guix/packages.scm (): New record type. (define-content-hash-constructor, build-content-hash) (content-hash): New macros. (print-content-hash): New procedure. (): Rename constructor to '%origin'. [sha256]: Remove field. [hash]: New field. Adjust users. (origin-compatibility-helper, origin): New macros. (origin-sha256): New deprecated procedure. (origin->derivation): Adjust accordingly. * tests/packages.scm ("package-source-derivation, origin, sha512"): New test. * guix/tests.scm: Hide (gcrypt hash) 'sha256' for proper syntax matching. * gnu/packages/aspell.scm (aspell-dictionary) (aspell-dict-ca, aspell-dict-it): Use 'hash' and 'content-hash' for proper syntax matching. * gnu/packages/bash.scm (bash-patch): Rename 'sha256' to 'sha256-bv'. * gnu/packages/bootstrap.scm (bootstrap-executable): Rename 'sha256' to 'bv= '. * gnu/packages/readline.scm (readline-patch): Likewise. * gnu/packages/virtualization.scm (qemu-patch): Rename 'sha256' to 'sha256-bv'. * guix/import/utils.scm: Hide (gcrypt hash) 'sha256'. --- doc/guix.texi | 34 ++++++++- gnu/packages/aspell.scm | 8 +- gnu/packages/bash.scm | 8 +- gnu/packages/bootstrap.scm | 6 +- gnu/packages/readline.scm | 8 +- gnu/packages/virtualization.scm | 4 +- guix/import/utils.scm | 2 +- guix/packages.scm | 126 +++++++++++++++++++++++++++++--- guix/tests.scm | 2 +- tests/packages.scm | 28 ++++++- 10 files changed, 192 insertions(+), 34 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index fdd9622211..71b10a141d 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5966,9 +5966,13 @@ specified in the @code{uri} field as a @code{git-ref= erence} object; a @end table =20 @item @code{sha256} -A bytevector containing the SHA-256 hash of the source. Typically the -@code{base32} form is used here to generate the bytevector from a -base-32 string. +A bytevector containing the SHA-256 hash of the source. This is +equivalent to providing a @code{content-hash} SHA256 object in the +@code{hash} field described below. + +@item @code{hash} +The @code{content-hash} object of the source---see below for how to use +@code{content-hash}. =20 You can obtain this information using @code{guix download} (@pxref{Invoking guix download}) or @code{guix hash} (@pxref{Invoking @@ -6013,6 +6017,30 @@ this is @code{#f}, a sensible default is used. @end table @end deftp =20 +@deftp {Data Type} content-hash @var{value} [@var{algorithm}] +Construct a content hash object for the given @var{algorithm}, and with +@var{value} as its hash value. When @var{algorithm} is omitted, assume +it is @code{sha256}. + +@var{value} can be a literal string, in which case it is base32-decoded, +or it can be a bytevector. + +The following forms are all equivalent: + +@lisp +(content-hash "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj") +(content-hash "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj" + sha256) +(content-hash (base32 + "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj")) +(content-hash (base64 "kkb+RPaP7uyMZmu4eXPVkM4BN8yhRd8BTHLslb6f/Rc=3D") + sha256) +@end lisp + +Technically, @code{content-hash} is currently implemented as a macro. +It performs sanity checks at macro-expansion time, when possible, such +as ensuring that @var{value} has the right size for @var{algorithm}. +@end deftp =20 @node Build Systems @section Build Systems diff --git a/gnu/packages/aspell.scm b/gnu/packages/aspell.scm index 7550736c40..22256f750b 100644 --- a/gnu/packages/aspell.scm +++ b/gnu/packages/aspell.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright =C2=A9 2013, 2014, 2015, 2017, 2018, 2019 Ludovic Court=C3= =A8s +;;; Copyright =C2=A9 2013, 2014, 2015, 2017, 2018, 2019, 2020 Ludovic Cour= t=C3=A8s ;;; Copyright =C2=A9 2015, 2016 Alex Kost ;;; Copyright =C2=A9 2016 John Darrington ;;; Copyright =C2=A9 2016, 2017, 2019 Efraim Flashner @@ -111,7 +111,7 @@ dictionaries, including personal ones.") (uri (string-append "mirror://gnu/aspell/dict/" dict-name "/" prefix dict-name "-" version ".tar.bz2")) - (sha256 sha256))) + (hash (content-hash sha256)))) (build-system gnu-build-system) (arguments `(#:phases @@ -163,7 +163,7 @@ dictionaries, including personal ones.") (method url-fetch) (uri (string-append "https://www.softcatala.org/pub/softcatala/as= pell/" version "/aspell6-ca-" version ".tar.bz2")) - (sha256 sha256))) + (hash (content-hash sha256)))) (home-page "https://www.softcatala.org/pub/softcatala/aspell/")))) =20 (define-public aspell-dict-de @@ -264,7 +264,7 @@ dictionaries, including personal ones.") (uri (string-append "mirror://sourceforge/linguistico/" "Dizionario%20italiano%20per%20Aspell/" versi= on "/" "aspell6-it-" version ".tar.bz2")) - (sha256 sha256))) + (hash (content-hash sha256)))) (home-page "http://linguistico.sourceforge.net/pages/dizionario_italiano.html= ")))) =20 diff --git a/gnu/packages/bash.scm b/gnu/packages/bash.scm index 1b342827c5..311e07a944 100644 --- a/gnu/packages/bash.scm +++ b/gnu/packages/bash.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright =C2=A9 2012, 2013, 2014, 2015, 2016, 2017, 2019 Ludovic Cour= t=C3=A8s +;;; Copyright =C2=A9 2012, 2013, 2014, 2015, 2016, 2017, 2019, 2020 Ludovi= c Court=C3=A8s ;;; Copyright =C2=A9 2014, 2015, 2018 Mark H Weaver ;;; Copyright =C2=A9 2015, 2017 Leo Famulari ;;; Copyright =C2=A9 2016, 2017, 2018, 2019 Efraim Flashner @@ -48,12 +48,12 @@ "Return the URL of Bash patch number SEQNO." (format #f "mirror://gnu/bash/bash-5.0-patches/bash50-~3,'0d" seqno)) =20 -(define (bash-patch seqno sha256) - "Return the origin of Bash patch SEQNO, with expected hash SHA256" +(define (bash-patch seqno sha256-bv) + "Return the origin of Bash patch SEQNO, with expected hash SHA256-BV." (origin (method url-fetch) (uri (patch-url seqno)) - (sha256 sha256))) + (sha256 sha256-bv))) =20 (define-syntax-rule (patch-series (seqno hash) ...) (list (bash-patch seqno (base32 hash)) diff --git a/gnu/packages/bootstrap.scm b/gnu/packages/bootstrap.scm index f58ce2de93..a3ecb6e692 100644 --- a/gnu/packages/bootstrap.scm +++ b/gnu/packages/bootstrap.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright =C2=A9 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovi= c Court=C3=A8s +;;; Copyright =C2=A9 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 = Ludovic Court=C3=A8s ;;; Copyright =C2=A9 2014, 2015, 2018, 2019 Mark H Weaver ;;; Copyright =C2=A9 2017, 2020 Efraim Flashner ;;; Copyright =C2=A9 2018, 2020 Jan (janneke) Nieuwenhuizen @@ -151,14 +151,14 @@ built for SYSTEM." (format #f (G_ "could not find bootstrap binary '~a' \ for system '~a'") program system)))))) - ((sha256) + ((bv) (origin (method url-fetch/executable) (uri (map (cute string-append <> (bootstrap-executable-file-name system program)) %bootstrap-executable-base-urls)) (file-name program) - (sha256 sha256))))))) + (hash (content-hash bv sha256)))))))) =20 ;;; diff --git a/gnu/packages/readline.scm b/gnu/packages/readline.scm index 5f61dcb735..8a36883347 100644 --- a/gnu/packages/readline.scm +++ b/gnu/packages/readline.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright =C2=A9 2012, 2013, 2014 Ludovic Court=C3=A8s +;;; Copyright =C2=A9 2012, 2013, 2014, 2020 Ludovic Court=C3=A8s ;;; Copyright =C2=A9 2016, 2019 Efraim Flashner ;;; Copyright =C2=A9 2016 Jan Nieuwenhuizen ;;; Copyright =C2=A9 2018 Tobias Geerinckx-Rice @@ -35,12 +35,12 @@ (format #f "mirror://gnu/readline/readline-~a-patches/readline~a-~3,'0d" version (string-join (string-split version #\.) "") seqno)) =20 -(define (readline-patch version seqno sha256) - "Return the origin of Readline patch SEQNO, with expected hash SHA256" +(define (readline-patch version seqno sha256-bv) + "Return the origin of Readline patch SEQNO, with expected hash SHA256-BV" (origin (method url-fetch) (uri (patch-url version seqno)) - (sha256 sha256))) + (sha256 sha256-bv))) =20 (define-syntax-rule (patch-series version (seqno hash) ...) (list (readline-patch version seqno (base32 hash)) diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.= scm index c2025c4fbe..da110bf8c6 100644 --- a/gnu/packages/virtualization.scm +++ b/gnu/packages/virtualization.scm @@ -104,14 +104,14 @@ #:use-module (srfi srfi-1) #:use-module (ice-9 match)) =20 -(define (qemu-patch commit file-name sha256) +(define (qemu-patch commit file-name sha256-bv) "Return an origin for COMMIT." (origin (method url-fetch) (uri (string-append "http://git.qemu.org/?p=3Dqemu.git;a=3Dcommitdiff_plain;h=3D" commit)) - (sha256 sha256) + (hash (content-hash sha256-bv sha256)) (file-name file-name))) =20 (define-public qemu diff --git a/guix/import/utils.scm b/guix/import/utils.scm index 3809c3d074..0cfa1f8321 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -24,7 +24,7 @@ (define-module (guix import utils) #:use-module (guix base32) #:use-module ((guix build download) #:prefix build:) - #:use-module (gcrypt hash) + #:use-module ((gcrypt hash) #:hide (sha256)) #:use-module (guix http-client) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix utils) diff --git a/guix/packages.scm b/guix/packages.scm index c1c4805ae9..3d9988d836 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -35,6 +35,8 @@ #:use-module (guix build-system) #:use-module (guix search-paths) #:use-module (guix sets) + #:use-module (guix deprecation) + #:use-module (guix i18n) #:use-module (ice-9 match) #:use-module (ice-9 vlist) #:use-module (ice-9 regex) @@ -44,16 +46,23 @@ #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) #:use-module (srfi srfi-35) + #:use-module (rnrs bytevectors) #:use-module (web uri) #:re-export (%current-system %current-target-system search-path-specification) ;for convenience - #:export (origin + #:export (content-hash + content-hash? + content-hash-algorithm + content-hash-value + + origin origin? this-origin origin-uri origin-method - origin-sha256 + origin-hash + origin-sha256 ;deprecated origin-file-name origin-actual-file-name origin-patches @@ -157,15 +166,79 @@ ;;; ;;; Code: =20 +;; Crytographic content hash. +(define-immutable-record-type + (%content-hash algorithm value) + content-hash? + (algorithm content-hash-algorithm) ;symbol + (value content-hash-value)) ;bytevector + +(define-syntax-rule (define-content-hash-constructor name + (algorithm size) ...) + "Define NAME as a constructor that ensures that (1) its +second argument is among the listed ALGORITHM, and (2), when possible, that +its first argument has the right size for the chosen algorithm." + (define-syntax name + (lambda (s) + (syntax-case s (algorithm ...) + ((_ bv algorithm) + (let ((bv* (syntax->datum #'bv))) + (when (and (bytevector? bv*) + (not (=3D size (bytevector-length bv*)))) + (syntax-violation 'content-hash "invalid content hash length"= s)) + #'(%content-hash 'algorithm bv))) + ...)))) + +(define-content-hash-constructor build-content-hash + (sha256 32) + (sha512 64)) + +(define-syntax content-hash + (lambda (s) + "Return a content hash with the given parameters. The default hash +algorithm is sha256. If the first argument is a literal string, it is dec= oded +as base32. Otherwise, it must be a bytevector." + ;; What we'd really want here is something like C++ 'constexpr'. + (syntax-case s () + ((_ str) + (string? (syntax->datum #'str)) + #'(content-hash str sha256)) + ((_ str algorithm) + (string? (syntax->datum #'str)) + (with-syntax ((bv (base32 (syntax->datum #'str)))) + #'(content-hash bv algorithm))) + ((_ (id str) algorithm) + (and (string? (syntax->datum #'str)) + (free-identifier=3D? #'id #'base32)) + (with-syntax ((bv (nix-base32-string->bytevector (syntax->datum #'s= tr)))) + #'(content-hash bv algorithm))) + ((_ (id str) algorithm) + (and (string? (syntax->datum #'str)) + (free-identifier=3D? #'id #'base64)) + (with-syntax ((bv (base64-decode (syntax->datum #'str)))) + #'(content-hash bv algorithm))) + ((_ bv) + #'(content-hash bv sha256)) + ((_ bv hash) + #'(build-content-hash bv hash))))) + +(define (print-content-hash hash port) + (format port "#" + (content-hash-algorithm hash) + (bytevector->nix-base32-string (content-hash-value hash)))) + +(set-record-type-printer! print-content-hash) + + ;; The source of a package, such as a tarball URL and fetcher---called ;; "origin" to avoid name clash with `package-source', `source', etc. (define-record-type* - origin make-origin + %origin make-origin origin? this-origin (uri origin-uri) ; string (method origin-method) ; procedure - (sha256 origin-sha256) ; bytevector + (hash origin-hash) ; (file-name origin-file-name (default #f)) ; optional file name =20 ;; Patches are delayed so that the 'search-patch' calls are made lazily, @@ -188,12 +261,37 @@ (patch-guile origin-patch-guile ; package or #f (default #f))) =20 +(define-syntax origin-compatibility-helper + (syntax-rules (sha256) + ((_ () (fields ...)) + (%origin fields ...)) + ((_ ((sha256 exp) rest ...) (others ...)) + (%origin others ... + (hash (content-hash exp sha256)) + rest ...)) + ((_ (field rest ...) (others ...)) + (origin-compatibility-helper (rest ...) + (others ... field))))) + +(define-syntax-rule (origin fields ...) + "Build an record, automatically converting 'sha256' field +specifications to 'hash'." + (origin-compatibility-helper (fields ...) ())) + +(define-deprecated (origin-sha256 origin) + origin-hash + (let ((hash (origin-hash origin))) + (unless (eq? (content-hash-algorithm hash) 'sha256) + (raise (condition (&message + (message (G_ "no SHA256 hash for origin")))))) + (content-hash-value hash))) + (define (print-origin origin port) "Write a concise representation of ORIGIN to PORT." (match origin - (($ uri method sha256 file-name patches) + (($ uri method hash file-name patches) (simple-format port "#" - uri (bytevector->base32-string sha256) + uri hash (force patches) (number->string (object-address origin) 16))))) =20 @@ -238,6 +336,7 @@ name of its URI." ;; git, svn, cvs, etc. reference #f)))) =20 + (define %supported-systems ;; This is the list of system types that are supported. By default, we ;; expect all packages to build successfully here. @@ -1388,14 +1487,19 @@ unless you know what you are doing." #:optional (system (%current-system))) "Return the derivation corresponding to ORIGIN." (match origin - (($ uri method sha256 name (=3D force ()) #f) + (($ uri method hash name (=3D force ()) #f) ;; No patches, no snippet: this is a fixed-output derivation. - (method uri 'sha256 sha256 name #:system system)) - (($ uri method sha256 name (=3D force (patches ...)) snippet + (method uri + (content-hash-algorithm hash) + (content-hash-value hash) + name #:system system)) + (($ uri method hash name (=3D force (patches ...)) snippet (flags ...) inputs (modules ...) guile-for-build) ;; Patches and/or a snippet. - (mlet %store-monad ((source (method uri 'sha256 sha256 name - #:system system)) + (mlet %store-monad ((source (method uri + (content-hash-algorithm hash) + (content-hash-value hash) + name #:system system)) (guile (package->derivation (or guile-for-build (default-guile)) system diff --git a/guix/tests.scm b/guix/tests.scm index 95a7d7c4b8..3ccf049a7d 100644 --- a/guix/tests.scm +++ b/guix/tests.scm @@ -26,7 +26,7 @@ #:use-module (guix monads) #:use-module ((guix utils) #:select (substitute-keyword-arguments)) #:use-module ((guix build utils) #:select (mkdir-p)) - #:use-module (gcrypt hash) + #:use-module ((gcrypt hash) #:hide (sha256)) #:use-module (guix build-system gnu) #:use-module (gnu packages base) #:use-module (gnu packages bootstrap) diff --git a/tests/packages.scm b/tests/packages.scm index c528d2080c..4935d4503e 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -29,7 +29,7 @@ #:renamer (lambda (name) (cond ((eq? name 'location) 'make-location) (else name)))) - #:use-module (gcrypt hash) + #:use-module ((gcrypt hash) #:hide (sha256)) #:use-module (guix derivations) #:use-module (guix packages) #:use-module (guix grafts) @@ -51,6 +51,7 @@ #:use-module (srfi srfi-34) #:use-module (srfi srfi-35) #:use-module (srfi srfi-64) + #:use-module (rnrs bytevectors) #:use-module (rnrs io ports) #:use-module (ice-9 vlist) #:use-module (ice-9 regex) @@ -497,6 +498,31 @@ (search-path %load-path "guix/base32.scm") get-bytevector-all))))) =20 +(test-equal "package-source-derivation, origin, sha512" + "hello" + (let* ((bash (search-bootstrap-binary "bash" (%current-system))) + (builder (add-text-to-store %store "my-fixed-builder.sh" + "echo -n hello > $out" '())) + (method (lambda* (url hash-algo hash #:optional name + #:rest rest) + (and (eq? hash-algo 'sha512) + (raw-derivation name bash (list builder) + #:sources (list builder) + #:hash hash + #:hash-algo hash-algo)))) + (source (origin + (method method) + (uri "unused://") + (file-name "origin-sha512") + (hash (content-hash + (bytevector-hash (string->utf8 "hello") + (hash-algorithm sha512)) + sha512)))) + (drv (package-source-derivation %store source)) + (output (derivation->output-path drv))) + (build-derivations %store (list drv)) + (call-with-input-file output get-string-all))) + (unless (network-reachable?) (test-skip 1)) (test-equal "package-source-derivation, snippet" "OK" --=20 2.26.2 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue May 19 14:00:45 2020 Received: (at 41382) by debbugs.gnu.org; 19 May 2020 18:00:45 +0000 Received: from localhost ([127.0.0.1]:50872 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jb6Xl-0001Qk-Fv for submit@debbugs.gnu.org; Tue, 19 May 2020 14:00:45 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:33879) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jb6Xj-0001QX-SX for 41382@debbugs.gnu.org; Tue, 19 May 2020 14:00:44 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 936735C00EA; Tue, 19 May 2020 14:00:38 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Tue, 19 May 2020 14:00:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-type; s=fm2; bh=YaBsNFmkX4PdEDYEf+X7Eb5t9Y 3KByRDz/1EWSimqHQ=; b=V/Pqwc79hxP7lkcZidxpsuXflQPvl2qY61sVkxvrE8 4CR1yMRxr9tGYFLX0DWu+1GEkA/TG0BgrvSwFSv9mqdOhXLgMswqSSE70U7gx4ex rvLU9o4UrhxTl/95bs2xeRstfb4ms7rW1VCytac3Hfe8JSH5nR9n3GskYzdpT0Qb 7xWzTFXmaZJPo84YT6C1USvSDYjskXRYYG2/V5+BDb3j0F5mubbiMUEMj+PA8fmB xD+pFzAViQEjfKfUEiPVkqlDRPJzOy+xviTmn8ce78AkCnIOkL1i5V4607EIWWqi Z9+6nyO8RRj4Ax36O2knO8N5jegda/1pJdAxVYNdYjkw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=YaBsNF mkX4PdEDYEf+X7Eb5t9Y3KByRDz/1EWSimqHQ=; b=G56Rutfk/QS+2+y/Mh1xXg jpYfyc9AyhV54nTOPDG1LUlCjv1HoFDXzvgYPdWtbJykFA/79xnU1rrF42GUuB6S svnF6EGfsM7vLch7jHymky/3cil8zovXUWxMwDc1poDFl6yuP+tnc5bO7YeoCN6d RGIxm0eqdsXLcSlDUIWRGwQq3MgMuD0Aeo6gBZjnn0/OL8dM/HV5xEayvox8m6+y YrS7OlJ2vbs0dll+AhXJ1Q5eeyigGHEQb9LJRK5BLuIoIRYhec6UTWwJHe+9wOpB 4R9VLbA1l7kmKHU/5/fyvBDHzUmIuX2y+3tkEUeURWKupO7RmDl+RRG7OffKrg/Q == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedruddtjedguddulecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvffujghffffkgggtsehgtderredttdejnecuhfhrohhmpeforghrihhu shcuuegrkhhkvgcuoehmsggrkhhkvgesfhgrshhtmhgrihhlrdgtohhmqeenucggtffrrg htthgvrhhnpeffvdfhhfejffelfedvgedviefgfffgleeljeeghfeggfekheefffehtedu heehleenucffohhmrghinhepfihikhhiphgvughirgdrohhrghenucfkphepkedvrdduge ekrdduledurddukeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhl fhhrohhmpehmsggrkhhkvgesfhgrshhtmhgrihhlrdgtohhm X-ME-Proxy: Received: from localhost (ip82-148-191-18.breiband.no [82.148.191.18]) by mail.messagingengine.com (Postfix) with ESMTPA id 17FEB328005E; Tue, 19 May 2020 14:00:37 -0400 (EDT) From: Marius Bakke To: Ludovic =?utf-8?Q?Court=C3=A8s?= , 41382@debbugs.gnu.org Subject: Re: [bug#41382] [PATCH 0/6] Allow for a cryptographic hash function migration In-Reply-To: <871rnggf4d.fsf@gnu.org> References: <20200518213116.23978-1-ludo@gnu.org> <871rnggf4d.fsf@gnu.org> Date: Tue, 19 May 2020 20:00:34 +0200 Message-ID: <87wo57reil.fsf@devup.no> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 41382 Cc: guix-devel@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic, (+ guix-devel) Ludovic Court=C3=A8s writes: > Hello, > > Ludovic Court=C3=A8s skribis: > >> Another option would be to create a data type that specifies >> its algorithm and its value. We=E2=80=99d replace the =E2=80=98sha256= =E2=80=99 field with >> a =E2=80=98hash=E2=80=99 field of that type (in a backward-compatible wa= y). Thinking >> about it, this is perhaps the better option. > > Here=E2=80=99s a v2 that does that: instead of adding a =E2=80=98sha512= =E2=80=99 field to > , it replaces the =E2=80=98sha256=E2=80=99 field with =E2=80=98ha= sh=E2=80=99 and introduces a > data type (similar to the data type we have). > > One can now write things like: > > (origin > ;; =E2=80=A6 > (hash (content-hash (base64 "=E2=80=A6") sha512))) > > Since it=E2=80=99s a bit verbose, one can also pass a literal string dire= ctly, > in which case it=E2=80=99s base32-decoded: > > (origin > ;; =E2=80=A6 > (hash (content-hash "=E2=80=A6"))) > > =E2=80=98content-hash=E2=80=99 uses macrology to validate as much as poss= ible at > macro-expansion time. > > There=E2=80=99s a compatibility =E2=80=98origin=E2=80=99 macro intended t= o allow people to keep > writing: > > (origin > (url =E2=80=A6) > (method =E2=80=A6) > (sha256 =E2=80=A6)) > > and to automatically =E2=80=9Cconvert=E2=80=9D the =E2=80=98sha256=E2=80= =99 field specification to a > =E2=80=98content-hash=E2=80=99. Due to the way identifiers are matched, = there are cases > where we can=E2=80=99t preserve the illusion of compatibility, as can be = seen > with the patch below. Perhaps that=E2=80=99s acceptable, though. > > Thoughts? This is a great initiative, and the patches LGTM. I think that if we are to move away from SHA256, we should go with something that is immune to length extension attacks[0] such as BLAKE2/3 or SHA-3 (Keccak). Although I don't know any Guile implementations of those as of yet. SHA512 does not improve much security-wise IMO, but maybe it's worthwhile as s stop-gap. 0: https://en.wikipedia.org/wiki/Length_extension_attack --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAl7EHsIACgkQoqBt8qM6 VPo0swf9EISGA6w8OHyAb9oF1cg53FppycJhVYBkKkM8axnXeUAuRv5V9p/p24AL a6klRPCqtd+5Gupt/s0yitsOkTbgC3YF3S0Xjm7d2kqCQSiO2UakhKjqS8xK6+1X yabYyUVt7mO4eWRcehwZ7lUzBPlwE9fmuBU66Eq+eg70i7b0mIqdsHSIaZlGwjII Lu8J/qoBoTIxWaiLYEDkGnJUvTg7MZlr1o4ZNvwWDnnI+JmjQHVUAAWeikX0IQmW h1wPSGazmMX3NQ8vzJ0DlQunbNp7cOnbdUdym9LiMn1jHfVxgakYDJXuI5RboB4L isjBbYAmJpcoAYFMCT3AwS2zmP2FLA== =woyR -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue May 19 14:20:11 2020 Received: (at 41382) by debbugs.gnu.org; 19 May 2020 18:20:11 +0000 Received: from localhost ([127.0.0.1]:50916 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jb6qY-0001wA-Vn for submit@debbugs.gnu.org; Tue, 19 May 2020 14:20:11 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:49571) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jb6qX-0001vv-O9 for 41382@debbugs.gnu.org; Tue, 19 May 2020 14:20:09 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id A9A555C01AA; Tue, 19 May 2020 14:20:04 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Tue, 19 May 2020 14:20:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=famulari.name; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=mesmtp; bh=Fqyynu7rkZhhrkDiQOJ2D+6D KXYN/+yJ7Vv/3nP0M9A=; b=vmWlUFzEzlLr6UbN+4sXA2aLAZmNoEysG1njeIoj 7QwP43WiAOITxa681wEHvEXOzCoardp9V8abZoB3tKUzMv4Oha7Oe6BvpDRXdYJI rspqRaYd0AxPk6t7Aj4tFMkRWeazEykvr7Yk5RyHVi9OCtCodxXDbv5itAecT1iK hAk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=Fqyynu 7rkZhhrkDiQOJ2D+6DKXYN/+yJ7Vv/3nP0M9A=; b=V4t/dQSu2Gjf5SQBNTvNfZ nZh4bGWnA3pP/ahT66CqlHEjuIHMMXKNbYlK0Oif+geRoUw8G7PlNj5ASzAOiwa+ kayx/2VGCNFv2usm1cmKGzzdsEQeBCC7EtG5kohgQ5KwRv0V1HIAVR789oL0b1jh V/GXOvIjHxGG8tYeK4A7zBRIuWzutVWZgbmdl2DHolNAJRmzYOeA5/IOmyTQi2YG 779pbFqXw6s3DjJVBTMuqzArcG/xHSNk8y1uKXV4OYuSIjPPFkpmQokzx4ya0s6t DbLxyy8TzLQUu3KspB1OoAI4lfXlh++oacxXFeRrqar13r8DRQUh1Mk+iUpn+kJQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedruddtjedguddvvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvuffkfhggtggujgesghdtreertddtvdenucfhrhhomhepnfgvohcu hfgrmhhulhgrrhhiuceolhgvohesfhgrmhhulhgrrhhirdhnrghmvgeqnecuggftrfgrth htvghrnhepudekveegteekleetgfeitdejgfejkeffudethedvhfeukeduleeikeejfeeh ffetnecukfhppeejiedruddvgedrudefkedrieefnecuvehluhhsthgvrhfuihiivgeptd enucfrrghrrghmpehmrghilhhfrhhomheplhgvohesfhgrmhhulhgrrhhirdhnrghmvg X-ME-Proxy: Received: from localhost (c-76-124-138-63.hsd1.pa.comcast.net [76.124.138.63]) by mail.messagingengine.com (Postfix) with ESMTPA id 3F848306643E; Tue, 19 May 2020 14:20:04 -0400 (EDT) Date: Tue, 19 May 2020 14:20:01 -0400 From: Leo Famulari To: Marius Bakke Subject: Re: [bug#41382] [PATCH 0/6] Allow for a cryptographic hash function migration Message-ID: <20200519182001.GA19528@jasmine.lan> References: <20200518213116.23978-1-ludo@gnu.org> <871rnggf4d.fsf@gnu.org> <87wo57reil.fsf@devup.no> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="/04w6evG8XlLl3ft" Content-Disposition: inline In-Reply-To: <87wo57reil.fsf@devup.no> X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 41382 Cc: 41382@debbugs.gnu.org, Ludovic =?iso-8859-1?Q?Court=E8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --/04w6evG8XlLl3ft Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, May 19, 2020 at 08:00:34PM +0200, Marius Bakke wrote: > This is a great initiative, and the patches LGTM. +1 > I think that if we are to move away from SHA256, we should go with > something that is immune to length extension attacks[0] such as BLAKE2/3 > or SHA-3 (Keccak). I think we are so far from needing to move away from SHA256 that we can't know what the best replacement will be when the time comes. It will probably be quite a long time, maybe decades. --/04w6evG8XlLl3ft Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAl7EI04ACgkQJkb6MLrK fwjNyg//RezzHAl3lY9p7ZmqFP0uWeyWQ5rayMHwZPmIsYUcmOXafE3geXV4bzob J1Fcyt+i8eV4EjunlJb5T8GEhHcOg9DOobLqUgWKBSGGjudvfyDSOK7cSQOqJoU8 tZKQ31lMqL4gCT9IOndmQm2yHoqwsVBOHtEEoUxggxxI9zMZwYHPHDmArL8vmbrQ VzwYYGHgqDUsc0N3+ut+nYFpHUpx1J8gZcP4Wp2rVezhMcg1BSVQzaNrlp3eHftt nlHOPequwQE3ub4qEmw1S8LfNWNj0VOrnnR9A4N+7PxUJLXZmxK8eP7pqTE6SO0I xudHCot/tfW1r40/OWvUoGHrFS0oOSoPE+S49eraZJWl8w/c0amshXl9HoHsfGA1 rRd7aF2g6J1KbXR8EVBoXZjgTDf/mD0GaVNqGSuoC09Kt/llXm1ZKAY5phi08+vx +oRC2N+zjxOEimUtX1j8unwHvpxcTABXzt6pCdSxo3xVnxn90k5f2wjOZV64yJvs k07Nfg29oUcCmGtD7nFi/KgO+BaHvxyz/o3O/TchGra+1phgPV20Kvd2YUwGqwDD aQClpBlHzelFRniqbArX4ygoeuzYhoMPNGw92IfhhUKrB//rRQ0T1E+tt/Gy7v0a 0LJhlHNO8MsmuIdqqKurfN5GTZO0tRCte749ae7RUT6FXTmryMQ= =qhqw -----END PGP SIGNATURE----- --/04w6evG8XlLl3ft-- From debbugs-submit-bounces@debbugs.gnu.org Thu May 21 16:46:28 2020 Received: (at 41382) by debbugs.gnu.org; 21 May 2020 20:46:28 +0000 Received: from localhost ([127.0.0.1]:57396 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jbs5E-0006lb-I1 for submit@debbugs.gnu.org; Thu, 21 May 2020 16:46:28 -0400 Received: from eggs.gnu.org ([209.51.188.92]:40282) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jbs5D-0006lO-4M for 41382@debbugs.gnu.org; Thu, 21 May 2020 16:46:27 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51643) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jbs57-0000p9-O9; Thu, 21 May 2020 16:46:21 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=57978 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jbs57-000588-Bp; Thu, 21 May 2020 16:46:21 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Marius Bakke Subject: Re: [bug#41382] [PATCH 0/6] Allow for a cryptographic hash function migration References: <20200518213116.23978-1-ludo@gnu.org> <871rnggf4d.fsf@gnu.org> <87wo57reil.fsf@devup.no> Date: Thu, 21 May 2020 22:46:18 +0200 In-Reply-To: <87wo57reil.fsf@devup.no> (Marius Bakke's message of "Tue, 19 May 2020 20:00:34 +0200") Message-ID: <875zcpc8yt.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41382 Cc: guix-devel@gnu.org, 41382@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Marius Bakke skribis: [...] >> and to automatically =E2=80=9Cconvert=E2=80=9D the =E2=80=98sha256=E2=80= =99 field specification to a >> =E2=80=98content-hash=E2=80=99. Due to the way identifiers are matched,= there are cases >> where we can=E2=80=99t preserve the illusion of compatibility, as can be= seen >> with the patch below. Perhaps that=E2=80=99s acceptable, though. >> >> Thoughts? > > This is a great initiative, and the patches LGTM. Great, thanks for taking a look. > I think that if we are to move away from SHA256, we should go with > something that is immune to length extension attacks[0] such as BLAKE2/3 > or SHA-3 (Keccak). That makes sense to me. I think we have time to think about it. When we choose to switch, we should change all the tools (importers, =E2=80=98guix download=E2=80=99, et= c.) and documentation to default to the new hash so migration can happen consistently. > Although I don't know any Guile implementations of those as of yet. Libgcrypt supports them, so we can definitely use them. I realize we also need to extend nix/libutil/hash.{cc,hh}. > SHA512 does not improve much security-wise IMO, but maybe it's > worthwhile as s stop-gap. Yeah, I=E2=80=99m not sure. We should definitely keep an eye on what other= s are doing and what crypto folks recommend. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu May 21 19:43:42 2020 Received: (at 41382-done) by debbugs.gnu.org; 21 May 2020 23:43:42 +0000 Received: from localhost ([127.0.0.1]:57585 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jbuqk-0002Y3-0Y for submit@debbugs.gnu.org; Thu, 21 May 2020 19:43:42 -0400 Received: from eggs.gnu.org ([209.51.188.92]:57462) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jbuqi-0002Xr-Bo for 41382-done@debbugs.gnu.org; Thu, 21 May 2020 19:43:41 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:53817) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jbuqd-0000A6-2i; Thu, 21 May 2020 19:43:35 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=49008 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jbuqc-000471-MB; Thu, 21 May 2020 19:43:34 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Marius Bakke Subject: Re: [bug#41382] [PATCH 0/6] Allow for a cryptographic hash function migration References: <20200518213116.23978-1-ludo@gnu.org> <871rnggf4d.fsf@gnu.org> <87wo57reil.fsf@devup.no> <875zcpc8yt.fsf@gnu.org> Date: Fri, 22 May 2020 01:43:32 +0200 In-Reply-To: <875zcpc8yt.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Thu, 21 May 2020 22:46:18 +0200") Message-ID: <87wo54c0rf.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41382-done Cc: guix-devel@gnu.org, 41382-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Pushed the whole series: ce0be5675b packages: Introduce and use it in . 56f7ca6e7c packages: Add 'base64' macro. 0e4e9c8e76 guix hash, guix download: Support base64 format. 18ae1ec3ec guix hash, guix download: Add '--hash'. 9418aaa00d tests: Test fixed-output derivations with several hash algorit= hms. 73b27eaa64 tests: Test 'add-to-store' with several hash algorithms. You=E2=80=99ll have to recompile due to the ABI change: make clean-go && make I realized several tests needed to be adjusted for proper syntax-case matching, which I did in ce0be5675b. Ludo=E2=80=99. From unknown Fri Jun 20 07:17:17 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 19 Jun 2020 11:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator