GNU bug report logs - #32196
[PATCH core-updates] packages, scripts, utils: Disable threaded xz compression.

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Marius Bakke <mbakke <at> fastmail.com>
Subject: bug#32196: closed (Re: [bug#32196] [PATCH core-updates] packages,
 scripts, utils: Disable threaded xz compression.)
Date: Sun, 22 Jul 2018 16:07:02 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#32196: [PATCH core-updates] packages, scripts, utils: Disable threaded xz compression.

which was filed against the guix-patches package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 32196 <at> debbugs.gnu.org.

-- 
32196: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=32196
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Marius Bakke <mbakke <at> fastmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 32196-done <at> debbugs.gnu.org
Subject: Re: [bug#32196] [PATCH core-updates] packages, scripts,
 utils: Disable threaded xz compression.
Date: Sun, 22 Jul 2018 18:06:04 +0200

[Message part 3 (text/plain, inline)]

ludo <at> gnu.org (Ludovic Courtès) writes:

> Hi Marius,
>
> Marius Bakke <mbakke <at> fastmail.com> skribis:
>
>> Archives produces by "xz --threads" are not reproducible: they depend on the
>> number of cores used for compressing.  See <https://bugs.gnu.org/31015>.
>>
>> * guix/packages.scm (patch-and-repack): Explicitly use 1 thread for compression.
>> * guix/scripts/pack.scm (%compressors, bootstrap-xz): Likewise.
>> * guix/utils.scm (decompressed-port, compressed-port, compressed-output-port):
>> Likewise.
>
> I hadn’t noticed -T0 had made it to ‘master’.
>
> I’d suggest removing 63102406f22412bb922de5549deb89d3594a38c0 on master
> (no rebuild needed), with a reference to #31015 in the commit log.  And
> then similarly reverting c8a3dea847bb9f87fa1876d0c6c3356d6226f121 on
> ‘core-updates’.
>
> Sounds good?

Done in commits e9be2c5409f37173d70b202aa06752e3814ccdc2 and
3e95125e9bd0676d4a9add9105217ad3eaef3ff0.

> If we’re concerned about speed, perhaps we should switch to lzip, which
> I think has better behavior.

That sounds great.  lzip has other benefits too, such as the ability to
recover from bit flips.  Recommended reading:
<https://www.nongnu.org/lzip/xz_inadequate.html>.

[signature.asc (application/pgp-signature, inline)]

[Message part 5 (message/rfc822, inline)]

From: Marius Bakke <mbakke <at> fastmail.com>
To: guix-patches <at> gnu.org
Subject: [PATCH core-updates] packages, scripts,
 utils: Disable threaded xz compression.
Date: Wed, 18 Jul 2018 14:10:28 +0200

Archives produces by "xz --threads" are not reproducible: they depend on the
number of cores used for compressing.  See <https://bugs.gnu.org/31015>.

* guix/packages.scm (patch-and-repack): Explicitly use 1 thread for compression.
* guix/scripts/pack.scm (%compressors, bootstrap-xz): Likewise.
* guix/utils.scm (decompressed-port, compressed-port, compressed-output-port):
Likewise.
---
 guix/packages.scm     | 2 +-
 guix/scripts/pack.scm | 4 ++--
 guix/utils.scm        | 6 +++---
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/guix/packages.scm b/guix/packages.scm
index c762fa7c3..ef88e0c59 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -633,7 +633,7 @@ specifies modules in scope when evaluating SNIPPET."
                      ;; threaded compression (introduced in
                      ;; 5.2.0), but it ignores the extra flag.
                      (string-append "--use-compress-program="
-                                    #+xz "/bin/xz --threads=0")
+                                    #+xz "/bin/xz --threads=1")
                      ;; avoid non-determinism in the archive
                      "--mtime=@0"
                      "--owner=root:0"
diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 7f087a3a3..e0d6f1dee 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -69,7 +69,7 @@
         (compressor "lzip"  ".lz"
                     #~(#+(file-append lzip "/bin/lzip") "-9"))
         (compressor "xz"    ".xz"
-                    #~(#+(file-append xz "/bin/xz") "-e -T0"))
+                    #~(#+(file-append xz "/bin/xz") "-e -T1"))
         (compressor "bzip2" ".bz2"
                     #~(#+(file-append bzip2 "/bin/bzip2") "-9"))
         (compressor "none" "" #f)))
@@ -77,7 +77,7 @@
 ;; This one is only for use in this module, so don't put it in %compressors.
 (define bootstrap-xz
   (compressor "bootstrap-xz" ".xz"
-              #~(#+(file-append %bootstrap-coreutils&co "/bin/xz") "-e -T0")))
+              #~(#+(file-append %bootstrap-coreutils&co "/bin/xz") "-e -T1")))
 
 (define (lookup-compressor name)
   "Return the compressor object called NAME.  Error out if it could not be
diff --git a/guix/utils.scm b/guix/utils.scm
index a5de9605e..84176a22e 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -175,7 +175,7 @@ a symbol such as 'xz."
   (match compression
     ((or #f 'none) (values input '()))
     ('bzip2        (filtered-port `(,%bzip2 "-dc") input))
-    ('xz           (filtered-port `(,%xz "-dc" "-T0") input))
+    ('xz           (filtered-port `(,%xz "-dc" "-T1") input))
     ('gzip         (filtered-port `(,%gzip "-dc") input))
     (else          (error "unsupported compression scheme" compression))))
 
@@ -185,7 +185,7 @@ a symbol such as 'xz."
   (match compression
     ((or #f 'none) (values input '()))
     ('bzip2        (filtered-port `(,%bzip2 "-c") input))
-    ('xz           (filtered-port `(,%xz "-c" "-T0") input))
+    ('xz           (filtered-port `(,%xz "-c" "-T1") input))
     ('gzip         (filtered-port `(,%gzip "-c") input))
     (else          (error "unsupported compression scheme" compression))))
 
@@ -242,7 +242,7 @@ program--e.g., '(\"--fast\")."
   (match compression
     ((or #f 'none) (values output '()))
     ('bzip2        (filtered-output-port `(,%bzip2 "-c" ,@options) output))
-    ('xz           (filtered-output-port `(,%xz "-c" "-T0" ,@options) output))
+    ('xz           (filtered-output-port `(,%xz "-c" "-T1" ,@options) output))
     ('gzip         (filtered-output-port `(,%gzip "-c" ,@options) output))
     (else          (error "unsupported compression scheme" compression))))
 
-- 
2.18.0

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.