From unknown Tue Aug 19 07:26:35 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#32196 <32196@debbugs.gnu.org> To: bug#32196 <32196@debbugs.gnu.org> Subject: Status: [PATCH core-updates] packages, scripts, utils: Disable threaded xz compression. Reply-To: bug#32196 <32196@debbugs.gnu.org> Date: Tue, 19 Aug 2025 14:26:35 +0000 retitle 32196 [PATCH core-updates] packages, scripts, utils: Disable thread= ed xz compression. reassign 32196 guix-patches submitter 32196 Marius Bakke severity 32196 normal tag 32196 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 18 08:10:47 2018 Received: (at submit) by debbugs.gnu.org; 18 Jul 2018 12:10:47 +0000 Received: from localhost ([127.0.0.1]:46202 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fflI6-00032f-Tw for submit@debbugs.gnu.org; Wed, 18 Jul 2018 08:10:47 -0400 Received: from eggs.gnu.org ([208.118.235.92]:60476) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fflI4-00032R-VU for submit@debbugs.gnu.org; Wed, 18 Jul 2018 08:10:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fflHy-0006Al-Kh for submit@debbugs.gnu.org; Wed, 18 Jul 2018 08:10:39 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_20,FREEMAIL_FROM, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:53847) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fflHy-0006Af-BC for submit@debbugs.gnu.org; Wed, 18 Jul 2018 08:10:38 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51444) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fflHw-0003h4-Sx for guix-patches@gnu.org; Wed, 18 Jul 2018 08:10:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fflHr-00069C-Su for guix-patches@gnu.org; Wed, 18 Jul 2018 08:10:36 -0400 Received: from out2-smtp.messagingengine.com ([66.111.4.26]:40005) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fflHr-00068V-Az for guix-patches@gnu.org; Wed, 18 Jul 2018 08:10:31 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id BB4E721CE1 for ; Wed, 18 Jul 2018 08:10:30 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Wed, 18 Jul 2018 08:10:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= date:from:message-id:subject:to:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=EUDSWE8zuiKT2f/aItORHrjfuLog54HToH+IlXTwu Xs=; b=bmhZP+N21nBpjOy+7xIs77BA30kg88e2+tcfxt83+EAkVNg0eH0IIC1L+ TxfDZgfdL2PowuGDjSYkxkPUb6rkAAif5YzJ432xRLBqKtcZ0DFfNVIQ5tNK3+bA 6FxVvsJe2qU2W7/06HHVl9G02HATU0cuXSuKbuQlV2tG9tGrA7OLSoksQAFo2nx3 HNJAseI4Cbr6fy4BPMAvCOpxZYs+YNhHZfX+xh8HG5/6jzzQ0vN8K7d2hrS10GyC H7RIux+tnaVil8ihtY8McbpC/sl2hxmy05nzZEbKjfIepRaWoMHFLKEvJxHDG+5K J0DY2TEBp2EemXJtT8tnXzJch5JyA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=date:from:message-id:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=EUDSWE8zuiKT2f/aI tORHrjfuLog54HToH+IlXTwuXs=; b=a2j2fF81sCzT6RoCVLrca2SO608MDKC9W oErwlNBDGf3KZKPyi1fOWJSV3rhdUXHxIMGJSAaq0gd6XRiH8+xSrNkfEbEIn+U0 hd7TyKx+r+BgLdpG1OumZZ+oPpp0bg6Zewq1E6MQYegrw1aTaOl13ynJNO8FfIFu tt3uY+btodS3OayijD6WNJrZ2GnqK3ZcvSlHPP4FKsJ0lkg5byevFNzzksXj/eg4 r4H1L16SosFtBsWMjo8vrv20seR/gJdQns5HXO3f4TmrZyowRFFIMRG+s1Y0C8Go E/NUron7vvajgsv7ic942ooQnO68kbKyBp1XuaFBODI2bf/nX+MhQ== X-ME-Proxy: X-ME-Sender: Received: from localhost (95.92-221-151.customer.lyse.net [92.221.151.95]) by mail.messagingengine.com (Postfix) with ESMTPA id 1A45B10261 for ; Wed, 18 Jul 2018 08:10:29 -0400 (EDT) From: Marius Bakke To: guix-patches@gnu.org Subject: [PATCH core-updates] packages, scripts, utils: Disable threaded xz compression. Date: Wed, 18 Jul 2018 14:10:28 +0200 Message-Id: <20180718121028.27939-1-mbakke@fastmail.com> X-Mailer: git-send-email 2.18.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.4 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.4 (-----) Archives produces by "xz --threads" are not reproducible: they depend on the number of cores used for compressing. See . * guix/packages.scm (patch-and-repack): Explicitly use 1 thread for compression. * guix/scripts/pack.scm (%compressors, bootstrap-xz): Likewise. * guix/utils.scm (decompressed-port, compressed-port, compressed-output-port): Likewise. --- guix/packages.scm | 2 +- guix/scripts/pack.scm | 4 ++-- guix/utils.scm | 6 +++--- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/guix/packages.scm b/guix/packages.scm index c762fa7c3..ef88e0c59 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -633,7 +633,7 @@ specifies modules in scope when evaluating SNIPPET." ;; threaded compression (introduced in ;; 5.2.0), but it ignores the extra flag. (string-append "--use-compress-program=" - #+xz "/bin/xz --threads=0") + #+xz "/bin/xz --threads=1") ;; avoid non-determinism in the archive "--mtime=@0" "--owner=root:0" diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 7f087a3a3..e0d6f1dee 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -69,7 +69,7 @@ (compressor "lzip" ".lz" #~(#+(file-append lzip "/bin/lzip") "-9")) (compressor "xz" ".xz" - #~(#+(file-append xz "/bin/xz") "-e -T0")) + #~(#+(file-append xz "/bin/xz") "-e -T1")) (compressor "bzip2" ".bz2" #~(#+(file-append bzip2 "/bin/bzip2") "-9")) (compressor "none" "" #f))) @@ -77,7 +77,7 @@ ;; This one is only for use in this module, so don't put it in %compressors. (define bootstrap-xz (compressor "bootstrap-xz" ".xz" - #~(#+(file-append %bootstrap-coreutils&co "/bin/xz") "-e -T0"))) + #~(#+(file-append %bootstrap-coreutils&co "/bin/xz") "-e -T1"))) (define (lookup-compressor name) "Return the compressor object called NAME. Error out if it could not be diff --git a/guix/utils.scm b/guix/utils.scm index a5de9605e..84176a22e 100644 --- a/guix/utils.scm +++ b/guix/utils.scm @@ -175,7 +175,7 @@ a symbol such as 'xz." (match compression ((or #f 'none) (values input '())) ('bzip2 (filtered-port `(,%bzip2 "-dc") input)) - ('xz (filtered-port `(,%xz "-dc" "-T0") input)) + ('xz (filtered-port `(,%xz "-dc" "-T1") input)) ('gzip (filtered-port `(,%gzip "-dc") input)) (else (error "unsupported compression scheme" compression)))) @@ -185,7 +185,7 @@ a symbol such as 'xz." (match compression ((or #f 'none) (values input '())) ('bzip2 (filtered-port `(,%bzip2 "-c") input)) - ('xz (filtered-port `(,%xz "-c" "-T0") input)) + ('xz (filtered-port `(,%xz "-c" "-T1") input)) ('gzip (filtered-port `(,%gzip "-c") input)) (else (error "unsupported compression scheme" compression)))) @@ -242,7 +242,7 @@ program--e.g., '(\"--fast\")." (match compression ((or #f 'none) (values output '())) ('bzip2 (filtered-output-port `(,%bzip2 "-c" ,@options) output)) - ('xz (filtered-output-port `(,%xz "-c" "-T0" ,@options) output)) + ('xz (filtered-output-port `(,%xz "-c" "-T1" ,@options) output)) ('gzip (filtered-output-port `(,%gzip "-c" ,@options) output)) (else (error "unsupported compression scheme" compression)))) -- 2.18.0 From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 19 05:34:10 2018 Received: (at 32196) by debbugs.gnu.org; 19 Jul 2018 09:34:10 +0000 Received: from localhost ([127.0.0.1]:47510 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fg5K6-0001QG-Cd for submit@debbugs.gnu.org; Thu, 19 Jul 2018 05:34:10 -0400 Received: from eggs.gnu.org ([208.118.235.92]:35412) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fg5K2-0001Pi-Uc for 32196@debbugs.gnu.org; Thu, 19 Jul 2018 05:34:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fg5Ju-0000pc-JL for 32196@debbugs.gnu.org; Thu, 19 Jul 2018 05:34:01 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:49704) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fg5Ju-0000pY-FJ; Thu, 19 Jul 2018 05:33:58 -0400 Received: from [193.50.110.227] (port=58548 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1fg5Jt-0002LD-Uu; Thu, 19 Jul 2018 05:33:58 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Marius Bakke Subject: Re: [bug#32196] [PATCH core-updates] packages, scripts, utils: Disable threaded xz compression. References: <20180718121028.27939-1-mbakke@fastmail.com> Date: Thu, 19 Jul 2018 11:33:56 +0200 In-Reply-To: <20180718121028.27939-1-mbakke@fastmail.com> (Marius Bakke's message of "Wed, 18 Jul 2018 14:10:28 +0200") Message-ID: <87sh4fwovv.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 32196 Cc: 32196@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Hi Marius, Marius Bakke skribis: > Archives produces by "xz --threads" are not reproducible: they depend on = the > number of cores used for compressing. See . > > * guix/packages.scm (patch-and-repack): Explicitly use 1 thread for compr= ession. > * guix/scripts/pack.scm (%compressors, bootstrap-xz): Likewise. > * guix/utils.scm (decompressed-port, compressed-port, compressed-output-p= ort): > Likewise. I hadn=E2=80=99t noticed -T0 had made it to =E2=80=98master=E2=80=99. I=E2=80=99d suggest removing 63102406f22412bb922de5549deb89d3594a38c0 on ma= ster (no rebuild needed), with a reference to #31015 in the commit log. And then similarly reverting c8a3dea847bb9f87fa1876d0c6c3356d6226f121 on =E2=80=98core-updates=E2=80=99. Sounds good? If we=E2=80=99re concerned about speed, perhaps we should switch to lzip, w= hich I think has better behavior. Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 22 12:06:08 2018 Received: (at 32196-done) by debbugs.gnu.org; 22 Jul 2018 16:06:08 +0000 Received: from localhost ([127.0.0.1]:52628 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fhGs3-0007DA-N4 for submit@debbugs.gnu.org; Sun, 22 Jul 2018 12:06:07 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:35653) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fhGs2-0007D3-GQ for 32196-done@debbugs.gnu.org; Sun, 22 Jul 2018 12:06:06 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 6240421B4C; Sun, 22 Jul 2018 12:06:06 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Sun, 22 Jul 2018 12:06:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= cc:content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=TB/Kr8IRmEXuTDNVx6CE/csA9nWX8dXlmUQ+FcAkviM=; b=rWvvst+f SRDY+RLolnCPW0t4kSnK1MZ1SuG94Sm5/0OLQIll6mbLWNYCZP5V8hO75lxckgnC t83Duo01Ht+JtBPB8q5XoIT8Gr0sxiBHj7T7tAIRGOyCexJR/DMMz0Ln6NBMWrnk hMFZAHiGOPe/ZyVW/ijHUqsiJO+ZJhmzLVqk0asnx4TrDAk5gqtqSAQRzUkfu7ur IxW461fiptapq7fuBHwQ2od4onwHOv934W/E9Ynxcy9LuPvfSegH/0tNgfLIAzJl ThuhDUQ4/+hQPqhAy6fRASmf4kzrlRH1AcFLvN1VKYjT5ki+yCXpcYTgga+ukonp f0izYqOZ586U2Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; bh=TB/Kr8IRmEXuTDNVx6CE/csA9nWX8 dXlmUQ+FcAkviM=; b=OsWk/Zcaj1rVOeLQ0zBSv8U32hnpZggxKJGLmuHrXVerF Qe9X72BOJj8y3cz/iI5rR0s6+m+IiPVWYQvbukRBfHf8UEBuxKCF0ksFUyC4Sz9Y g/Y0cyYWncKwKUixn9PY0cfkx54ug8xpdt7iTklQR1fuDB5uAdHbPa9cwjppDAe/ 6ai21RczBrTNsjGltXcsEF2YbQFCrQ6LloGPD9mGoljddcBy3BwYg6yejGEtC6q+ 9jbqYKI1qu8P5T2T2st34jSqdP743pSyGB+eaUr8zovObKYdq05JUrmbNbGtKHJe CtD6dJijvm4fUKXYPr8HMTIAfQmMwpRxIe6+Ni+hw== X-ME-Proxy: X-ME-Sender: Received: from localhost (95.92-221-151.customer.lyse.net [92.221.151.95]) by mail.messagingengine.com (Postfix) with ESMTPA id C44D31026A; Sun, 22 Jul 2018 12:06:05 -0400 (EDT) From: Marius Bakke To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#32196] [PATCH core-updates] packages, scripts, utils: Disable threaded xz compression. In-Reply-To: <87sh4fwovv.fsf@gnu.org> References: <20180718121028.27939-1-mbakke@fastmail.com> <87sh4fwovv.fsf@gnu.org> User-Agent: Notmuch/0.27 (https://notmuchmail.org) Emacs/26.1 (x86_64-pc-linux-gnu) Date: Sun, 22 Jul 2018 18:06:04 +0200 Message-ID: <87va9745n7.fsf@fastmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 32196-done Cc: 32196-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Hi Marius, > > Marius Bakke skribis: > >> Archives produces by "xz --threads" are not reproducible: they depend on= the >> number of cores used for compressing. See . >> >> * guix/packages.scm (patch-and-repack): Explicitly use 1 thread for comp= ression. >> * guix/scripts/pack.scm (%compressors, bootstrap-xz): Likewise. >> * guix/utils.scm (decompressed-port, compressed-port, compressed-output-= port): >> Likewise. > > I hadn=E2=80=99t noticed -T0 had made it to =E2=80=98master=E2=80=99. > > I=E2=80=99d suggest removing 63102406f22412bb922de5549deb89d3594a38c0 on = master > (no rebuild needed), with a reference to #31015 in the commit log. And > then similarly reverting c8a3dea847bb9f87fa1876d0c6c3356d6226f121 on > =E2=80=98core-updates=E2=80=99. > > Sounds good? Done in commits e9be2c5409f37173d70b202aa06752e3814ccdc2 and 3e95125e9bd0676d4a9add9105217ad3eaef3ff0. > If we=E2=80=99re concerned about speed, perhaps we should switch to lzip,= which > I think has better behavior. That sounds great. lzip has other benefits too, such as the ability to recover from bit flips. Recommended reading: . --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAltUq2wACgkQoqBt8qM6 VPqK5Af+J1Ske1e5ZNs26sOTFnhH+P5m7PiOLyJ5QRBGb5KczqPL1C9u/xsqu0YG vZ86mrBpH7z/8Th2cigKfhq6bHAWZKDGiIfZke4BepkfivTuHkpoQVJlLCeZNnY9 zCbYQqYJs/axILuBSlihJRmnxhgwSefDhkE7ieaKb9EeQxYfIT7GO6bR6Ayv1vKb wQ/FLgPNY4pTCYlWnxtMiKG1SUo3EjapcBrLmkJIdhCROVarATbtk0Dxdt3mADn8 RlCAoB542Qig+PdtULZmJKnrvOxaGST7F6P7V7+2xlGZw4GPSTH/yyCLBxv66ubo k5+Dvfa5nQI2szkspMCiFzpS34Amwg== =eqKB -----END PGP SIGNATURE----- --=-=-=-- From unknown Tue Aug 19 07:26:35 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 20 Aug 2018 11:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator