GNU bug report logs - #26316
Enable threaded compression of source tarballs

Previous Next

Full log

View this message in rfc822 format

From: Kei Kebreau <kei <at> openmailbox.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 26316 <at> debbugs.gnu.org
Subject: bug#26316: Enable threaded compression of source tarballs
Date: Fri, 31 Mar 2017 14:15:36 -0400

[Message part 1 (text/plain, inline)]

Leo Famulari <leo <at> famulari.name> writes:

> This patch enables multi-threaded xz compression in the repacking stage
> of building source tarballs.
>
> With it applied, I get a ~2.2x speedup building the source of
> linux-libre with 4 threads, compared to 1 thread.
>
> Specifically, with the upstream linux-libre tarball in the page cache,
> `guix build -S linux-libre` took ~366 seconds using a single thread for
> xz, and ~164 seconds using 4 threads.
>
> From 78547a09e50440c649c3d28d7691f32fdd47cc25 Mon Sep 17 00:00:00 2001
> From: Leo Famulari <leo <at> famulari.name>
> Date: Thu, 30 Mar 2017 03:10:48 -0400
> Subject: [PATCH] packages: Enable threaded compression of source tarballs.
>
> This provides a ~2x speedup when using 4 threads.
>
> * guix/packages.scm (patch-and-repack)[build]: Invoke xz with
> '--threads=0' when re-packing tarballs.
> ---
>  guix/packages.scm | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/guix/packages.scm b/guix/packages.scm
> index 4bc4b017f..30be5bf38 100644
> --- a/guix/packages.scm
> +++ b/guix/packages.scm
> @@ -573,7 +573,12 @@ specifies modules in scope when evaluating SNIPPET."
>                                                      #:fail-on-error? #t)))))
>                          (zero? (apply system*
>                                        (string-append #+tar "/bin/tar")
> -                                      "cvfa" #$output
> +                                      "cvf" #$output
> +                                      ;; The bootstrap xz does not support
> +                                      ;; threaded compression (introduced in
> +                                      ;; 5.2.0), but it ignores the extra flag.
> +                                      (string-append "--use-compress-program="
> +                                                     #+xz "/bin/xz --threads=0")
>                                        ;; avoid non-determinism in the archive
>                                        "--mtime=@0"
>                                        "--owner=root:0"

LGTM.

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.