GNU bug report logs - #33643
[PATCH] gnu-build-system: Enable xz to decompress in parallel.

Previous Next

Full log

Message #32 received at 33643 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 33643 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: [bug#33643] [PATCH] gnu-build-system: Enable xz to decompress in
 parallel.
Date: Wed, 13 May 2020 22:07:21 +0300

[Message part 1 (text/plain, inline)]

On Wed, May 13, 2020 at 07:20:08PM +0100, Christopher Baines wrote:
> 
> Christopher Baines <mail <at> cbaines.net> writes:
> 
> > It can take a little while to decompress some packages with large xz
> > compressed source tar files. xz includes support for parallelism, so enable
> > this using the parallel job count for the overall derivation.
> >
> > * guix/build/gnu-build-system.scm (unpack): Set XZ_OPT to pass the -T option
> > to xz to enable it to work in parallel if appropriate.
> > ---
> >  guix/build/gnu-build-system.scm | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
> > index e5f3197b0..9d11e5b1e 100644
> > --- a/guix/build/gnu-build-system.scm
> > +++ b/guix/build/gnu-build-system.scm
> > @@ -147,7 +147,7 @@ chance to be set."
> >                locale (strerror (system-error-errno args)))
> >        #t)))
> >
> > -(define* (unpack #:key source #:allow-other-keys)
> > +(define* (unpack #:key source parallel-build? #:allow-other-keys)
> >    "Unpack SOURCE in the working directory, and change directory within the
> >  source.  When SOURCE is a directory, copy it in a sub-directory of the current
> >  working directory."
> > @@ -161,6 +161,10 @@ working directory."
> >          (copy-recursively source "."
> >                            #:keep-mtime? #t))
> >        (begin
> > +        (when parallel-build?
> > +          (setenv "XZ_OPT"
> > +                  (format #f "-T~d" (parallel-job-count))))
> > +
> >          (if (string-suffix? ".zip" source)
> >              (invoke "unzip" source)
> >              (invoke "tar" "xvf" source))
> 
> It's been a long long while, but now that core-updates has recently been
> merged, I'd like to try and take a look at this again.
> 
> I think the consensus was that this will only help for xz compressed
> files where they have been compressed in parallel. I think it's still
> worth doing though, as some of the big xz files that need decompressing
> have been compressed in parallel, and this will speed up the builds when
> multiple cores are available.
> 
> Thanks,
> 
> Chris

I thought the last time we looked into this we figured out that there
was a mistake in release notes or something and that parallel
decompression isn't actually supported.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.