GNU bug report logs - #39728
[PATCH] Allow parallel downloads and builds

Previous Next

Full log

View this message in rfc822 format

From: Julien Lepiller <julien <at> lepiller.eu>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 39728 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>, zimoun <zimon.toutoune <at> gmail.com>
Subject: [bug#39728] [PATCH v2] Allow parallel downloads and builds
Date: Thu, 25 Nov 2021 14:01:45 +0100

Le Thu, 25 Nov 2021 13:53:25 +0100,
Ludovic Courtès <ludo <at> gnu.org> a écrit :

> Hello,
> 
> Julien Lepiller <julien <at> lepiller.eu> skribis:
> 
> > After so long, I managed to find the time to go over the comments
> > and improve my patches. I tested the new daemon for a bit, and it's
> > working as expected so far :D  
> 
> On a recent daemon, have you seen cases where having multiple
> downloads in parallel speeds things up?
> 
> The analysis in
> <https://guix.gnu.org/en/blog/2021/getting-bytes-to-disk-more-quickly/>
> suggests that at the time you first submitted this patch, substitution
> speed (which is different from raw download speed) was often
> CPU-bound. This is no longer the case, meaning that downloads should
> now be network-bound or almost.
> 
> Thanks,
> Ludo’.

I would still say yes, because the output from berlin is often much
less than my throughput. With multiple downloads in parallel it at
least feels quicker, probably because I can download at full speed.

In any case, I see often a build start while downloads are in progress,
so I think it's still a win if you can get a few derivations built
while waiting for a big download to finish at the same time :)

At some point we might want to prioritize builds/downloads that help
unlock as much builds as possible early, so we don't have builds
waiting for downloads.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.