GNU bug report logs - #51472
substitute servers should be preferred according to their coverage rate

Previous Next

Full log

Message #8 received at 51472 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 51472 <at> debbugs.gnu.org
Subject: Re: bug#51472: substitute servers should be preferred according to
 their coverage rate
Date: Sun, 07 Nov 2021 16:11:04 +0100

Hi,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:

> When using substitute servers discovery, I've noticed that if one of the
> substitute servers doesn't have any substitutes available, it'll keep
> getting tried instead of others, leading to a slide-show of substitutes
> updates such as:
>
> normalized load on machine '127.0.0.1' is 0.04
> building /gnu/store/ajd0hx104702jpz2ycdwgrnyrv8jsp6d-xorg-server-21.1.0.tar.xz.drv...
> process 9195 acquired build slot '/var/guix/offload/127.0.0.1:6666/1'
> normalized load on machine '127.0.0.1' is 0.04
> building /gnu/store/49rqi3wpvdm5pv6in9pamzdvg0wscrl8-xorgproto-2021.5.drv...
> substitute: updating substitutes from 'http://192.168.10.102:80'...   0.0%
> substitute: updating substitutes from 'http://192.168.10.102:80'...   0.0%
> substitute: updating substitutes from 'http://192.168.10.102:80'...   0.0%
> substitute: updating substitutes from 'http://192.168.10.102:80'...   0.0%
> substitute: updating substitutes from 'http://192.168.10.102:80'...   0.0%

We’d need to check why this particular server is checked repeatedly.
The fact that it displays “0.0%” doesn’t mean that the server lacks
substitutes, but that it does not reply to ‘GET /xyz.narinfo’ requests,
for example because it’s off-line (see
<https://issues.guix.gnu.org/48808>.)

> We should implement some scheme to prefer querying high-substitute
> servers first, instead of wasting time querying servers always failed
> queries; this would greatly improve performance when using substitute
> discovery for example combined with low coverage.

There are several problems with that.  First one is that you can’t tell
what substitute coverage is until you’ve actually made those GET
requests.  Second one is that substitute coverage varies and it’s not an
absolute measure; for example, if a server provides substitutes for only
0.1% of all the packages, but that’s precisely the 0.1% you care about,
it’s more valuable than the one that has 99% of the packages but lacks
those you want.

There are other issues such as the fact that current semantics is to
respect the order of substitute URLs, which is presumably chosen by the
user according to their own criteria: download speed, bandwidth usage,
etc.

I hope this makes sense!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.