Subject: bug#44574: guix weather not checking all substitute urls

From: zimoun <[email-masked]>
Date: Mon, 16 Nov 2020 18:12:54 +0100

Hi Efraim,

On Wed, 11 Nov 2020 at 13:30, Efraim Flashner <[email-masked]> wrote:
> I would have assumed that it would check for all of the substitute urls
> I listed and not just the first one. Most of the packages listed below
> I've test built on my pine64 so they shouldn't appear in the list
> anymore.

I think #44688 [1] fixes it.  Please let me know.

1: <http://issues.guix.gnu.org/issue/44688>

Examples:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix weather --substitute-urls="https://bayfront.guix.gnu.org https://ci.guix.gnu.org"
computing 15,391 package derivations for x86_64-linux...
looking for 16,707 store items on https://bayfront.guix.gnu.org...
updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0%
https://bayfront.guix.gnu.org
  55.6% substitutes available (9,289 out of 16,707)
  at least 51,869.2 MiB of nars (compressed)
  82,986.2 MiB on disk (uncompressed)
  0.001 seconds per request (24.6 seconds in total)
  679.0 requests per second

  0.3% (20 out of 7,418) of the missing items are queued
  32 queued builds
      x86_64-linux: 32 (100.0%)
  build rate: 13.83 builds per hour
      x86_64-linux: 13.83 builds per hour
looking for 16,707 store items on https://ci.guix.gnu.org...
updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
https://ci.guix.gnu.org
  88.9% substitutes available (14,860 out of 16,707)
  at least 78,929.4 MiB of nars (compressed)
  130,797.2 MiB on disk (uncompressed)
  0.002 seconds per request (26.9 seconds in total)
  620.2 requests per second

  0.5% (10 out of 1,847) of the missing items are queued
  at least 1,000 queued builds
      x86_64-linux: 493 (49.3%)
      aarch64-linux: 140 (14.0%)
      i686-linux: 346 (34.6%)
      armhf-linux: 21 (2.1%)
  build rate: 38.98 builds per hour
      x86_64-linux: 13.29 builds per hour
      i686-linux: 12.32 builds per hour
      aarch64-linux: 6.78 builds per hour
      armhf-linux: 6.59 builds per hour
--8<---------------cut here---------------end--------------->8---
      
or

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix weather --substitute-urls="https://guix.tobias.gr https://ci.guix.gnu.org" -c 100
computing 15,391 package derivations for x86_64-linux...
looking for 16,707 store items on https://guix.tobias.gr...
updating substitutes from 'https://guix.tobias.gr'... 100.0%
https://guix.tobias.gr
  35.0% substitutes available (5,855 out of 16,707)
  unknown substitute sizes
  52,230.7 MiB on disk (uncompressed)
  0.017 seconds per request (291.8 seconds in total)
  57.3 requests per second
  (continuous integration information unavailable)
updating substitutes from 'https://guix.tobias.gr'... 100.0%
6777 packages are missing from 'https://guix.tobias.gr' for 'x86_64-linux', among which:
  16564	bootar@1a	/gnu/store/x2iy3ka299sk1x97an783jl4pfbn60pj-bootar-1a 
  16561	[email-masked]	/gnu/store/a4m8c80mrys8w3vlm6vdnq05c7a1warl-bootstrap-mes-rewired-0.19 
  16495	ld-wrapper-boot3@0	/gnu/store/v3wwjj9i9zqglq4g2yczf6cxsh028qfb-ld-wrapper-boot3-0 
  9743	[email-masked]	/gnu/store/rxli27pbpr9x9c8mk9cbp6masnwaszrs-bison-3.5.3 
  1831	[email-masked]	/gnu/store/fjwjb14b5irngvv9r06dmkhhds566vw2-xkbcomp-intermediate-1.4.3 
   788	bootar@1a	/gnu/store/x2iy3ka299sk1x97an783jl4pfbn60pj-bootar-1a 
   785	[email-masked]	/gnu/store/a4m8c80mrys8w3vlm6vdnq05c7a1warl-bootstrap-mes-rewired-0.19 
   719	ld-wrapper-boot3@0	/gnu/store/v3wwjj9i9zqglq4g2yczf6cxsh028qfb-ld-wrapper-boot3-0 
   595	bootar@1a	/gnu/store/x2iy3ka299sk1x97an783jl4pfbn60pj-bootar-1a 
   592	[email-masked]	/gnu/store/a4m8c80mrys8w3vlm6vdnq05c7a1warl-bootstrap-mes-rewired-0.19 
   526	ld-wrapper-boot3@0	/gnu/store/v3wwjj9i9zqglq4g2yczf6cxsh028qfb-ld-wrapper-boot3-0 
   339	[email-masked]	/gnu/store/rxli27pbpr9x9c8mk9cbp6masnwaszrs-bison-3.5.3 
   305	[email-masked]	/gnu/store/asw6676iy7i0a8azpra2870w5hkq7bvv-go-1.4-bootstrap-20171003-doc /gnu/store/j20pik1y8psi78v7zb5ksqybn198inx7-go-1.4-bootstrap-20171003 /gnu/store/csfqsyyysbg9afnck405wmkhrlrblmza-go-1.4-bootstrap-20171003-tests 
   211	[email-masked]	/gnu/store/fjwjb14b5irngvv9r06dmkhhds566vw2-xkbcomp-intermediate-1.4.3 
   121	[email-masked]	/gnu/store/rxli27pbpr9x9c8mk9cbp6masnwaszrs-bison-3.5.3 
   113	[email-masked]	/gnu/store/k217j4c4ra2hfdldva075dmvq89ww1dn-r-codetools-0.2-16 
looking for 16,707 store items on https://ci.guix.gnu.org...
https://ci.guix.gnu.org
  88.9% substitutes available (14,860 out of 16,707)
  at least 78,929.4 MiB of nars (compressed)
  130,797.2 MiB on disk (uncompressed)
  0.001 seconds per request (10.1 seconds in total)
  1,650.0 requests per second

  0.5% (10 out of 1,847) of the missing items are queued
  at least 1,000 queued builds
      x86_64-linux: 493 (49.3%)
      aarch64-linux: 140 (14.0%)
      i686-linux: 346 (34.6%)
      armhf-linux: 21 (2.1%)
  build rate: 38.70 builds per hour
      x86_64-linux: 13.20 builds per hour
      i686-linux: 12.23 builds per hour
      aarch64-linux: 6.73 builds per hour
      armhf-linux: 6.54 builds per hour
updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
1251 packages are missing from 'https://ci.guix.gnu.org' for 'x86_64-linux', among which:
   112	[email-masked]	/gnu/store/mq1xagq5sn9mfhpfg7j2cn1dn4vsds72-java-snappy-1.1.7.5 
--8<---------------cut here---------------end--------------->8---

compared to the current behaviour:

--8<---------------cut here---------------start------------->8---
$ guix weather --substitute-urls="https://bayfront.guix.gnu.org https://ci.guix.gnu.org"
computing 15,287 package derivations for x86_64-linux...
looking for 16,584 store items on https://bayfront.guix.gnu.org...
updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0%
https://bayfront.guix.gnu.org
  54.4% substitutes available (9,026 out of 16,584)
  at least 49,365.8 MiB of nars (compressed)
  81,792.9 MiB on disk (uncompressed)
  0.003 seconds per request (41.6 seconds in total)
  398.0 requests per second

  0.1% (10 out of 7,558) of the missing items are queued
  32 queued builds
      x86_64-linux: 32 (100.0%)
  build rate: 13.84 builds per hour
      x86_64-linux: 13.84 builds per hour
--8<---------------cut here---------------end--------------->8---

(Note that Bayfront is working hard. ;-))

All the best,
simon