Subject: bug#46942: closed (Close)

From: [email-masked] (GNU bug Tracking System)
Date: Mon, 07 Jul 2025 11:07:02 +0000

Your bug report

#46942: ci.guix.gnu.org is slow from my system

which was filed against the guix package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
46942: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46942
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Close
From: Andreas Enge <[email-masked]>
To: [email-masked]
Date: Mon, 7 Jul 2025 13:05:53 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Closing as probably network related, and it is unclear what we could do.

Andreas

Subject: ci.guix.gnu.org is slow from my system
From: raid5atemyhomework <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Fri, 05 Mar 2021 11:22:17 +0000
List-Id: <debbugs-submit.debbugs.gnu.org>

Downloading substitutes from ci.guix.gnu.org is slow from my two Guix-using computers.

One is a pure Guix System install without any channels, the other one is a foreign Guix install with the-channel-that-cannot-be-named.

```
downloading from https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0 ...
 wine64-6.0  54.4MiB                                                                                                                                 10KiB/s 01:55 [                  ]   2.1%
```

This can be very slow, including as slow as 4KiB/s at times. This is fairly awful since sometimes I get this result:

```
downloading from https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0 ...
 wine64-6.0  54.4MiB                                                                                                                                 49KiB/s 06:01 [#####             ]  31.9%guix substitute: error: TLS error in procedure 'read_from_session_record_port': Error decoding the received TLS packet.
substitution of /gnu/store/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0 failed
guix package: error: some substitutes for the outputs of derivation `/gnu/store/wr9kf2bgcsvwxcmhnl9lf047nr8xcklc-wine64-6.0.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source
```

Because of the failed incomplete download, I have to ***restart the download again from 0%***, which is awful, awful UX. It would be really nice if:

1. The guix download process would make an effort to ***retry*** this a few times.
2. Continue the previous download instead of restarting from 0%.

The problem is not on my ISP, or at least not solely on my ISP. Doing a `wget` from `github.com`:

```
$ wget https://github.com/zfsonlinux/zfs/releases/download/zfs-2.0.3/zfs-2.0.3.tar.gz
--2021-03-05 17:39:46--  https://github.com/zfsonlinux/zfs/releases/download/zfs-2.0.3/zfs-2.0.3.tar.gz
Resolving github.com (github.com)... 13.229.188.59
Connecting to github.com (github.com)|13.229.188.59|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://github.com/openzfs/zfs/releases/download/zfs-2.0.3/zfs-2.0.3.tar.gz [following]
--2021-03-05 17:39:47--  https://github.com/openzfs/zfs/releases/download/zfs-2.0.3/zfs-2.0.3.tar.gz
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://github-releases.githubusercontent.com/437011/71526a80-6ba3-11eb-893f-4ceb55b479d6?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210305%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210305T093947Z&X-Amz-Expires=300&X-Amz-Signature=9454446348219aa0731af009a5232bf5772e3d45a3a34052fa61f64f04c3c979&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=437011&response-content-disposition=attachment%3B%20filename%3Dzfs-2.0.3.tar.gz&response-content-type=application%2Foctet-stream [following]
--2021-03-05 17:39:47--  https://github-releases.githubusercontent.com/437011/71526a80-6ba3-11eb-893f-4ceb55b479d6?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210305%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210305T093947Z&X-Amz-Expires=300&X-Amz-Signature=9454446348219aa0731af009a5232bf5772e3d45a3a34052fa61f64f04c3c979&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=437011&response-content-disposition=attachment%3B%20filename%3Dzfs-2.0.3.tar.gz&response-content-type=application%2Foctet-stream
Resolving github-releases.githubusercontent.com (github-releases.githubusercontent.com)... 185.199.108.154, 185.199.109.154, 185.199.110.154, ...
Connecting to github-releases.githubusercontent.com (github-releases.githubusercontent.com)|185.199.108.154|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 13114404 (13M) [application/octet-stream]
Saving to: ‘zfs-2.0.3.tar.gz’

zfs-2.0.3.tar.gz                                100%[=====================================================================================================>]  12.51M  9.34MB/s    in 1.3s

2021-03-05 17:39:49 (9.34 MB/s) - ‘zfs-2.0.3.tar.gz’ saved [13114404/13114404]

```

As can be seen above, I get 9.34MB/s elsewhere, which is better than the >60Mbit/s promised by my ISP.

It's possible that the problem is between my ISP and ci.guix.gnu.org. If I `wget` directly:

```
$ wget https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0
--2021-03-05 18:56:21--  https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0
Resolving ci.guix.gnu.org (ci.guix.gnu.org)... 141.80.181.40
Connecting to ci.guix.gnu.org (ci.guix.gnu.org)|141.80.181.40|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 57048561 (54M) [application/octet-stream]
Saving to: ‘1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0’

1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0       0%[                                                                                                      ] 167.79K  13.4KB/s    eta 69m 20s^C
```

HOWEVER, if I `torify wget`:

```
$ torify wget https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0
--2021-03-05 18:57:00--  https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0
Resolving ci.guix.gnu.org (ci.guix.gnu.org)... 141.80.181.40
Connecting to ci.guix.gnu.org (ci.guix.gnu.org)|141.80.181.40|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 57048561 (54M) [application/octet-stream]
Saving to: ‘1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0.1’

1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0.1    13%[============>                                                                                         ]   7.13M   683KB/s    eta 72s    ^C
```

Which is a big WHAT THE FUCK? Tor should not be 45x faster (683KB/s) than directly from my ISP (13KB/s). I am not using any special workarounds for Tor, like meek or obfs4, which makes me doubt that my ISP is attempting to censor --- if my ISP is the one doing the censoring, it would probably target Tor first before ci.guix.gnu.org, but I can access Tor just fine without special workarounds needed for very oppressive countries.

I've tried this a few times.  Consistently, using `torify wget` is much faster than `wget` alone, I don't have to bang my head in boredom with `torify wget`.  This is not my expectation given what I understand of the Tor network and the public network.

Is `ci.guix.gnu.org` doing some kind of per-IP or per-ISP or other throttling?  This looks very suspicious given the massive speed difference when using Tor and not using Tor.

Doing `torify guix package -m manifest.scm` does not seem to perform the download inside Tor, which makes sense since it's probably `guix-daemon` that is doing the download.

So my questions are:

* WHY IS DIRECT DOWNLOAD FROM MY ISP SLOWER THAN TOR? What can I do to check if it's my ISP that is attempting to censor ci.guix.gnu.org?
* Is there a way to make `guix-daemon` use a Tor proxy?  I have two systems using Guix, one is a Guix System, the other is using a foreign distro, and I'd like to adjust both to use Tor instead since it's faster.
  * If not, is there a way to tell `guix-daemon` that I have these `nar` files and it can put them in its store somehow instead of me waiting for it to ***SLOOOOOOOOWLY*** download it?
* Are there any mirrors I can use for substitutes instead? How do I change my systems (both the Guix System and the foreign distro) to use the mirrors? Is there some configuration option to do this?
* How hard would it be to make the substitute download process at least make an effort to attempt to continue a failed download instead of failing immediately and forcing a restart from 0%? `texlive` is something like 400Mb, it takes hours at 10KiB/s, I once had it fail at >90% which was very painful because it had to restart from 0%, I had to do `guix package -m manifest.scm` in a loop over and over again until it finished download. It would probably cut down on people hammering on the server as well.

Thanks
raid5atemyhomework