Package: guix;
Reported by: raid5atemyhomework <raid5atemyhomework <at> protonmail.com>
Date: Fri, 5 Mar 2021 11:23:01 UTC
Severity: normal
View this message in rfc822 format
From: raid5atemyhomework <raid5atemyhomework <at> protonmail.com> To: 46942 <at> debbugs.gnu.org Subject: bug#46942: ci.guix.gnu.org is slow from my system Date: Fri, 05 Mar 2021 11:22:17 +0000
Downloading substitutes from ci.guix.gnu.org is slow from my two Guix-using computers. One is a pure Guix System install without any channels, the other one is a foreign Guix install with the-channel-that-cannot-be-named. ``` downloading from https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0 ... wine64-6.0 54.4MiB 10KiB/s 01:55 [ ] 2.1% ``` This can be very slow, including as slow as 4KiB/s at times. This is fairly awful since sometimes I get this result: ``` downloading from https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0 ... wine64-6.0 54.4MiB 49KiB/s 06:01 [##### ] 31.9%guix substitute: error: TLS error in procedure 'read_from_session_record_port': Error decoding the received TLS packet. substitution of /gnu/store/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0 failed guix package: error: some substitutes for the outputs of derivation `/gnu/store/wr9kf2bgcsvwxcmhnl9lf047nr8xcklc-wine64-6.0.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source ``` Because of the failed incomplete download, I have to ***restart the download again from 0%***, which is awful, awful UX. It would be really nice if: 1. The guix download process would make an effort to ***retry*** this a few times. 2. Continue the previous download instead of restarting from 0%. The problem is not on my ISP, or at least not solely on my ISP. Doing a `wget` from `github.com`: ``` $ wget https://github.com/zfsonlinux/zfs/releases/download/zfs-2.0.3/zfs-2.0.3.tar.gz --2021-03-05 17:39:46-- https://github.com/zfsonlinux/zfs/releases/download/zfs-2.0.3/zfs-2.0.3.tar.gz Resolving github.com (github.com)... 13.229.188.59 Connecting to github.com (github.com)|13.229.188.59|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://github.com/openzfs/zfs/releases/download/zfs-2.0.3/zfs-2.0.3.tar.gz [following] --2021-03-05 17:39:47-- https://github.com/openzfs/zfs/releases/download/zfs-2.0.3/zfs-2.0.3.tar.gz Reusing existing connection to github.com:443. HTTP request sent, awaiting response... 302 Found Location: https://github-releases.githubusercontent.com/437011/71526a80-6ba3-11eb-893f-4ceb55b479d6?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210305%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210305T093947Z&X-Amz-Expires=300&X-Amz-Signature=9454446348219aa0731af009a5232bf5772e3d45a3a34052fa61f64f04c3c979&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=437011&response-content-disposition=attachment%3B%20filename%3Dzfs-2.0.3.tar.gz&response-content-type=application%2Foctet-stream [following] --2021-03-05 17:39:47-- https://github-releases.githubusercontent.com/437011/71526a80-6ba3-11eb-893f-4ceb55b479d6?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210305%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210305T093947Z&X-Amz-Expires=300&X-Amz-Signature=9454446348219aa0731af009a5232bf5772e3d45a3a34052fa61f64f04c3c979&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=437011&response-content-disposition=attachment%3B%20filename%3Dzfs-2.0.3.tar.gz&response-content-type=application%2Foctet-stream Resolving github-releases.githubusercontent.com (github-releases.githubusercontent.com)... 185.199.108.154, 185.199.109.154, 185.199.110.154, ... Connecting to github-releases.githubusercontent.com (github-releases.githubusercontent.com)|185.199.108.154|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 13114404 (13M) [application/octet-stream] Saving to: ‘zfs-2.0.3.tar.gz’ zfs-2.0.3.tar.gz 100%[=====================================================================================================>] 12.51M 9.34MB/s in 1.3s 2021-03-05 17:39:49 (9.34 MB/s) - ‘zfs-2.0.3.tar.gz’ saved [13114404/13114404] ``` As can be seen above, I get 9.34MB/s elsewhere, which is better than the >60Mbit/s promised by my ISP. It's possible that the problem is between my ISP and ci.guix.gnu.org. If I `wget` directly: ``` $ wget https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0 --2021-03-05 18:56:21-- https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0 Resolving ci.guix.gnu.org (ci.guix.gnu.org)... 141.80.181.40 Connecting to ci.guix.gnu.org (ci.guix.gnu.org)|141.80.181.40|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 57048561 (54M) [application/octet-stream] Saving to: ‘1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0’ 1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0 0%[ ] 167.79K 13.4KB/s eta 69m 20s^C ``` HOWEVER, if I `torify wget`: ``` $ torify wget https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0 --2021-03-05 18:57:00-- https://ci.guix.gnu.org/nar/lzip/1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0 Resolving ci.guix.gnu.org (ci.guix.gnu.org)... 141.80.181.40 Connecting to ci.guix.gnu.org (ci.guix.gnu.org)|141.80.181.40|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 57048561 (54M) [application/octet-stream] Saving to: ‘1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0.1’ 1bdldr80p39g1mjnh76xw6hmwqrrb8lz-wine64-6.0.1 13%[============> ] 7.13M 683KB/s eta 72s ^C ``` Which is a big WHAT THE FUCK? Tor should not be 45x faster (683KB/s) than directly from my ISP (13KB/s). I am not using any special workarounds for Tor, like meek or obfs4, which makes me doubt that my ISP is attempting to censor --- if my ISP is the one doing the censoring, it would probably target Tor first before ci.guix.gnu.org, but I can access Tor just fine without special workarounds needed for very oppressive countries. I've tried this a few times. Consistently, using `torify wget` is much faster than `wget` alone, I don't have to bang my head in boredom with `torify wget`. This is not my expectation given what I understand of the Tor network and the public network. Is `ci.guix.gnu.org` doing some kind of per-IP or per-ISP or other throttling? This looks very suspicious given the massive speed difference when using Tor and not using Tor. Doing `torify guix package -m manifest.scm` does not seem to perform the download inside Tor, which makes sense since it's probably `guix-daemon` that is doing the download. So my questions are: * WHY IS DIRECT DOWNLOAD FROM MY ISP SLOWER THAN TOR? What can I do to check if it's my ISP that is attempting to censor ci.guix.gnu.org? * Is there a way to make `guix-daemon` use a Tor proxy? I have two systems using Guix, one is a Guix System, the other is using a foreign distro, and I'd like to adjust both to use Tor instead since it's faster. * If not, is there a way to tell `guix-daemon` that I have these `nar` files and it can put them in its store somehow instead of me waiting for it to ***SLOOOOOOOOWLY*** download it? * Are there any mirrors I can use for substitutes instead? How do I change my systems (both the Guix System and the foreign distro) to use the mirrors? Is there some configuration option to do this? * How hard would it be to make the substitute download process at least make an effort to attempt to continue a failed download instead of failing immediately and forcing a restart from 0%? `texlive` is something like 400Mb, it takes hours at 10KiB/s, I once had it fail at >90% which was very painful because it had to restart from 0%, I had to do `guix package -m manifest.scm` in a loop over and over again until it finished download. It would probably cut down on people hammering on the server as well. Thanks raid5atemyhomework
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.