From unknown Fri Jun 13 10:35:07 2025 X-Loop: help-debbugs@gnu.org Subject: bug#49810: source tarballs potentially built for each derivation Resent-From: Vagrant Cascadian Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 01 Aug 2021 20:47:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 49810 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 49810@debbugs.gnu.org Cc: guix-devel X-Debbugs-Original-To: bug-guix@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.162785078620077 (code B ref -1); Sun, 01 Aug 2021 20:47:02 +0000 Received: (at submit) by debbugs.gnu.org; 1 Aug 2021 20:46:26 +0000 Received: from localhost ([127.0.0.1]:36999 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mAILp-0005Dl-T7 for submit@debbugs.gnu.org; Sun, 01 Aug 2021 16:46:26 -0400 Received: from lists.gnu.org ([209.51.188.17]:59624) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mAILn-0005Da-69 for submit@debbugs.gnu.org; Sun, 01 Aug 2021 16:46:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42776) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mAILl-0004Z2-Cv; Sun, 01 Aug 2021 16:46:21 -0400 Received: from cascadia.aikidev.net ([173.255.214.101]:50378) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mAILc-0001Ka-QE; Sun, 01 Aug 2021 16:46:20 -0400 Received: from localhost (unknown [IPv6:2600:3c01:e000:21:21:21:0:100b]) (Authenticated sender: vagrant@cascadia.debian.net) by cascadia.aikidev.net (Postfix) with ESMTPSA id 986B71A9FD; Sun, 1 Aug 2021 13:46:05 -0700 (PDT) From: Vagrant Cascadian In-Reply-To: <87k0ngg6zn.fsf@yucca> References: <87lf8xvrgj.fsf@yucca> <87k0ngg6zn.fsf@yucca> Date: Sun, 01 Aug 2021 13:45:41 -0700 Message-ID: <87o8ag6gi2.fsf@yucca> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: none client-ip=173.255.214.101; envelope-from=vagrant@debian.org; helo=cascadia.aikidev.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Turning this conversation into a bug, original thread around here: https://lists.gnu.org/archive/html/guix-devel/2021-05/threads.html#00427 On 2021-05-29, Vagrant Cascadian wrote: > On 2021-05-01, Leo Famulari wrote: >> On Sat, May 01, 2021 at 06:45:32PM -0700, Vagrant Cascadian wrote: >>> Pragmatically speaking, on slower platforms this is a huge resource >>> overhead. So much so that ci.guix.gnu.org *usually* times out when >>> generating the linux-libre aarch64 tarballs: >>>=20 >>> https://ci.guix.gnu.org/search?query=3Dsystem%3Aaarch64-linux+linux-l= ibre-arm64-generic >> >> Thanks for letting me know. I didn't know this was happening. >> >> The immediate solution is for me to make sure the tarballs have built >> before committing the updates. I already do this for x86_64 and I can >> start doing it for aarch64 too. > > This has definitely helped sometimes, thanks! I even saw a substitute of > linux-libre for aarch64 earlier today! :) > > Still, I'm noticing another problem with the way way the tarballs are > generated on ci.guix.gnu.org ... > > When it generates a tarball, all the various packages independently try > to recreate the source tarball; so you have at least fours jobs > ("linux-libre", "linux-libre-arm64-generic", "linux-libre-headers", > "linux-libre-bpf") all concurrently trying to build the very same > very-slow-to-build tarball on ci.guix.gnu.org. Sometimes one of them > might succeed, but the others may not, and even though one of them > succeeded, none of the failing ones retry... > > Not knowing exactly how ci.guix.gnu.org works, would it make sense to > create a tarball package instead of the ... computed origin(?) tarball, > so it could be better represented in the package dependency graph, and > the various linux-libre-* packages can wait till it is available rather > than all trying to recreate the same thing? > > That still requires the tarball generation to not time out in the first > place, but maybe it would help with the resource limitations a bit to > only build the source tarball once per architecture? This seems to still be an issue for ci.guix.gnu.org, but the linux-libre* substitutes for aarch64 seem to be available on https://bordeaux.guix.gnu.org ... $ guix weather linux-libre linux-libre-arm64-generic computing 2 package derivations for aarch64-linux... looking for 2 store items on https://ci.guix.gnu.org... https://ci.guix.gnu.org 0.0% substitutes available (0 out of 2) unknown substitute sizes 0.0 MiB on disk (uncompressed) 0.740 seconds per request (0.7 seconds in total) 1.4 requests per second 0.0% (0 out of 2) of the missing items are queued 1 queued builds aarch64-linux: 1 (100.0%) build rate: .00 builds per hour x86_64-linux: 0.00 builds per hour aarch64-linux: 0.00 builds per hour i686-linux: 0.00 builds per hour looking for 2 store items on https://bordeaux.guix.gnu.org... https://bordeaux.guix.gnu.org 100.0% substitutes available (2 out of 2) 83.9 MiB of nars (compressed) 202.2 MiB on disk (uncompressed) (continuous integration information unavailable) live well, vagrant --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYQcH9gAKCRDcUY/If5cW qp1LAP9eMIp0x8ygd5rht//VE9o8QBYm4cXec2LBxwOevDVwKwEAn10lZqhwXTPH 1XPirQBwEbLSI5V0P2x8DWY+PwgGIAU= =uCAx -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Jun 13 10:35:07 2025 X-Loop: help-debbugs@gnu.org Subject: bug#49810: source tarballs potentially built for each derivation Resent-From: Vagrant Cascadian Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 01 Aug 2021 21:06:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49810 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 49810@debbugs.gnu.org Cc: guix-devel X-Debbugs-Original-To: bug-guix@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.162785190222216 (code B ref -1); Sun, 01 Aug 2021 21:06:01 +0000 Received: (at submit) by debbugs.gnu.org; 1 Aug 2021 21:05:02 +0000 Received: from localhost ([127.0.0.1]:37042 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mAIdq-0005mF-BS for submit@debbugs.gnu.org; Sun, 01 Aug 2021 17:05:02 -0400 Received: from lists.gnu.org ([209.51.188.17]:39260) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mAIdo-0005lo-UF for submit@debbugs.gnu.org; Sun, 01 Aug 2021 17:05:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44570) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mAIdo-0002LN-KY; Sun, 01 Aug 2021 17:05:00 -0400 Received: from cascadia.aikidev.net ([173.255.214.101]:50422) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mAIdn-0005Nt-8E; Sun, 01 Aug 2021 17:05:00 -0400 Received: from localhost (unknown [IPv6:2600:3c01:e000:21:21:21:0:100b]) (Authenticated sender: vagrant@cascadia.debian.net) by cascadia.aikidev.net (Postfix) with ESMTPSA id 814391A9FD; Sun, 1 Aug 2021 14:04:57 -0700 (PDT) From: Vagrant Cascadian In-Reply-To: <87o8ag6gi2.fsf@yucca> References: <87lf8xvrgj.fsf@yucca> <87k0ngg6zn.fsf@yucca> <87o8ag6gi2.fsf@yucca> Date: Sun, 01 Aug 2021 14:04:52 -0700 Message-ID: <87im0o6fm3.fsf@yucca> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: none client-ip=173.255.214.101; envelope-from=vagrant@debian.org; helo=cascadia.aikidev.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain On 2021-08-01, Vagrant Cascadian wrote: > Turning this conversation into a bug, original thread around here: > > https://lists.gnu.org/archive/html/guix-devel/2021-05/threads.html#00427 For reference: bug#49810: source tarballs potentially built for each derivation live well, vagrant --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYQcMdQAKCRDcUY/If5cW qqOTAP0VHHcUNYSak2xx0iyNqub2Bj4Cu1AFN9M2rxHIQFokxwEAv64LslHkI6YT fOPWh6ADXXU+0KahK6NPk4WL1fXU/A0= =M4gC -----END PGP SIGNATURE----- --=-=-=--