From unknown Mon Jun 16 17:18:35 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#65720 <65720@debbugs.gnu.org> To: bug#65720 <65720@debbugs.gnu.org> Subject: Status: Guile-Git-managed checkouts grow way too much Reply-To: bug#65720 <65720@debbugs.gnu.org> Date: Tue, 17 Jun 2025 00:18:35 +0000 retitle 65720 Guile-Git-managed checkouts grow way too much reassign 65720 guix submitter 65720 Ludovic Court=C3=A8s severity 65720 important thanks From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 03 16:44:52 2023 Received: (at submit) by debbugs.gnu.org; 3 Sep 2023 20:44:52 +0000 Received: from localhost ([127.0.0.1]:47021 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qctxj-000252-Nn for submit@debbugs.gnu.org; Sun, 03 Sep 2023 16:44:52 -0400 Received: from lists.gnu.org ([2001:470:142::17]:45444) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qctxf-00024l-3P for submit@debbugs.gnu.org; Sun, 03 Sep 2023 16:44:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qctxa-0005j4-46 for bug-guix@gnu.org; Sun, 03 Sep 2023 16:44:42 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qctxZ-0000zE-RW for bug-guix@gnu.org; Sun, 03 Sep 2023 16:44:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to: references; bh=Jkb+5XJZ/cpgVmsVrZkaJedRMIuJpZjLE4M6ySVFbqo=; b=NRPelYOW3/og8H 1G8z8V2RBe7B5QmopsEMT7R1gOPOPj2MRata4axsYyyj5rqcowJAhWvjCxU3v2ZMNyu7L4BktujqV dPngiUPsF43aAXM9QJUmLGhJN6SBDLV+RGitx3il/dkCCwfuW/byz0UWWISq9sKfbUJEghY8au/Os wbLjTLITTlND7RrqeX3urkNJoOxD0TTqQyDOEOzUfWjNpR9uiNutDUzjeFdE2xYv75imRhDSxsOAx JnsM8RLB9m9aKk6qqm6phnuSKK26JqXvwuepCJOh+8Xq/+3cfMQ0Y0w7W6/pug5ki685Sz97D2Vqv bfUnShKQ+fGxZ9U+1LGA==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: bug-guix@gnu.org Subject: Guile-Git-managed checkouts grow way too much X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Septidi 17 Fructidor an 231 de la =?utf-8?Q?R=C3=A9v?= =?utf-8?Q?olution=2C?= jour de la =?utf-8?Q?Card=C3=A8re?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 03 Sep 2023 22:44:39 +0200 Message-ID: <87bkejc7go.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello! As reported by Tobias on IRC (in the context of =E2=80=98hpcguix-web=E2=80= =99), checkouts managed by Guile-Git appear to grow beyond reason. As an example, here=E2=80=99s the same =E2=80=98.git=E2=80=99 managed with Guile-= Git and with Git: --8<---------------cut here---------------start------------->8--- $ du -hs ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27sh= j7apsnalwq 6.7G /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywm= isyr27shj7apsnalwq $ du -hs .git 517M .git --8<---------------cut here---------------end--------------->8--- It would seem that libgit2 doesn=E2=80=99t do the equivalent of =E2=80=98gi= t gc=E2=80=99. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 04 17:13:41 2023 Received: (at control) by debbugs.gnu.org; 4 Sep 2023 21:13:41 +0000 Received: from localhost ([127.0.0.1]:52818 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdGtB-0000MR-9d for submit@debbugs.gnu.org; Mon, 04 Sep 2023 17:13:41 -0400 Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104]:10633) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdGt9-0000Lz-1Q for control@debbugs.gnu.org; Mon, 04 Sep 2023 17:13:39 -0400 Authentication-Results: mail3-relais-sop.national.inria.fr; dkim=none (message not signed) header.i=none; spf=SoftFail smtp.mailfrom=ludo@gnu.org; dmarc=fail (p=none dis=none) d=gnu.org X-IronPort-AV: E=Sophos;i="6.02,227,1688421600"; d="scan'208";a="65072453" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail3-relais-sop.national.inria.fr with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Sep 2023 23:13:33 +0200 Date: Mon, 04 Sep 2023 23:13:31 +0200 Message-Id: <87zg21od50.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #65720 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) severity 65720 important quit From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 04 17:48:03 2023 Received: (at 65720) by debbugs.gnu.org; 4 Sep 2023 21:48:03 +0000 Received: from localhost ([127.0.0.1]:52859 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdHQQ-0003u2-Hy for submit@debbugs.gnu.org; Mon, 04 Sep 2023 17:48:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43088) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdHQM-0003sn-7m for 65720@debbugs.gnu.org; Mon, 04 Sep 2023 17:48:01 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdHQG-0006vJ-Lm for 65720@debbugs.gnu.org; Mon, 04 Sep 2023 17:47:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=BurYVm1rImaTmYgUS5+9Jz3dIfEvZ64VE5iDtpY6cmI=; b=Ec4/3vhf5zNdNW4lKvpy 4XoBmWZ7JEUZt0CvLm9yNWuCgOJxEBWGa3ratQojSKu6/QiCo0xWe2muZVgsAQCCTWfyVNtLqAHKF DAOJ/s5K4mWL5ZASq59Fa8Lq+8O8w72B7rdh4yMk1RbmjXQm80F3a/25EHfP/buVA5QMtN8s6oRye U3YnB3ZZ7e+NwO+ZS62u/3SJCaOV9xUvkoVHCoLIODb+GfJn+2nIktYkugnmVUBXREel7MpSCI4fq Sk/ncFUDJgI6LMCvtoV2iMcZzu0OJHGXg0By+5B2qk9criXQf+mLk1DFLvHKLk2NyuLBbcs59Q1yq kO4HjFO8jDdi6w==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 65720@debbugs.gnu.org Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much References: <87bkejc7go.fsf@inria.fr> Date: Mon, 04 Sep 2023 23:47:49 +0200 In-Reply-To: <87bkejc7go.fsf@inria.fr> ("Ludovic =?utf-8?Q?Court=C3=A8s=22?= =?utf-8?Q?'s?= message of "Sun, 03 Sep 2023 22:44:39 +0200") Message-ID: <87fs3tobju.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65720 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s skribis: > As reported by Tobias on IRC (in the context of =E2=80=98hpcguix-web=E2= =80=99), > checkouts managed by Guile-Git appear to grow beyond reason. As an > example, here=E2=80=99s the same =E2=80=98.git=E2=80=99 managed with Guil= e-Git and with Git: > > $ du -hs ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27= shj7apsnalwq > 6.7G /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlay= wmisyr27shj7apsnalwq > $ du -hs .git > 517M .git Unsurprisingly, GC makes a big difference: --8<---------------cut here---------------start------------->8--- $ cp -r ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj= 7apsnalwq /tmp/checkout $ (cd /tmp/checkout/; git gc) Enumerating objects: 717785, done. Counting objects: 100% (717785/717785), done. Delta compression using up to 4 threads Compressing objects: 100% (154644/154644), done. Writing objects: 100% (717785/717785), done. Total 717785 (delta 569440), reused 710535 (delta 562274), pack-reused 0 Enumerating cruft objects: 103412, done. Traversing cruft objects: 81753, done. Counting objects: 100% (64171/64171), done. Delta compression using up to 4 threads Compressing objects: 100% (17379/17379), done. Writing objects: 100% (64171/64171), done. Total 64171 (delta 52330), reused 58296 (delta 46792), pack-reused 0 Expanding reachable commits in commit graph: 133730, done. $ du -hs /tmp/checkout 539M /tmp/checkout --8<---------------cut here---------------end--------------->8--- > It would seem that libgit2 doesn=E2=80=99t do the equivalent of =E2=80=98= git gc=E2=80=99. Confirmed: . My inclination for the short term would be to work around this limitation by (1) finding a heuristic to determine is a checkout has likely accumulated too much cruft, and (2) considering such checkouts as expired (thereby forcing a re-clone) or running =E2=80=98git gc=E2=80=99 on= them if =E2=80=98git=E2=80=99 is available. I can=E2=80=99t think of a good heuristic for (1). Birth time could be one= , but we=E2=80=99d need statx(2): --8<---------------cut here---------------start------------->8--- $ stat ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7= apsnalwq | tail -4 Access: 2023-09-04 23:13:54.668279105 +0200 Modify: 2023-09-04 11:34:41.665385000 +0200 Change: 2023-09-04 11:34:41.661629102 +0200 Birth: 2021-08-09 10:48:17.748722151 +0200 --8<---------------cut here---------------end--------------->8--- Lacking statx(2), we can approximate creation time by looking at =E2=80=98.git/config=E2=80=99: --8<---------------cut here---------------start------------->8--- $ stat ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7= apsnalwq/.git/config | tail -3 Modify: 2021-08-09 10:50:28.031760953 +0200 Change: 2021-08-09 10:50:28.031760953 +0200 Birth: 2021-08-09 10:50:28.031760953 +0200 --8<---------------cut here---------------end--------------->8--- This strategy can be implemented like this: --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/guix/git.scm b/guix/git.scm index ebe2600209..ed3fa56bc8 100644 --- a/guix/git.scm +++ b/guix/git.scm @@ -405,7 +405,16 @@ (define cached-checkout-expiration ;; Use the mtime rather than the atime to cope with file systems mounted ;; with 'noatime'. - (file-expiration-time (* 90 24 3600) stat:mtime)) + (let ((ttl (* 90 24 3600)) + (max-checkout-retention (* 9 30 24 3600))) + (lambda (file) + (match (false-if-exception (lstat file)) + (#f 0) ;FILE may have been deleted in the meantime + (st (min (pk 'ttl (+ (stat:mtime st) ttl)) + (pk 'maxttl (match (false-if-exception + (lstat (in-vicinity file ".git/config"))) + (#f +inf.0) + (st (+ (stat:mtime st) max-checkout-retention)))))))))) (define %checkout-cache-cleanup-period ;; Period for the removal of expired cached checkouts. --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Namely, a cached checkout as considered as =E2=80=9Cexpired=E2=80=9D after = 9 months. In my case, it gives this: --8<---------------cut here---------------start------------->8--- scheme@(guix git)> (cached-checkout-expiration "/home/ludo/.cache/guix/chec= kouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/") ;;; (ttl 1701596081) ;;; (maxttl 1651827028) $6 =3D 1651827028 --8<---------------cut here---------------end--------------->8--- Of course having to re-clone entire repositories every 9 months is ridiculous, but storing gigabytes of packs is worse IMO (I=E2=80=99m specifically thinking about the Guix repo, which every users copies via =E2=80=98guix pull=E2=80=99). Thoughts? Thanks, Ludo=E2=80=99. --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 05 04:18:55 2023 Received: (at 65720) by debbugs.gnu.org; 5 Sep 2023 08:18:55 +0000 Received: from localhost ([127.0.0.1]:53458 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdRGw-0003gJ-Os for submit@debbugs.gnu.org; Tue, 05 Sep 2023 04:18:55 -0400 Received: from jpoiret.xyz ([206.189.101.64]:55990) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdRGt-0003g9-0b for 65720@debbugs.gnu.org; Tue, 05 Sep 2023 04:18:53 -0400 Received: from authenticated-user (jpoiret.xyz [206.189.101.64]) by jpoiret.xyz (Postfix) with ESMTPA id 968F91851DF; Tue, 5 Sep 2023 08:18:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jpoiret.xyz; s=dkim; t=1693901929; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=8pM8T1310gSR8edOn+Ojn9vP6FDTUMbm5DyEBYbX+RI=; b=E5fqAWKR36KGu3A3dWgbPYvmxOS6zdgYXQaFn0sY6toc3M6Vl8n7iH8o9OO9Y2qm/iwym1 zgSUgNHbcungycJPm0MLBLOVu6Wkk6wyduVNr3ikIRY6sp/YSZrp2Bgd0Gj+HKWFvb0t7O qN5EBIvXJ7Tod6fj+RSHbv01NsGsQhzxEB5RL8Pf34mzEvDFgqYlRcdG5LhBzq3ddd0U3s FmV8aXD/mYuE3lOpePWx8AOFY9f9prlilxl81LlFBR9of/nFj6XW1rWXohZRIr3G3dclES TRqSO8aixv/w4wYwV44Sass8VOvHX5CE6ZrzHNCbtcjkEDiaWtpzUcZrh1e34g== From: Josselin Poiret To: Ludovic =?utf-8?Q?Court=C3=A8s?= , 65720@debbugs.gnu.org Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much In-Reply-To: <87fs3tobju.fsf@gnu.org> References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> Date: Tue, 05 Sep 2023 10:18:30 +0200 Message-ID: <87tts9uj6x.fsf@jpoiret.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spamd-Bar: -- Authentication-Results: jpoiret.xyz; auth=pass smtp.auth=jpoiret@jpoiret.xyz smtp.mailfrom=dev@jpoiret.xyz X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65720 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo, Ludovic Court=C3=A8s writes: > My inclination for the short term would be to work around this > limitation by (1) finding a heuristic to determine is a checkout has > likely accumulated too much cruft, and (2) considering such checkouts as > expired (thereby forcing a re-clone) or running =E2=80=98git gc=E2=80=99 = on them if > =E2=80=98git=E2=80=99 is available. I think using the git binary instead of libgit2 as a workaround is a good idea. We can consider building it directly as well, so that people who don't have it in their profiles can still benefit from it. We could even consider using git commands in most places and using libgit2 only where we really need the tight coupling. IIUC, libgit2 is eternally trying to catch up to git and often performs in a counter-intuitive way (I expect the various bugs with stale deleted files in checkouts to be caused by this). Maybe it could also let us use bare repository and directly extract the refs we want without having to mess with checkouts? Best, =2D-=20 Josselin Poiret --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQHEBAEBCgAuFiEEOSSM2EHGPMM23K8vUF5AuRYXGooFAmT25FYQHGRldkBqcG9p cmV0Lnh5egAKCRBQXkC5Fhcain93DACHuEyLuP52K5rHucB5+rmiiaHAqwh6U7Us GdD98bt8ggLcGkuJviQJKAL7sWrbLZLZGoFvGOVSIFU71zixL0aDy0vLHdLrr9kw nhlp9FBrsTE1WJ87n6cqN7QEGAKdecX8QSHzAPSgOiPniby8DDML/EZ5qkJA8HeA x8atrhamPs/j9lUiWMR94O9eTAo0iOrZ3V+o6Phc7711vRWXiSOneIkcrCcylLfs gtCjJUaVp2AwZXNvvCj8Lf17wWcEacsdvi4e6hTzhnT75xiDcCCc5O682F+lLPy2 XzaDc6GAql8y2tt+/zuNUa2S+anBXFGaqqz5Dxzmm3Vi/AEPA24dbCUyjmw+gX+z yzCbSSdt2wWKT9U3Us6jQWbtzztwRjHEqNLFS7NqZvCRZA2UiDF9XxZSLlB6jZB8 01biNYDABcz1SDgEFj007l2iBoxtTWwDeOt1bTdrYP3pmTZpojf0mCox4RXkviHB z6d2UwRQq8C9aGdHVCM0n6FSzs6a7l4= =6KMj -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 05 04:23:13 2023 Received: (at 65720) by debbugs.gnu.org; 5 Sep 2023 08:23:13 +0000 Received: from localhost ([127.0.0.1]:53475 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdRL7-0003no-5m for submit@debbugs.gnu.org; Tue, 05 Sep 2023 04:23:13 -0400 Received: from mout01.posteo.de ([185.67.36.65]:59791) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdRL1-0003nE-Tl for 65720@debbugs.gnu.org; Tue, 05 Sep 2023 04:23:11 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id AAA06240027 for <65720@debbugs.gnu.org>; Tue, 5 Sep 2023 10:23:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1693902181; bh=6gQz5IO3eRHSw9H7uOYRWnTisVSy8sLhtyfs7TlisDI=; h=Content-Transfer-Encoding:From:Mime-Version:Subject:Date: Message-Id:Cc:To:From; b=eWuu7e2M/zsNmRcz/5KpaLuRDYhWhTYqUi8CBZxys5Tcgnli7MfG7rrZtpBisXTto i3Tz1PMoBRndFdC2nY9WFOK1L5gmUYO0XTiFWJ9WQn8U3EfBLaJLyWWFidRgy7AhRN WuFH/4OMfyDPjMP9zy4TfkfCeY7kUKB2RjcK3Lt/XEz7tbu77nK7vcXicYNIwsskkI RAt4O90Y8Rs0RHCV1Zis9sbo8LNuwI0ziEW4mHvwPqzbT/fclTAIsuFO40CqKBv6Da sAWLcraM5RSveomfX61b+XQr6UvWDLAgLL7/8a+3+JOP9Tn45DfgjrfVtosOxHwarl ABJ1I71WFDTEg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Rfz5x0bcyz6trs; Tue, 5 Sep 2023 10:23:00 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Jelle Licht Mime-Version: 1.0 (1.0) Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much Date: Tue, 5 Sep 2023 08:22:48 +0000 Message-Id: References: <87fs3tobju.fsf@gnu.org> In-Reply-To: <87fs3tobju.fsf@gnu.org> To: =?utf-8?Q?Ludovic_Court=C3=A8s?= X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 Cc: 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Ludo, >=20 > On 4 Sep 2023, at 23:49, Ludovic Court=C3=A8s wrote: >=20 > Of course having to re-clone entire repositories every 9 months is > ridiculous, but storing gigabytes of packs is worse IMO (I=E2=80=99m > specifically thinking about the Guix repo, which every users copies via > =E2=80=98guix pull=E2=80=99). Please ignore if it doesn=E2=80=99t make sense, or would not make a practica= l difference for the current issue, but wouldn=E2=80=99t a local clone do th= e trick here? As in, clone from the =E2=80=98clogged=E2=80=99 local repo, mo= ve over fresh clone to old location. Kr, Jelle= From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 05 10:12:03 2023 Received: (at 65720) by debbugs.gnu.org; 5 Sep 2023 14:12:03 +0000 Received: from localhost ([127.0.0.1]:57465 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdWmh-0001vH-6E for submit@debbugs.gnu.org; Tue, 05 Sep 2023 10:12:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58188) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdWmd-0001ul-O6 for 65720@debbugs.gnu.org; Tue, 05 Sep 2023 10:12:01 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdWmX-00063x-P7 for 65720@debbugs.gnu.org; Tue, 05 Sep 2023 10:11:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=Gq6HBLNSzUAf5+rzaCwbVT8YuN3r8ctRRGblQ3J+5X4=; b=UZsaQ744GAFUzmxGVZFj 0iSOUWoAHNugEitWhVLK/ru+d/Qj6Qhxa4rzE40Kb3XrYNW4zuvegyuJs+snjCDkbrDIyzw1WnZ0P VQ7OnRds3wWPcV7wfoF0PAYlJYVmYkn2ygyZEMnBeyAtP2M9661GP7W5r3a+cWyKtYP3ixPDYc+KP VLZe3m4XX3L5c3Lk0nzx2/zyqeKHi+suJZj6Zx6zt07nwtY5OvBpNE5trYJ4ZYpBLd/qi6dh+arGS g3bRDGy7IpavDvKhJ0GJG4XGNkvIREoKdRza/Cyhs+eHGHjjbFlb5jELZ8FdqUNZOvducPviMbwvU +wjclxceVmYbqw==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 65720@debbugs.gnu.org Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much References: <87bkejc7go.fsf@inria.fr> Date: Tue, 05 Sep 2023 16:11:33 +0200 In-Reply-To: <87bkejc7go.fsf@inria.fr> ("Ludovic =?utf-8?Q?Court=C3=A8s=22?= =?utf-8?Q?'s?= message of "Sun, 03 Sep 2023 22:44:39 +0200") Message-ID: <87wmx4lnfu.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > $ du -hs ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27= shj7apsnalwq > 6.7G /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlay= wmisyr27shj7apsnalwq Another data point, with Cuirass instances: --8<---------------cut here---------------start------------->8--- ludo@berlin ~$ sudo du -hs /var/lib/cuirass/.cache/guix/checkouts/pjmkglp4t= 7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq 65G /var/lib/cuirass/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3t= nlaywmisyr27shj7apsnalwq ludo@berlin ~$ sudo stat /var/lib/cuirass/.cache/guix/checkouts/pjmkglp4t7z= nuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq | tail -1 Birth: 2022-07-30 23:15:45.582559879 +0200 --8<---------------cut here---------------end--------------->8--- =E2=80=A6 and: --8<---------------cut here---------------start------------->8--- ludo@guix-hpc4 ~$ sudo du -hs /var/lib/cuirass/.cache/guix/checkouts/pjmkgl= p4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq 86G /var/lib/cuirass/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3t= nlaywmisyr27shj7apsnalwq ludo@guix-hpc4 ~$ sudo stat /var/lib/cuirass/.cache/guix/checkouts/pjmkglp4= t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq | tail -1 Cr=C3=A9=C3=A9=C2=A0: 2021-06-01 11:48:48.854669310 +0200 --8<---------------cut here---------------end--------------->8--- So yeah, problem we have. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 05 10:18:30 2023 Received: (at 65720) by debbugs.gnu.org; 5 Sep 2023 14:18:30 +0000 Received: from localhost ([127.0.0.1]:57485 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdWsw-00024t-0q for submit@debbugs.gnu.org; Tue, 05 Sep 2023 10:18:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37652) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdWss-00024d-PT for 65720@debbugs.gnu.org; Tue, 05 Sep 2023 10:18:27 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdWsm-00075u-MQ; Tue, 05 Sep 2023 10:18:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=gZEO0byQQf3bv9yNUhHvKI8UoLiI2n6yEJlAHLep+v4=; b=WrTkjU1vm18VaVFLfJxK oes9R9tjL00VRNzF8RvCpCHnfa0Ksjqa2Cp+VJW1waO3RmmL1luFIhoIRwd9WKO35cuoqvjLi31be 3iTS/6GoK1vGVHH8z7Pdd4gvnC2G0hCRbq6dkXEVuIYBegoA5CEbRLBZ7EWNO/zQuZ7KBs9l0vuYZ Muq4z43NPF6L1VZXKIEWrl9rTUcvilQjTf9rjp7DlDC+V3dXDoVivDCqUDnQG2bkzny7I6z2wILB5 LINW4/Qj2J2q+Ro3rM0wcgB0J+6Lt7NR1FYy58g7ueNrCZcxxITAQmnpi5kG2Y7RSz4fiJSuTuvwN P3VHhV1vXRCmaQ==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Josselin Poiret Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> Date: Tue, 05 Sep 2023 16:18:17 +0200 In-Reply-To: <87tts9uj6x.fsf@jpoiret.xyz> (Josselin Poiret's message of "Tue, 05 Sep 2023 10:18:30 +0200") Message-ID: <87msy0ln4m.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 Cc: 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Josselin Poiret skribis: > I think using the git binary instead of libgit2 as a workaround is a > good idea. We can consider building it directly as well, so that people > who don't have it in their profiles can still benefit from it. We could > even consider using git commands in most places and using libgit2 only > where we really need the tight coupling. Surely you=E2=80=99d agree that it would suck though: depending on two Git implementations because one doesn=E2=80=99t have a proper API and the other= one lacks a bunch of features. It would also be pretty bad for closure size: --8<---------------cut here---------------start------------->8--- $ guix size guile-git | tail -1 total: 106.6 MiB $ guix size guile-git git-minimal | tail -1 total: 169.8 MiB --8<---------------cut here---------------end--------------->8--- It=E2=80=99s also not clear concretely how we=E2=80=99d add that dependency= . Try invoking =E2=80=98git=E2=80=99 from $PATH and print a warning if it doesn= =E2=80=99t work? But then, what about applications like Cuirass and hpcguix-web? Tricky, tricky. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 05 10:21:27 2023 Received: (at 65720) by debbugs.gnu.org; 5 Sep 2023 14:21:28 +0000 Received: from localhost ([127.0.0.1]:57493 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdWvn-00029Y-Iz for submit@debbugs.gnu.org; Tue, 05 Sep 2023 10:21:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49908) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdWvk-00029K-5Q for 65720@debbugs.gnu.org; Tue, 05 Sep 2023 10:21:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdWvZ-0007oP-6U; Tue, 05 Sep 2023 10:21:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=Bt5Z+dmoKEFfEqZncGEW60V+4NqlHWt73Qz997Jox3U=; b=YpIru6c1+oKA4CaaAvWl sWF1RBZXf+anUYk/Dhb19VU/MDJwQix9LFo/RBJKwFloZzaoW8WY7CaPsa4cvdt2VDoS/RI+7ca1y jnK2l0EO67wrU8Zqc1AMdJTyOkjrKOTSxC0iEsvKgHDyXGJm8L7O1F58D3w8Oo6a5ZzVu792pUFgz yABKzFSmGn1xSrYfvXpLP8cN6unB4DOCq5Kpq+tLZe06dc5UEs2MGc9Hq5KFEjy51CdULuERBjS9h PwQH4x7eOhFYIhhyunZIPHY8eOU5qCMkutzz2FgVcoYiytXxHmB9CVqE14l4x4npzW+Aa9wimW8lN P1C6Yt02569ZKA==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Jelle Licht Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much References: <87fs3tobju.fsf@gnu.org> Date: Tue, 05 Sep 2023 16:20:57 +0200 In-Reply-To: (Jelle Licht's message of "Tue, 5 Sep 2023 08:22:48 +0000") Message-ID: <87il8oln06.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 Cc: 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, Jelle Licht skribis: >> On 4 Sep 2023, at 23:49, Ludovic Court=C3=A8s wrote: >>=20 >> Of course having to re-clone entire repositories every 9 months is >> ridiculous, but storing gigabytes of packs is worse IMO (I=E2=80=99m >> specifically thinking about the Guix repo, which every users copies via >> =E2=80=98guix pull=E2=80=99). > > Please ignore if it doesn=E2=80=99t make sense, or would not make a pract= ical difference for the current issue, but wouldn=E2=80=99t a local clone d= o the trick here? As in, clone from the =E2=80=98clogged=E2=80=99 local rep= o, move over fresh clone to old location. Good question. --8<---------------cut here---------------start------------->8--- scheme@(guix git)> ,use(git) scheme@(guix git)> (clone "/home/ludo/.cache/guix/checkouts/pjmkglp4t7znuug= eurpurzikxq3tnlaywmisyr27shj7apsnalwq/" "/tmp/fresh-clone") $7 =3D # scheme@(guix git)> (system* "du" "-hs" "/tmp/fresh-clone") 6.7G /tmp/fresh-clone $8 =3D 0 scheme@(guix git)> (system* "du" "-hs" "/tmp/fresh-clone/.git") 6.6G /tmp/fresh-clone/.git $9 =3D 0 scheme@(guix git)> (system* "du" "-hs" "/home/ludo/.cache/guix/checkouts/pj= mkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/") 6.7G /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisy= r27shj7apsnalwq/ $10 =3D 0 --8<---------------cut here---------------end--------------->8--- Conclusion: it makes no difference. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 05 15:01:20 2023 Received: (at 65720) by debbugs.gnu.org; 5 Sep 2023 19:01:20 +0000 Received: from localhost ([127.0.0.1]:59287 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdbId-0003sQ-RT for submit@debbugs.gnu.org; Tue, 05 Sep 2023 15:01:20 -0400 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]:38129) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdbIZ-0003rn-Lg for 65720@debbugs.gnu.org; Tue, 05 Sep 2023 15:01:19 -0400 Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-401ce65dfc4so9350215e9.0 for <65720@debbugs.gnu.org>; Tue, 05 Sep 2023 12:01:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693940469; x=1694545269; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=mCfMYEy946HhJdJ3oyKFtf36lO0XRBfdG9cIIRIKuwQ=; b=gqPXDMQTtZd2XIEOandnyWc2j/Rp+6Q8Um0a5zCjPWBKWmHEVzOzDTtybVgmW8bQky 8jaqroq/U3ZUakSPLHPx8EYYeAJmYturVNXyaDBicIgiUpjEXIxA3xEwaLBSxFJf+Or6 vLtCJX9Ev4eS1Tanv0S85odW88jPbuC2TzMJE9tvEGT8bG3mzilDKofBWAPyCtJRNODW agnLUXihkZC6b6wr/5nqE3xA+NN5WxtzX7nHJ1UOltt05N9+5ME/pLqmd6SWi0+mcgKa k38qnW/v/f2yZ+u86+TFgOeX+ohq5xfknn3QLpypyS65nDswhfpztWVWrEJwSIOORzWm DWZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693940469; x=1694545269; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:to:from:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=mCfMYEy946HhJdJ3oyKFtf36lO0XRBfdG9cIIRIKuwQ=; b=fNUEAENc6OWDaEoyMkH4+ltgXOqkX6kNquEARTuEY6q9uTYyONVl9yP9omveB2IRp3 oAGnNZvJORdkoKXUxtWp5w2OsMjQozrgPmpqv4fqz68DBVGFkwRn9g18eXKF95WlUZYv AWdpnRzRa0W99XFwqBTs9s4Go6dF66hQrm9tTzKTQ0foCh1Aubt1a9/HMI2yzWPKFqaz jzzcHbQSfH6I+R2PdVIXTiNECO1io3kV5dYHYj/Bhje7l8Yq7A4b/0liStUBwRlsTJso dtRlvpTKTEfIUFXM0QLRDGDL+eGVa9vZ7Z3hKjAKIVVT9uljEY3n+r5tFwd4c1PiMVJC n0Xw== X-Gm-Message-State: AOJu0Yw4cAQSirTYFkQOfP1peKs8lIEBunhaQwxXPdgnIhXKiaXwgJ0B K0r5fIbUFelB8SX+dE55dSPDU+oRwQQ= X-Google-Smtp-Source: AGHT+IGqfVbO7szTpGsEGwU/DGl9YdLE1sHWu/1G247m6RjDE09ds/itQgaLLpMAwm/cMo4OcywgsQ== X-Received: by 2002:a05:6000:141:b0:31a:e29f:5eab with SMTP id r1-20020a056000014100b0031ae29f5eabmr9325264wrx.7.1693940468913; Tue, 05 Sep 2023 12:01:08 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id bt9-20020a056000080900b0031c5ce91ad6sm14297502wrb.97.2023.09.05.12.01.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 12:01:08 -0700 (PDT) From: Simon Tournier To: Ludovic =?utf-8?Q?Court=C3=A8s?= , 65720@debbugs.gnu.org Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much In-Reply-To: <87fs3tobju.fsf@gnu.org> References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> Date: Tue, 05 Sep 2023 20:59:07 +0200 Message-ID: <86edjcqwec.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65720 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, On Mon, 04 Sep 2023 at 23:47, Ludovic Court=C3=A8s wrote: >> It would seem that libgit2 doesn=E2=80=99t do the equivalent of =E2=80= =98git gc=E2=80=99. > > Confirmed: . Ouch! The goals of the project haven't changed, and neither have the tradeoffs. If one were to rewrite git-gc on top of libgit2, the best-case scenario is ending up with what we already had. If you want to use regular maintenance on some repostories, use git gc, that's what it's there for. https://github.com/libgit2/libgit2/issues/3247#issuecomment-1525080= 40 > My inclination for the short term would be to work around this > limitation by (1) finding a heuristic to determine is a checkout has > likely accumulated too much cruft, and (2) considering such checkouts > as expired (thereby forcing a re-clone) or running =E2=80=98git gc=E2=80= =99 on them if > =E2=80=98git=E2=80=99 is available. About (1) maybe we could add a =E2=80=9Ccounter=E2=80=9D and teach after X = updates of the checkout then let run (2). Well, I guess the number of crufts is more or less proportional with the number of checkout updates; that=E2=80= =99s the heuristic I would use. The most annoying is (2). Because forcing a re-clone does not appear to me a solution; I prefer to waste disk space (and probably run myself and manually =E2=80=98git gc=E2=80=99) than re-clone=E2=80=A6 Somehow this re-c= lone would always happen when I am using a poor network. Moreover, assuming this clean-up (2) would be run once every while, we could imagine to invoke something like, guix shell -C git-minimal -- git -C ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr2= 7shj7apsnalwq gc when the checkout is updated. And maybe we could provide another =E2=80=9C= guix pull=E2=80=9D command-line option for turning off this and mark it as done (reset the =E2=80=9Ccounter=E2=80=9D). Well, that=E2=80=99s a poor solution but we can assume that git-minimal is = at worse available using =E2=80=9Cguix shell git-minimal=E2=80=9D. Note that = the closure of git-minimal is far less than re-cloning the full Guix repository. Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 06 04:04:26 2023 Received: (at 65720) by debbugs.gnu.org; 6 Sep 2023 08:04:26 +0000 Received: from localhost ([127.0.0.1]:60737 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdnWU-0004DF-36 for submit@debbugs.gnu.org; Wed, 06 Sep 2023 04:04:26 -0400 Received: from jpoiret.xyz ([206.189.101.64]:34952) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdnWR-0004D6-9H for 65720@debbugs.gnu.org; Wed, 06 Sep 2023 04:04:24 -0400 Received: from authenticated-user (jpoiret.xyz [206.189.101.64]) by jpoiret.xyz (Postfix) with ESMTPA id 91F69184D5F; Wed, 6 Sep 2023 08:04:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jpoiret.xyz; s=dkim; t=1693987460; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gyi8zkcyvxV3/1pT9N7jwCyamv5CshT2blTVJG3Xnrc=; b=fQg34g6JyiP+/me0aYN7zBWmsq5wxDC+grcS0a3n7Du2UVpugp4VzeUthbr+yLvznPY/6G C3U4KKoK9LcFooMUDcARSzBrmBOO4iR/rQbocaerR7nCoJ9Q4YnEu00vKyUYItgdTeNudo tN86jN9oE7QlfvXjImUROsHF0B0EBnNqz6zR58ljB8ztH4iH9LtE2tAK4cSUuizuePgQix KwJsjMGGcw6o1tSBvDi2OiZcz24nW7b1sqU70Aaoa0yTu7k1+nBIMBhtcPYUtFPizu1iZN X57Oaq9rFdJFZnGuf0mVP5h4JMvLbZx+U/BNFHo1ingazKEaqviwAS/DCFH/hA== From: Josselin Poiret To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much In-Reply-To: <87msy0ln4m.fsf@gnu.org> References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> Date: Wed, 06 Sep 2023 10:04:11 +0200 Message-ID: <87pm2vvibo.fsf@jpoiret.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spamd-Bar: -- Authentication-Results: jpoiret.xyz; auth=pass smtp.auth=jpoiret@jpoiret.xyz smtp.mailfrom=dev@jpoiret.xyz X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65720 Cc: 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo, Ludovic Court=C3=A8s writes: > Surely you=E2=80=99d agree that it would suck though: depending on two Git > implementations because one doesn=E2=80=99t have a proper API and the oth= er one > lacks a bunch of features. Right, although I wouldn't necessarily say that the former doesn't have a proper API, but rather that it has a Unix-oriented API. That leads to performance issues on e.g. Windows but on Linux I'm not sure there's much of a difference. > It would also be pretty bad for closure size: > > --8<---------------cut here---------------start------------->8--- > $ guix size guile-git | tail -1 > total: 106.6 MiB > $ guix size guile-git git-minimal | tail -1 > total: 169.8 MiB > --8<---------------cut here---------------end--------------->8--- > > It=E2=80=99s also not clear concretely how we=E2=80=99d add that dependen= cy. Try > invoking =E2=80=98git=E2=80=99 from $PATH and print a warning if it doesn= =E2=80=99t work? > But then, what about applications like Cuirass and hpcguix-web? > > Tricky, tricky. We could consider replacing the guile-git dependency with another library built directly on top of git-minimal, and have this be a dependency of Guix. Not ideal though, and not really scalable either: we can't just add every VCS as direct dependencies. From=20what I've seen, people are now scaling back on their use of libgit2 because of the impedence mismatch and are resorting more and more to git plumbing. From a pragmatic point of view, I'd prefer the latter, since it is more stable and feature-complete. Best, =2D-=20 Josselin Poiret --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQHEBAEBCgAuFiEEOSSM2EHGPMM23K8vUF5AuRYXGooFAmT4MnsQHGRldkBqcG9p cmV0Lnh5egAKCRBQXkC5FhcairUvDACAJZdGUEBC2qVWbsms7Xk6OUTUgfjucIC3 XOLFH74Ewo4OdrUJUrADcWP4GKjrEmglO1hQjRlTwpo60TB7CkFyZIC39Dkm0MPm R7Oc8BYnPByFHihy3RJwrtk0zH1jOaRd/A6cvdIXCrXk1rnlTBn9EEAKpYlA1OrG 7al4FfxKfFkea48xZsGVM8uc1fsqiHrycZH3gLCbT8V0O4BtNY2rhYLf1eTjFkQl PbcAdHmkOcnQaZR2WzUHUUH/9GGrHwcXkqSZtgnJ8y/zauig5nyWzX3Zgej4K+VJ nN0l6QCVtkaAIgCN2+8zTg2ml+WeXjWcE1gGjVyv7748ICmpP3jXE0uGTSGvM0MX ZFfi+TKU4wJUl8fCZLf41P5v7P1jSy2TbthqbAHkRKlsasYMf/KZMhSh6jypVIN0 VOFPWMUIJhZNQMP2DtVOMC/thc+O8BcfkbFSORNJ5XYaLiNvC4/ODwsKCGxYOa2h 4XlDMtm7YgAewG19hO9fYvbS1oXKLZM= =olI3 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 06 20:43:43 2023 Received: (at 65720) by debbugs.gnu.org; 7 Sep 2023 00:43:43 +0000 Received: from localhost ([127.0.0.1]:38163 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qe37X-0004h6-9x for submit@debbugs.gnu.org; Wed, 06 Sep 2023 20:43:43 -0400 Received: from mail-wm1-x32e.google.com ([2a00:1450:4864:20::32e]:41365) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qe37T-0004gS-N9 for 65720@debbugs.gnu.org; Wed, 06 Sep 2023 20:43:40 -0400 Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-401e6ce2d9fso1031245e9.1 for <65720@debbugs.gnu.org>; Wed, 06 Sep 2023 17:43:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694047412; x=1694652212; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=hMZsS0KKScfuGIZ10c6RNyODKqltKAVWn2apINlpol0=; b=m6oBDHt7M6ajRVArZs+ixrIWhJfiwlY6vh8olqLpIeyp/XvBV9+5YhyDY/1ia3XHc5 IR84Z6seEgfUM7zKjFVAS8RRUMt/195EXH9Sac1d1lsl1ToZzwk3gPjwsCJZY4M+ihEW HokuC/bNeS2bhGEGVrZu6OobxIsUrrSWkYOQNaEQ4rG96LhbFaY7ScW6WUURAGs6NcQ4 nhKokFqLnExUlbct+JRQaEOpZb8TzS2eSdsIL2Ozhdbi4pGU62P2H1n0IoKBKpyISY00 Ays54sQPCT4JJLo+1Kax5MVCUxX2jbJc7oVRrD1paROP5w6LwU+9Dyz2LeoDENdJzJ77 DuUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1694047412; x=1694652212; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hMZsS0KKScfuGIZ10c6RNyODKqltKAVWn2apINlpol0=; b=OAB6G43AoHpOSn89n+tnuxzBGwm9gCDRASrptNuoQ+Iotpbh19LnbzkeuBfF80PI+X wjBF7ePsirZy2tnP3PSZMni5y6mifkW/qgeogKolpt5CLy/NTpwxxouDfwtgMw7oJQBA KXkWinKwwofUmF8hB6KTdr3z2HgOgvROsaIgfN/r+Nm2Opn071BTxjHBl+fnZvFnmTGZ IrsznP8Sxo7E6CAnB/QhXvHo5sJaoTUHfv3mn0zIfFCZtncByR5ibrhsSi25vHpUSwMk Td6UqcSze8a7ffYn5EJ58IQJh1OMKOOfsNn2KKupQAm2K5tr+wa5pfb3wJmmn8OnjR4J hksA== X-Gm-Message-State: AOJu0YxeMuZOHAQSl6+kev6NBHERiBEJKN+AaZWjvv5ROMTqxakDapLr BOtvTsd7M6Mdwcn5mVnqXQIS52vC3F0= X-Google-Smtp-Source: AGHT+IGp9HK6PcjPwF89K2Tuc+A8RlkJd9zGQlDjGUfWgtXaB4XR4uhZs1naGE3781v4/pHe19yn0A== X-Received: by 2002:a5d:4b87:0:b0:31a:ea18:c516 with SMTP id b7-20020a5d4b87000000b0031aea18c516mr13157801wrt.3.1694047412002; Wed, 06 Sep 2023 17:43:32 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id y1-20020adff6c1000000b00317a29af4b2sm21595830wrp.68.2023.09.06.17.43.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Sep 2023 17:43:31 -0700 (PDT) From: Simon Tournier To: Ludovic =?utf-8?Q?Court=C3=A8s?= , Josselin Poiret Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much In-Reply-To: <87msy0ln4m.fsf@gnu.org> References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> Date: Thu, 07 Sep 2023 02:41:54 +0200 Message-ID: <86il8mn7al.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65720 Cc: 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, On Tue, 05 Sep 2023 at 16:18, Ludovic Court=C3=A8s wrote: > It would also be pretty bad for closure size: > > --8<---------------cut here---------------start------------->8--- > $ guix size guile-git | tail -1 > total: 106.6 MiB > $ guix size guile-git git-minimal | tail -1 > total: 169.8 MiB > --8<---------------cut here---------------end--------------->8--- > > It=E2=80=99s also not clear concretely how we=E2=80=99d add that dependen= cy. Try > invoking =E2=80=98git=E2=80=99 from $PATH and print a warning if it doesn= =E2=80=99t work? > But then, what about applications like Cuirass and hpcguix-web? I think we can rely on something like, guix shell -C git-minimal -- git gc It would be invoked internally using the Scheme API for inferiors and friends. Doing so, it would add nothing to the closure size. It appears to me safe to assume that this command can be run from any Guix installation. Since the Git GC would only be done once every X Git fetches, the overhead would be much lower. Hum, am I repeating myself [1]? :-) And I would run this =E2=80=9Cgit gc=E2=80=9D via =E2=80=9Cguix gc=E2=80=9D= , not via =E2=80=9Cguix pull=E2=80=9D. Well, I do not like all these automatic removals happening based on date (last-expiry-cleanup) with some usual commands. It always happens when I do not want. ;-) Contrary to =E2=80=9Cguix gc=E2=80=9D. Bah, another sto= ry. :-) Cheers, simon 1: bug#65720: Guile-Git-managed checkouts grow way too much Simon Tournier Tue, 05 Sep 2023 20:59:07 +0200 id:86edjcqwec.fsf@gmail.com https://issues.guix.gnu.org//65720 https://issues.guix.gnu.org/msgid/86edjcqwec.fsf@gmail.com https://yhetil.org/guix/86edjcqwec.fsf@gmail.com From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 08 13:08:18 2023 Received: (at 65720) by debbugs.gnu.org; 8 Sep 2023 17:08:19 +0000 Received: from localhost ([127.0.0.1]:45452 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qeexu-0005F7-Jp for submit@debbugs.gnu.org; Fri, 08 Sep 2023 13:08:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33820) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qeexs-0005Es-6M for 65720@debbugs.gnu.org; Fri, 08 Sep 2023 13:08:17 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qeexk-0001Zu-7i; Fri, 08 Sep 2023 13:08:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=Lhl/xDj71JknxWuiD+ZkuXeTLLdfQqIwQpKUD7oloTI=; b=BU2IfNbirdykSUNnWi08 6jnLCqvdp7Ds6yb0zLjunNjuqe42RJ3gPulzG3WEEpWLdIzmgkiSo9PWAo2lJs1rcuF17Z/Q3DUWK Wo0nUWUGaUCRXUV8kjYkOtCjRKaHVRzbnE48QlA7TcO2oYMGeCz9mNKMLqDOUfEhlCLRRas0Q8ScT vxCxPp7uO9oSRFM5ZuTACDAx3CGgjJElVyk6RFMPY83x9y8ZCqFe3NgMtBK3wlLfq8HFdTTc035SA ifiQxoXFzkOMHH2s5XAOzvo7wrnihk+BFeIrZ62yfKOFYRAXkP6UFXneLYDPdmutAaFLXdc+o7e8T D8DYM6qXN3ywvQ==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Josselin Poiret Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> <87pm2vvibo.fsf@jpoiret.xyz> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Duodi 22 Fructidor an 231 de la =?utf-8?Q?R=C3=A9vol?= =?utf-8?Q?ution=2C?= jour de la Noisette X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 08 Sep 2023 19:08:05 +0200 In-Reply-To: <87pm2vvibo.fsf@jpoiret.xyz> (Josselin Poiret's message of "Wed, 06 Sep 2023 10:04:11 +0200") Message-ID: <87pm2s385m.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 Cc: 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello! Josselin Poiret skribis: > Right, although I wouldn't necessarily say that the former doesn't have > a proper API, but rather that it has a Unix-oriented API. That leads to > performance issues on e.g. Windows but on Linux I'm not sure there's > much of a difference. [...] > We could consider replacing the guile-git dependency with another > library built directly on top of git-minimal, and have this be a > dependency of Guix. Not ideal though, and not really scalable either: > we can't just add every VCS as direct dependencies. I cannot imagine a viable implementation of things like =E2=80=98commit-clo= sure=E2=80=99 and =E2=80=98commit-relation=E2=80=99 from (guix git) done by shelling out = to =E2=80=98git=E2=80=99. I=E2=80=99m quite confident this would be slow and brittle. It looks like there=E2=80=99s no option other than carrying the two implementations. ~~~ Years ago, Andy Wingo sketched a plan for GNU hackers to implement Git in pure Scheme. That was on April 1st though, so people mistakenly assumed it was a joke and the project was never carried out. I digress, but I wonder: is there not even a viable Haskell or OCaml implementation of Git? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 08 13:10:06 2023 Received: (at 65720) by debbugs.gnu.org; 8 Sep 2023 17:10:06 +0000 Received: from localhost ([127.0.0.1]:45457 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qeeze-0005Hz-43 for submit@debbugs.gnu.org; Fri, 08 Sep 2023 13:10:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37014) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qeezc-0005HO-Vj for 65720@debbugs.gnu.org; Fri, 08 Sep 2023 13:10:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qeezV-0002fC-7a; Fri, 08 Sep 2023 13:09:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=dXf8knm9Rru+7HimoEi0QYUzsGr+djsbxH5qp3mxxis=; b=MEXysHIhl9LU6xc/ryAr wsN5uY0LVxBQsOPTkyMVaH2/0E/xemumCJbEtSYpDE47wSpNymyuSM8AeRxepDIGf3jEE6lvdR+Nd dIdmcFOtqHiRBCXySBGkZgXi4kOwEw2+hyk+rTqO31H8hBfhw0yvHY5OzwQUqfTxsTR8Fl4EJD7+n jl4Wcx13VcxJaKh9nbYyo/Zkem2SJAx6n7LwKlNupzkXMJ5Ca5CzbobXVADiO8m0mnWAtu5HTjD+m +N5cCIVFXhfUfipaP7ff6yX36shIQcPYxfZqB5sS6oEZLcR3daNYcSLv6a3kBGs9PeihxQJ66SjJx o5Z0IfQX6ZJtCg==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Simon Tournier Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> <86il8mn7al.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Duodi 22 Fructidor an 231 de la =?utf-8?Q?R=C3=A9vol?= =?utf-8?Q?ution=2C?= jour de la Noisette X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 08 Sep 2023 19:09:54 +0200 In-Reply-To: <86il8mn7al.fsf@gmail.com> (Simon Tournier's message of "Thu, 07 Sep 2023 02:41:54 +0200") Message-ID: <87jzt0382l.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 Cc: Josselin Poiret , 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Simon Tournier skribis: > On Tue, 05 Sep 2023 at 16:18, Ludovic Court=C3=A8s wrote: > >> It would also be pretty bad for closure size: >> >> --8<---------------cut here---------------start------------->8--- >> $ guix size guile-git | tail -1 >> total: 106.6 MiB >> $ guix size guile-git git-minimal | tail -1 >> total: 169.8 MiB >> --8<---------------cut here---------------end--------------->8--- >> >> It=E2=80=99s also not clear concretely how we=E2=80=99d add that depende= ncy. Try >> invoking =E2=80=98git=E2=80=99 from $PATH and print a warning if it does= n=E2=80=99t work? >> But then, what about applications like Cuirass and hpcguix-web? > > I think we can rely on something like, > > guix shell -C git-minimal -- git gc We=E2=80=99re talking about the implementation of a cache (meant to speed up operations), that would actually fill said cache plus do a whole bunch of expensive operations? Nah. :-) Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 09 08:33:13 2023 Received: (at 65720) by debbugs.gnu.org; 9 Sep 2023 12:33:13 +0000 Received: from localhost ([127.0.0.1]:46359 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qex9F-0000SZ-9p for submit@debbugs.gnu.org; Sat, 09 Sep 2023 08:33:13 -0400 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:39422) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qex9A-0000S5-2e for 65720@debbugs.gnu.org; Sat, 09 Sep 2023 08:33:08 -0400 Received: by mail-wr1-x434.google.com with SMTP id ffacd0b85a97d-3175e1bb38cso481221f8f.1 for <65720@debbugs.gnu.org>; Sat, 09 Sep 2023 05:33:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694262779; x=1694867579; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=d6ewT6orRq96ryi4+ofr8KXNoRevq8PABtO1IA0+1mQ=; b=XYIjXe3ZYWc1X3QQqNofjFpv+cs0GRZcYFG+vrEmoZI4S0ZejGdSI7rTQLDhfta8nd GuJlldccjO31QGuVVr2d3RcwrXCDXWBzk0vSDAC8udAnKbb1EqnWg3JzX8idhf8IMfeT AwWSr5h1X/srL8/EaAcPcn8vBROodeep8WlioHgiZjIUuLr4qydGfxakAC8x8AKZs38Y CTQMar/tY9JnRm36GlMz2mfPLcLjRulh+3iAlj3YuDKVU0iBRZWvjlMnYvrvpAGyJIDh judu0S7+3+hKWeRe1p09oiMtm31mDx513cyBqXr513KW1JfbNmOMU/HlOqXB4lESd+9h BLGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694262779; x=1694867579; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d6ewT6orRq96ryi4+ofr8KXNoRevq8PABtO1IA0+1mQ=; b=UvCXaSGn0+f27zOtL9JBqs93VJM6vBA0KfvEu8lNq8Uiaxo+ZQgiLEXYogPJrYHHdo ZvtxwfMmOa4Fwj1k0SCyQNNNyjGvzR6T7j7YbrS0emU6ikG8PV7MywWgsJnIuWnZywyM 4FNNGQdzQxF5T50rlAITbviiZDG163JYwc81zNsCdOZw2B3xwVWRaasjX/DIHEr2nC4g WGbKs21O3Yvi6UZLBR9Klj/mqbCqBOkcyXSO4voWd1kXJ7Hw0pG2dqgKT998cPSH+HUb +yl+ZtvrLJqF5/Cl/CGri8rkajAwAaYsQg5TL0Eph2LRATFXq7uuv6dKwW/GA40o3LkB 5DAQ== X-Gm-Message-State: AOJu0Yy9Rv8GKkSCkD8BLhWNCJGs9WmeZiUZMlNKYFcbdcHewsjJ5c7s MC9MAl2mUP9KJ6GxowdF1OeJhmAUlhg= X-Google-Smtp-Source: AGHT+IHapoioggDK+q/PoFsj4M5AULl2AXhOseWEAOC9iCQL+ZAdc32q2RdfLcHAQ5DvZ0ZWZcEDtQ== X-Received: by 2002:a5d:620f:0:b0:319:7624:4ca2 with SMTP id y15-20020a5d620f000000b0031976244ca2mr4014016wru.0.1694262779211; Sat, 09 Sep 2023 05:32:59 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id x15-20020a5d60cf000000b00317a04131c5sm4704478wrt.57.2023.09.09.05.32.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Sep 2023 05:32:58 -0700 (PDT) From: Simon Tournier To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much In-Reply-To: <87jzt0382l.fsf@gnu.org> References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> <86il8mn7al.fsf@gmail.com> <87jzt0382l.fsf@gnu.org> Date: Sat, 09 Sep 2023 12:31:48 +0200 Message-ID: <86cyyrskmj.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65720 Cc: Josselin Poiret , 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, On Fri, 08 Sep 2023 at 19:09, Ludovic Court=C3=A8s wrote: >>> It would also be pretty bad for closure size: >>> >>> --8<---------------cut here---------------start------------->8--- >>> $ guix size guile-git | tail -1 >>> total: 106.6 MiB >>> $ guix size guile-git git-minimal | tail -1 >>> total: 169.8 MiB >>> --8<---------------cut here---------------end--------------->8--- >>> >>> It=E2=80=99s also not clear concretely how we=E2=80=99d add that depend= ency. Try >>> invoking =E2=80=98git=E2=80=99 from $PATH and print a warning if it doe= sn=E2=80=99t work? >>> But then, what about applications like Cuirass and hpcguix-web? >> >> I think we can rely on something like, >> >> guix shell -C git-minimal -- git gc > > We=E2=80=99re talking about the implementation of a cache (meant to speed= up > operations), that would actually fill said cache plus do a whole bunch > of expensive operations? Nah. :-) I do not think. If I understand correctly, we need to run =E2=80=9Cgit gc= =E2=80=9D at some point, therefore git-minimal needs to me around. The question is how and when. Well, maybe I am missing what the bug is about. For me, it is about running =E2=80=98git gc=E2=80=99 for cleaning the Git checkout cache, no? Solution #1. Add git-minimal as inputs. It increases the closure and the extra load (on average) is about the ratio between the rate of =E2=80= =9Cguix pull=E2=80=9D and the rate of the git-minimal changes. Assuming, that people are running =E2=80=9Cguix pull=E2=80=9D once per week= and say =E2=80=9Cgit gc=E2=80=9D is run after 50 pulls. (These both number are totally arbitrar= y and based on my personal estimate). Data Service [1] tells: 2023-07-07 15:45:22 2023-09-08 21:22:08 2023-05-11 16:10:48 2023-07-07 14:21:45 2023-05-01 16:40:08 2023-05-11 14:36:16 2023-04-25 13:34:54 2023-05-01 15:19:55 2023-04-25 13:34:54 2023-09-08 21:22:08=20=20=20=20=20=20=20=20 2023-03-06 17:22:28 2023-04-25 12:27:33 2023-01-17 23:49:19 2023-03-06 16:48:43 2022-11-08 13:06:42 2023-01-17 15:11:47 2022-10-08 05:14:46 2022-11-08 09:56:31 2022-09-06 15:00:08 2022-10-08 04:15:43 2022-08-13 22:02:31 2022-09-06 12:58:52 =E2=80=A6 It means that an user will download ~10 times git-minimal for nothing. Solution #2. The one I am proposing. :-) Download git-minimal only when Guix needs it for running =E2=80=9Cgit gc=E2=80=9D. Yeah, there is pr= obably a small overload with some operations. But, I bet this overload is much smaller than the one of solution #1. Well, it depends on the number of times people are updating the cache vs the rate of change of git-minimal. For sure, if one updates 100 times per week the cache, having git-minimal as inputs is far better. But I do not think that the regular usage on average. :-) That=E2=80=99s why I am proposing to have an option for turning off this = =E2=80=9Cgit gc=E2=80=9C operation. Well, we have lived since years without running =E2=80=98git gc=E2=80=99 so= running it once per year on average is probably enough to keep the cache size reasonable. And git-minimal is changing every month. Maybe, there is some solution #3. ;-) Cheers, simon 1: https://data.guix.gnu.org/repository/1/branch/master/package/git-minimal= /output-history From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 11 03:06:17 2023 Received: (at submit) by debbugs.gnu.org; 11 Sep 2023 07:06:17 +0000 Received: from localhost ([127.0.0.1]:51990 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfazw-0006gh-2H for submit@debbugs.gnu.org; Mon, 11 Sep 2023 03:06:17 -0400 Received: from lists.gnu.org ([2001:470:142::17]:43114) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfazu-0006gO-03 for submit@debbugs.gnu.org; Mon, 11 Sep 2023 03:06:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfazh-0007YJ-O5 for bug-guix@gnu.org; Mon, 11 Sep 2023 03:06:02 -0400 Received: from mx0.riseup.net ([198.252.153.6]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfaze-0008N9-6O; Mon, 11 Sep 2023 03:06:01 -0400 Received: from fews02-sea.riseup.net (fews02-sea-pn.riseup.net [10.0.1.112]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx0.riseup.net (Postfix) with ESMTPS id 4Rkd6C1GS0z9t0C; Mon, 11 Sep 2023 07:05:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1694415955; bh=DvgxFBmsqF7lmWFKoPyuRScC1d3IK0LLKh3DGJfCs7w=; h=References:From:To:Cc:Subject:Date:In-reply-to:From; b=sgMvSPrS8Pqcb73dlwo1+I2JXpvmCx1F0Tt0CTVbKQnkF7lWAczBAiDV6jyhC1sCT zfuyja8n50BiNer/VIBbZQtZRK/F2f0KqQ9v3JkgROW0dP+ALQ8y9V8WBh3knp4cUh fR8lYF8fzNw66grNOuDMoXis7/WCR9JYn1gNymyY= X-Riseup-User-ID: A0079F29EF7EC8FC5519C967AEF101960BA8D34E079E6B32377221C8FCF49627 Received: from [127.0.0.1] (localhost [127.0.0.1]) by fews02-sea.riseup.net (Postfix) with ESMTPSA id 4Rkd5z3M0dzFqjr; Mon, 11 Sep 2023 07:05:43 +0000 (UTC) References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> <87pm2vvibo.fsf@jpoiret.xyz> <87pm2s385m.fsf@gnu.org> From: Csepp To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much Date: Mon, 11 Sep 2023 09:00:07 +0200 In-reply-to: <87pm2s385m.fsf@gnu.org> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=198.252.153.6; envelope-from=raingloom@riseup.net; helo=mx0.riseup.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: submit Cc: Josselin Poiret , 65720@debbugs.gnu.org, bug-guix@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Ludovic Court=C3=A8s writes: > Hello! > > Josselin Poiret skribis: > >> Right, although I wouldn't necessarily say that the former doesn't have >> a proper API, but rather that it has a Unix-oriented API. That leads to >> performance issues on e.g. Windows but on Linux I'm not sure there's >> much of a difference. > > [...] > >> We could consider replacing the guile-git dependency with another >> library built directly on top of git-minimal, and have this be a >> dependency of Guix. Not ideal though, and not really scalable either: >> we can't just add every VCS as direct dependencies. > > I cannot imagine a viable implementation of things like =E2=80=98commit-c= losure=E2=80=99 > and =E2=80=98commit-relation=E2=80=99 from (guix git) done by shelling ou= t to =E2=80=98git=E2=80=99. > I=E2=80=99m quite confident this would be slow and brittle. > > It looks like there=E2=80=99s no option other than carrying the two > implementations. > > ~~~ > > Years ago, Andy Wingo sketched a plan for GNU hackers to implement Git > in pure Scheme. That was on April 1st though, so people mistakenly > assumed it was a joke and the project was never carried out. > > I digress, but I wonder: is there not even a viable Haskell or OCaml > implementation of Git? > > Thanks, > Ludo=E2=80=99. For sake of completeness: There is an alternative implentation in C for Plan 9 that I've used and is now mature enough that the 9front project switched to it from Mercurial. It might be possible to compile it with the plan9port compiler wrapper. There is also a Git implementation in OCaml that some MirageOS unikernels use to serve static content from a git repository. Also the Irmin "database" is based on git and is written in OCaml. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 11 03:10:13 2023 Received: (at submit) by debbugs.gnu.org; 11 Sep 2023 07:10:13 +0000 Received: from localhost ([127.0.0.1]:51998 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfb3l-0006ma-Aw for submit@debbugs.gnu.org; Mon, 11 Sep 2023 03:10:13 -0400 Received: from lists.gnu.org ([2001:470:142::17]:57050) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfb3j-0006m6-Qu for submit@debbugs.gnu.org; Mon, 11 Sep 2023 03:10:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfb3Z-0001oZ-4L for bug-guix@gnu.org; Mon, 11 Sep 2023 03:10:01 -0400 Received: from mx1.riseup.net ([198.252.153.129]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfb3W-0000fI-Hk; Mon, 11 Sep 2023 03:10:00 -0400 Received: from fews02-sea.riseup.net (fews02-sea-pn.riseup.net [10.0.1.112]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx1.riseup.net (Postfix) with ESMTPS id 4RkdBr30skzDqpZ; Mon, 11 Sep 2023 07:09:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1694416196; bh=jCBwtrOdu4IjIK1DNoGSobWo/knpWw03ag0O4gDJiso=; h=References:From:To:Cc:Subject:Date:In-reply-to:From; b=p8OnFoG5FSZ4qVyuqU7eoO1NHJb8FeMLpDQu9pEtagI3uq2ku2qWHNqtfOFkZMiSk P9inRFGTA0r8s30MFTbGMkkBqwFCiediO2TsptW5zYwKk0QKIZglisSWlbs+QkSsCG CDxEWNaOUY4BssSxgZaE29mtnumAa53HgUXUmdhU= X-Riseup-User-ID: 558C9CB603439A5DAFC0FDC808E10CF6C3CA907716EB7D430B23A41F042139CA Received: from [127.0.0.1] (localhost [127.0.0.1]) by fews02-sea.riseup.net (Postfix) with ESMTPSA id 4RkdBb6bTLzFrwy; Mon, 11 Sep 2023 07:09:43 +0000 (UTC) References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> <86il8mn7al.fsf@gmail.com> <87jzt0382l.fsf@gnu.org> <86cyyrskmj.fsf@gmail.com> From: Csepp To: Simon Tournier Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much Date: Mon, 11 Sep 2023 09:06:58 +0200 In-reply-to: <86cyyrskmj.fsf@gmail.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=198.252.153.129; envelope-from=raingloom@riseup.net; helo=mx1.riseup.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: submit Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , 65720@debbugs.gnu.org, bug-guix@gnu.org, Josselin Poiret X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Simon Tournier writes: > Hi, > > On Fri, 08 Sep 2023 at 19:09, Ludovic Court=C3=A8s wrote: > >>>> It would also be pretty bad for closure size: >>>> >>>> --8<---------------cut here---------------start------------->8--- >>>> $ guix size guile-git | tail -1 >>>> total: 106.6 MiB >>>> $ guix size guile-git git-minimal | tail -1 >>>> total: 169.8 MiB >>>> --8<---------------cut here---------------end--------------->8--- >>>> >>>> It=E2=80=99s also not clear concretely how we=E2=80=99d add that depen= dency. Try >>>> invoking =E2=80=98git=E2=80=99 from $PATH and print a warning if it do= esn=E2=80=99t work? >>>> But then, what about applications like Cuirass and hpcguix-web? >>> >>> I think we can rely on something like, >>> >>> guix shell -C git-minimal -- git gc >> >> We=E2=80=99re talking about the implementation of a cache (meant to spee= d up >> operations), that would actually fill said cache plus do a whole bunch >> of expensive operations? Nah. :-) > > I do not think. If I understand correctly, we need to run =E2=80=9Cgit g= c=E2=80=9D at > some point, therefore git-minimal needs to me around. The question is > how and when. > > Well, maybe I am missing what the bug is about. For me, it is about > running =E2=80=98git gc=E2=80=99 for cleaning the Git checkout cache, no? > > > Solution #1. Add git-minimal as inputs. It increases the closure and > the extra load (on average) is about the ratio between the rate of =E2=80= =9Cguix > pull=E2=80=9D and the rate of the git-minimal changes. > > Assuming, that people are running =E2=80=9Cguix pull=E2=80=9D once per we= ek and say =E2=80=9Cgit > gc=E2=80=9D is run after 50 pulls. (These both number are totally arbitr= ary and > based on my personal estimate). > > Data Service [1] tells: > > 2023-07-07 15:45:22 2023-09-08 21:22:08 > 2023-05-11 16:10:48 2023-07-07 14:21:45 > 2023-05-01 16:40:08 2023-05-11 14:36:16 > 2023-04-25 13:34:54 2023-05-01 15:19:55 > 2023-04-25 13:34:54 2023-09-08 21:22:08=20=20=20=20=20=20=20=20 > 2023-03-06 17:22:28 2023-04-25 12:27:33 > 2023-01-17 23:49:19 2023-03-06 16:48:43 > 2022-11-08 13:06:42 2023-01-17 15:11:47 > 2022-10-08 05:14:46 2022-11-08 09:56:31 > 2022-09-06 15:00:08 2022-10-08 04:15:43 > 2022-08-13 22:02:31 2022-09-06 12:58:52 > =E2=80=A6 > > It means that an user will download ~10 times git-minimal for nothing. > > > Solution #2. The one I am proposing. :-) Download git-minimal only > when Guix needs it for running =E2=80=9Cgit gc=E2=80=9D. Yeah, there is = probably a > small overload with some operations. But, I bet this overload is much > smaller than the one of solution #1. > > Well, it depends on the number of times people are updating the cache vs > the rate of change of git-minimal. > > For sure, if one updates 100 times per week the cache, having > git-minimal as inputs is far better. But I do not think that the > regular usage on average. :-) > > That=E2=80=99s why I am proposing to have an option for turning off this = =E2=80=9Cgit > gc=E2=80=9C operation. > > Well, we have lived since years without running =E2=80=98git gc=E2=80=99 = so running it > once per year on average is probably enough to keep the cache size > reasonable. And git-minimal is changing every month. > > > Maybe, there is some solution #3. ;-) > > Cheers, > simon > > > 1: https://data.guix.gnu.org/repository/1/branch/master/package/git-minim= al/output-history Please don't create another situation like with guix system roll-back, where a crucial sysadmin operation doesn't work without network access. Or at least make it configurable, so things that are likely to be needed for future operations are pre-fetched. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 11 07:24:16 2023 Received: (at 65720) by debbugs.gnu.org; 11 Sep 2023 11:24:16 +0000 Received: from localhost ([127.0.0.1]:52246 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qff1c-0007RU-4b for submit@debbugs.gnu.org; Mon, 11 Sep 2023 07:24:16 -0400 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]:39077) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qff1W-0007R6-O6 for 65720@debbugs.gnu.org; Mon, 11 Sep 2023 07:24:14 -0400 Received: by mail-wr1-x433.google.com with SMTP id ffacd0b85a97d-31f2ab21231so655497f8f.0 for <65720@debbugs.gnu.org>; Mon, 11 Sep 2023 04:24:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694431441; x=1695036241; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=C8OuW9Kc/Bl4fezquUaFGZUMX86ENwX4OYueChGdRGI=; b=G+mlTAwXUNAeLYJ36WBqUOa6RNy4wwHwiL02IEmXpKmxWM+rI3zX8zorKnI0iCmofH 69zmMk718qaaLe8gO1naMAEIAnNlYKrmuH1C5YFNfLUDgMCHJ3JCWasgiAAgI/sEdGZB b6kQ6lnt+K2tTpqT9s6abh3Hzk6jscGlp94y/QTgVjrv6uYiHNvecA1X74Tl5jG9PlRJ IvGuFNJKcwOwpxxQktK4vFYqLPu9MwNP+ngOCV6i0FJxa86zKHmfsh2XwKPYSz0tKcfp nhUYQO4owx0UoTI7FGF1Ivem7C6YGk1QTNXZSvuQT1yBOTtX95WOxTmczf2wKGLIhauV k7VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694431441; x=1695036241; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C8OuW9Kc/Bl4fezquUaFGZUMX86ENwX4OYueChGdRGI=; b=Sxn1Ut1/T+hy66up0Sfv6LKq+Oj9goelZ5McDFQg8UTBA4OQ3ecLTbz4ahMvzy8JGw hyIymOy2P37+D+wW7Pnht7obN7Tto/irShKQZI+nr+scnGc7m8wbub9LrCkzDqJehQ5+ ebscD/EEmrnePzVZTmUZe5n55XM7KA0hUy7OK4XTvkyJJNS8hVecHU5kyfcipjMJD/HE uvHSWcm4gNDbZYyvlL2SeFJ58hjKqZR32NP6fJWJFQJgqosfEQl+FVaf0ul1Oqo5B5SU 1au73q8LurJZQLNVFjfEE4B5tPL0AMKC1pJTfFARZqv7L6Zq21nfoCzYoadHl+mgcaeI kdbg== X-Gm-Message-State: AOJu0YzPmYPBsjV1a6vkp6iiDxAepvWYuDuhyzQgJSQ4ellq82y/s9GR 3DI9cIl4lzxXJ/asgT1lCSAcFvr1PqM= X-Google-Smtp-Source: AGHT+IEm53aH9z6OxtH/I6npDph9yH7BvLZyuJ9ib2+pNIYApogUVArOxNt9CXz3Y8I58fR/1FfdBQ== X-Received: by 2002:a5d:5707:0:b0:317:3da0:7606 with SMTP id a7-20020a5d5707000000b003173da07606mr7133550wrv.4.1694431440677; Mon, 11 Sep 2023 04:24:00 -0700 (PDT) Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id y19-20020a05600c365300b003fef5e76f2csm3498262wmq.0.2023.09.11.04.24.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Sep 2023 04:24:00 -0700 (PDT) From: Simon Tournier To: Ludovic =?utf-8?Q?Court=C3=A8s?= , Josselin Poiret Subject: Digression about Git implementations (was Re: bug#65720: Guile-Git-managed checkouts grow way too much) In-Reply-To: <87pm2s385m.fsf@gnu.org> References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> <87pm2vvibo.fsf@jpoiret.xyz> <87pm2s385m.fsf@gnu.org> Date: Mon, 11 Sep 2023 10:42:35 +0200 Message-ID: <87zg1tje2s.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65720 Cc: 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludo, On Fri, 08 Sep 2023 at 19:08, Ludovic Court=C3=A8s wrote: > Years ago, Andy Wingo sketched a plan for GNU hackers to implement Git > in pure Scheme. That was on April 1st though, so people mistakenly > assumed it was a joke and the project was never carried out. Well, that is a piece of work. :-) Maybe there is an hope with: git-std-lib. Subject: Proposal/Discussion: Turning parts of Git into libraries From: Emily Shaffer To: Git List Date: Fri, 17 Feb 2023 13:12:23 -0800=09=20=20=20=20 https://lore.kernel.org/git/CAJoAoZ=3DCig_kLocxKGax31sU7Xe4=3D=3DBGzC__= Bg2_pr7krNq6MA@mail.gmail.com/ And some patches are starting to float around. https://public-inbox.org/git/20230810163346.274132-1-calvinwan@google.com/ > I digress, but I wonder: is there not even a viable Haskell or OCaml > implementation of Git? It depends on what means =E2=80=9Cviable=E2=80=9D. :-) https://github.com/mirage/ocaml-git https://hackage.haskell.org/package/git Irmin [1] is an OCaml library for building mergeable, branchable distributed data stores =E2=80=93 A Distributed Database Built on the Same Principles as Git. And irmin relies on ocaml-git. 1: https://github.com/mirage/irmin Then there is a pure Go implementation and another using Java. https://git-scm.com/book/en/v2/Appendix-B%3A-Embedding-Git-in-your-Applicat= ions-go-git https://git-scm.com/book/en/v2/Appendix-B%3A-Embedding-Git-in-your-Applicat= ions-JGit I do not know all that are =E2=80=9Cviable=E2=80=9D. Well, I do not know i= f =E2=80=99git gc=E2=80=99 is implemented. And I do not know which plumbing is implemented and which porcelain is available. Last, SWH uses dulwich [2] which is a pure Python implementation of Git. 2: https://www.dulwich.io/ To my knowledge, there is no =E2=80=9Cdulwich gc=E2=80=9D but they implemen= t =E2=80=9Cdulwich fsck=E2=80=9D and =E2=80=9Cdulwich repack=E2=80=9D. Back on 10 Years of Guix or at UNESCO on February =E2=80=93 I do not rememb= er exactly when =E2=80=93 we were discussing about implementation of Git. And= we mentioned an implementation in Rust. Maybe this one: https://github.com/Byron/gitoxide Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 11 10:38:05 2023 Received: (at 65720) by debbugs.gnu.org; 11 Sep 2023 14:38:05 +0000 Received: from localhost ([127.0.0.1]:54341 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfi3B-00071H-0k for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:38:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58936) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfi38-00070l-2v for 65720@debbugs.gnu.org; Mon, 11 Sep 2023 10:38:02 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfi2y-0005Kc-Kx; Mon, 11 Sep 2023 10:37:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=slniUQC9+028PxuFEpuMCgSmgL9gNYctBBnUkHGl+Yw=; b=sf+0dp0sJRNtHMqTh3ih /idbj+n0lEPqG2VR0ik70c6apt0jY0eLCNwLX2h56fGQlv/Xm3uCKeRFpYt8EDeG0TlI0j6PrL/1q lmQEVEaRRbKaiWiV58ZYIhLhidxr1hdjZ8CvN/F+quY+uqmAZkGJDsgHwWY89FQN1AqGhCVgGtugc 1tOuJQoIUR9K+g4e3KhaJxf/IMHGd6oq1+1fDkIFRW2wByB8M6vavu+XKrtPooVQFulJpR/TmuSca vMnCPAnoSlQn4+aQvVRHbLDXKGU2+F4bDfwzHkj2WspJY5F1gXa1bAmpMS7sEMdrGpo2v5dHSNJZH VTeCYnVtU1u0DQ==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Josselin Poiret Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> Date: Mon, 11 Sep 2023 16:37:50 +0200 In-Reply-To: <87msy0ln4m.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Tue, 05 Sep 2023 16:18:17 +0200") Message-ID: <87jzswsrlt.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 Cc: 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > It would also be pretty bad for closure size: > > $ guix size guile-git | tail -1 > total: 106.6 MiB > $ guix size guile-git git-minimal | tail -1 > total: 169.8 MiB > > It=E2=80=99s also not clear concretely how we=E2=80=99d add that dependen= cy. Try > invoking =E2=80=98git=E2=80=99 from $PATH and print a warning if it doesn= =E2=80=99t work? A solution to this particular problem is coming: https://issues.guix.gnu.org/65866 Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 11 10:43:11 2023 Received: (at 65720) by debbugs.gnu.org; 11 Sep 2023 14:43:11 +0000 Received: from localhost ([127.0.0.1]:54353 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfi86-00078r-Rb for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:43:11 -0400 Received: from wolfsden.cz ([37.205.8.62]:49238) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfi82-00078e-Kf for 65720@debbugs.gnu.org; Mon, 11 Sep 2023 10:43:08 -0400 Received: by wolfsden.cz (Postfix, from userid 104) id 55CFE25A352; Mon, 11 Sep 2023 14:43:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=wolfsden.cz; s=mail; t=1694443380; bh=9UzFheRlBpqwhqYLwxRvcCIW6Z1Q4wMYjKudYbhoL0k=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=bTC939dBzFFk6esxnKSOdRbxsN5LTCAxeP+eUlfD6WXcMK1fPf+c/OMD3TJE9JWA0 V8YqtIuX2v7qqB89+jlxymepApZVyeZ7Wmx8KIFblCz3mp0xyBQDZHKu1e8H00TIak OW/PStKUBtzR554J/Q9rLtq6H8yKWbiAovHXG/A4nhLajO9+HQMNcC/e+ylMIrKVdy /UmiT1xTV8gMEw7Exngp0L283yRGtMMtiEmwhWNVKDxFpEo87xKvQ4aLmyyG0QB/f+ BhuIBpNggW9jk28mNB6Kk5Asyuow9HNWHKb+HVVTL3xmcaYFadGIT/quzNvgC0lLZj JiWf/jNTmRn6VDaCRi6ajjahLrCPMQiz7mTsYBDTjPv5Wr8i/5oYrYs+QXBKIBsvqY MT8pkNdbw/lVnM4hvHLiCJE8sCE70/1/clKzTRvn6r0H2s+J+ZSi4iTfhZvWD+r+6R ccpQERSnXMD41vYkc2Y5WLpn//7B7YSkdtw/JzE7kHkr6SN+MThp+kx2iJ1zGfc4lB nuHT3phEsdDNEcneitKWh94rPJRts0e+4GgFEyHaXF5ftnL86Sh0lYNDtPAKbPE5K+ 1BDl/U/kivM0/KiFoK/vF+8MPCgORoiwvA/azIgShwwU9OvMqPdid84h3N5gMojtSp GVKm6eEfYHasfGQwNWdFF92s= X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on wolfsden X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 Received: from localhost (unknown [193.32.127.154]) by wolfsden.cz (Postfix) with ESMTPSA id BB86725B039; Mon, 11 Sep 2023 14:42:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=wolfsden.cz; s=mail; t=1694443379; bh=9UzFheRlBpqwhqYLwxRvcCIW6Z1Q4wMYjKudYbhoL0k=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=qW4/Mjo471X6YMVfJ3cUHTAk8/ewQ6oCyMfjBcToj1Mv2H4UhilOAq1NCf1lSotaT y2uyBO/bSj67hec6z/vW4AKC/LgD2tNhMoF8f9JY279CfZvnk5fYPiyyCthi5/47rG XoamN6Xl8UofI4jdT3viwzZPvVEfMQmftv1RtzAu656kXEKOK9+SGrLOa/DSrV/s6w Eatt+StL0NiiO5ViFmo7gRhgivSan2kHyg5eJElLwigstsapE2ufuo/KclZFsMIwF3 CuRjyo/kBdHwntelqHWRtVF7fVT7EhiRjk5kRHPsOl71/WeuyVIH6w27GenRJ44Vpp F7L7on1u+T3oJX/LiBM4+ngE/FY4V8Vfv8P+P9HMBwTES5gjMso6V8DOEQeRfudaE1 Ty+skOfuJFR6VSOHARKjYgOjFUcdNeHHAr3KzEahVG+HWANkX2SMNLAhD1JFO/f5oI unnqL/o4rEPogGsXbMGQvU6SWRMBDSv4dFO6EesebJB9n/Vqmyg4WneRi8u9SZhZeR q//58iTfvXhyUgMIwOxztlSg7ZRTSvBEZyodhehLcVNDPGDtsJqTYZK9+WifadKXvs TtZz7d4GUFPSBg11ahInlTs5nZfaXzz/P2HvM5UZ+lu6eW7nddAdiH82I9fM0VujUR tknbqMsHpZgDuE9KxmKPTTpo= Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 99c8a95f; Mon, 11 Sep 2023 14:42:59 +0000 (UTC) Date: Mon, 11 Sep 2023 16:42:59 +0200 From: wolf To: Ludovic =?iso-8859-1?Q?Court=E8s?= Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much Message-ID: References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> <87pm2vvibo.fsf@jpoiret.xyz> <87pm2s385m.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="38x2NUBZrcLd/9Sh" Content-Disposition: inline In-Reply-To: <87pm2s385m.fsf@gnu.org> X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 65720 Cc: Josselin Poiret , 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) --38x2NUBZrcLd/9Sh Content-Type: multipart/mixed; boundary="ZmJCXHxmcj67AztS" Content-Disposition: inline --ZmJCXHxmcj67AztS Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2023-09-08 19:08:05 +0200, Ludovic Court=C3=A8s wrote: > Hello! >=20 > Josselin Poiret skribis: >=20 > > Right, although I wouldn't necessarily say that the former doesn't have > > a proper API, but rather that it has a Unix-oriented API. That leads to > > performance issues on e.g. Windows but on Linux I'm not sure there's > > much of a difference. >=20 > [...] >=20 > > We could consider replacing the guile-git dependency with another > > library built directly on top of git-minimal, and have this be a > > dependency of Guix. Not ideal though, and not really scalable either: > > we can't just add every VCS as direct dependencies. >=20 > I cannot imagine a viable implementation of things like =E2=80=98commit-c= losure=E2=80=99 > and =E2=80=98commit-relation=E2=80=99 from (guix git) done by shelling ou= t to =E2=80=98git=E2=80=99. I am sure I must be missing some part of the contract of the function, but = at least the commit-relation seems fairly straightforward: (define (shelling-commit-relation old new) (let ((h-old (oid->string (commit-id old))) (h-new (oid->string (commit-id new)))) (cond ((eq? old new) 'self) ((zero? (git-C %repo "merge-base" "--is-ancestor" h-old h-new= )) 'ancestor) ((zero? (git-C %repo "merge-base" "--is-ancestor" h-new h-old= )) 'descendant) (else 'unrelated)))) I would argue it is even somewhat more readable than the current implementa= tion. > I=E2=80=99m quite confident this would be slow My version is ~2000x faster compared to (guix git): Guix: 1048.620992ms Git: 0.532143ms Again, I am sure I must have miss something, either in the implementation o= r in the measurements, because it is pretty hard to believe there is so much roo= m for improvement. The full script I used is attached to this email. > and brittle. In general git plumbing command are design to have stable CLI interface in = order to be usable in scripting. So I am not sure where the brittleness would co= me =66rom. >=20 > It looks like there=E2=80=99s no option other than carrying the two > implementations. Assuming I made no mistake (hard to believe), it is probably worth explorin= g the feasibility of just shelling out to the git binary some more. >=20 > ~~~ >=20 > Years ago, Andy Wingo sketched a plan for GNU hackers to implement Git > in pure Scheme. That was on April 1st though, so people mistakenly > assumed it was a joke and the project was never carried out. >=20 > I digress, but I wonder: is there not even a viable Haskell or OCaml > implementation of Git? >=20 > Thanks, > Ludo=E2=80=99. > W. --=20 There are only two hard things in Computer Science: cache invalidation, naming things and off-by-one errors. --ZmJCXHxmcj67AztS Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="test.scm" Content-Transfer-Encoding: quoted-printable #!/bin/sh # -*-scheme-*- exec guile -s "$0" "$@" !# (use-modules (git) (guix git)) (define %repo "/tmp/guix-fork") (define h1 "72745172d155e489936f694d6b9013cb76272370") (define h2 "6d60d7ccba5a8e06c17d55a1772fa7f4529b5eff") (define h3 "c3db650680f995f0556d3ddce567cdc1c33e4603") ;;; r has to still be defined when the commit-relation is called. There is= *no* ;;; error, but it always returns 'unrelated. Quite a footgun. (define r (repository-open %repo)) (define c1 (commit-lookup r (string->oid h1))) (define c2 (commit-lookup r (string->oid h2))) (define c3 (commit-lookup r (string->oid h3))) (define (git-C dir . args) (apply system* "git" "-C" dir args)) (define (shelling-commit-relation old new) (let ((h-old (oid->string (commit-id old))) (h-new (oid->string (commit-id new)))) (cond ((eq? old new) 'self) ;; In real code, git-C should probably return #t (for 0), #f (for= 1) ;; or raise (for anything else). ((zero? (git-C %repo "merge-base" "--is-ancestor" h-old h-new)) 'ancestor) ((zero? (git-C %repo "merge-base" "--is-ancestor" h-new h-old)) 'descendant) (else 'unrelated)))) ;;; Make sure it actually works. (let ((tests `((,c1 . ,c1) (,c1 . ,c2) (,c2 . ,c1) (,c1 . ,c3)))) (for-each (=CE=BB (c) (format #t "Guix: ~a\nGit: ~a\n\n" (commit-relation (car c) (cdr c)) (shelling-commit-relation (car c) (cdr c)))) tests)) (define (time proc) (let* ((start (get-internal-run-time)) (_ (proc)) (end (get-internal-run-time))) (exact->inexact (* 1000 (/ (- end start) internal-time-units-per-second= ))))) (format #t "Guix: ~ams\nGit: ~ams\n" (time (=CE=BB () (commit-relation c1 c2))) (time (=CE=BB () (shelling-commit-relation c1 c2)))) --ZmJCXHxmcj67AztS-- --38x2NUBZrcLd/9Sh Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEt4NJs4wUfTYpiGikL7/ufbZ/wakFAmT/J20ACgkQL7/ufbZ/ walA7BAAioswpeyaAYJlo/HjxXOUviMoZ49RJ0vjoWkcdKDJyZCvY1bSaa6E+o38 4mjw+8qT2VH3Su+GKTWgYJ66O6PT2IZh7kybzqPCdIFFXAK3KHNP2cQlweDgl6jG YhktsUBWalhzk06rEy3JXNPqrIinGHmMqm/pIxMQXPcOLN5/d90TMB304YqbjAio J5sCeNNYNhVL0A1jY7rZMefUcHISKX8B3XvsNr2A0AvofGv6OQrftf3OMEX4OeE1 5KFeukwv9FRZ38Cc6+Ob3Jw+Atmz5WrOutTPMXAbp4fxxXHQguG9/fIP3JinAtd1 3ruwT7Q4V5n6pGcz81vMYTR+24Tfbcs4thDqKfIM2uoPOvCh1c6dQ3ap2hI4uvls DlCSviISQkjjCqR30jj2ZhHIHF3kPDl+DnaDCn/LIKBRwEbLDJJ+eW9Bv9JJLG2h 6TCouuRrJzCZ7OpkTg6psZI7mhzwYNdJO2wIkGib8eI2U+/GxFDgWTi8U9HHQFiR Z8/97ph5AdoIObDz0R/hezyvpWOJMYuhI0IhKvBksyx8UYOnpM0lIaSASQt2DqU8 xmRztjNazvoUbTASBg9l4MedSejPcDVn6FFFQ+QpkBORXTMYJ5E572BVxOOmnTbu s2K2nZZMczHKbOgWyJt4rafRzSZeJRY6fr062Cu+TrHs9TO0i70= =UIrN -----END PGP SIGNATURE----- --38x2NUBZrcLd/9Sh-- From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 13 14:10:54 2023 Received: (at 65720) by debbugs.gnu.org; 13 Sep 2023 18:10:54 +0000 Received: from localhost ([127.0.0.1]:35997 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgUKD-0003s5-Tp for submit@debbugs.gnu.org; Wed, 13 Sep 2023 14:10:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55598) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgUKB-0003rr-Fn for 65720@debbugs.gnu.org; Wed, 13 Sep 2023 14:10:52 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qgUJz-0000vz-VB; Wed, 13 Sep 2023 14:10:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=3thjA8phpPJbXWv6yUX3y+eTpXZIAV+/cobGCrMybw8=; b=CrAXPm3gUTQVSoIHo0It dujVEAB9jcGUjfAteAjNhmleILF7r1mN8eI6CcyouDv7K0/Og/6EeZun2KZQhISUNgksfr7syNG+3 27jHO8gy4CpeNwsPfGUmOK/c/F/ryiqRkh0wM3Ki4Me7jUxSISy23BGe+Pz5uwJj7w5ciktokA8OT AmzxJ03Lmawxqn8azP3FGpKaJp6RznDFMljQPr40Mm5gCk+HaC7HasyxqTX8i0S8dL6oNgolUy+zw AI61eCtvrZVy4KM673AK3KG/yqqmAhfsZrxBzbmqjd8ma95nNNOZNHbQGJhOOyLyA5HghFC+d3Fa6 cqGP7FB0plg9hA==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: wolf Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> <87pm2vvibo.fsf@jpoiret.xyz> <87pm2s385m.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Septidi 27 Fructidor an 231 de la =?utf-8?Q?R=C3=A9v?= =?utf-8?Q?olution=2C?= jour de la Verge-d'or X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 13 Sep 2023 20:10:38 +0200 In-Reply-To: (wolf@wolfsden.cz's message of "Mon, 11 Sep 2023 16:42:59 +0200") Message-ID: <874jjylza9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 Cc: Josselin Poiret , 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, wolf skribis: > (define (time proc) > (let* ((start (get-internal-run-time)) > (_ (proc)) > (end (get-internal-run-time))) > (exact->inexact (* 1000 (/ (- end start) internal-time-units-per-seco= nd))))) > > (format #t "Guix: ~ams\nGit: ~ams\n" > (time (=CE=BB () (commit-relation c1 c2))) > (time (=CE=BB () (shelling-commit-relation c1 c2)))) =E2=80=98get-internal-run-time=E2=80=99 returns =E2=80=9Cunits of processor= time=E2=80=9D used by the current process (info "(guile) Time"). When shelling out, the process calls waitpid(2) and does nothing, so naturally its processor time is close to zero. =E2=80=98get-internal-real-time=E2=80=99 should give something closer to el= apsed time. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 13 18:52:33 2023 Received: (at 65720) by debbugs.gnu.org; 13 Sep 2023 22:52:33 +0000 Received: from localhost ([127.0.0.1]:36301 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgYim-0005MA-Sv for submit@debbugs.gnu.org; Wed, 13 Sep 2023 18:52:33 -0400 Received: from mail-wm1-x330.google.com ([2a00:1450:4864:20::330]:41143) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgYih-0005La-UV for 65720@debbugs.gnu.org; Wed, 13 Sep 2023 18:52:29 -0400 Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-3fe4f3b5f25so867035e9.0 for <65720@debbugs.gnu.org>; Wed, 13 Sep 2023 15:52:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694645536; x=1695250336; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=8/Ddynd1Q5u7aJf6LPAQyd1FEGPttudrblI+TDhDRAc=; b=k769VeRGXZWuc9m7cbX/X9TmncOtapPVnQBIXTji6+Eb0HkMFjPiz11rqvEhKhHYlP VjGqSc1K3ynkNwIDQS9svy/g/g0zCQ1m6rdZMZ5ssT59wLJr17ICLKPXT7jVSQZnoy1a TYqIF58/vMRG7ndn+YN9OAf0TagXvdm/p9CgkiCc+6qiKl4LBqGFV0DoaHyBPA8lBGmP 2kbugskd6wAuEYvTfRFzGLqRKmkHz6Ry72p5ljAt7pCHmtW0/Jc4Jwj4Iio7SintPjjJ zDgdzxHob9erUQYlzWQunD0jCKbrBV2GVVR2BwxobYB7gd0W7K1ltaPFvThSqQtBzfkO 6fDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694645536; x=1695250336; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8/Ddynd1Q5u7aJf6LPAQyd1FEGPttudrblI+TDhDRAc=; b=ocuDaebx2iIhL2hlV4huM0CqHGUNJh0JELdOs0VqHfpRQ2GfnSu+ecDGKE3rOoyMIB 4YG/8/CecX9GLna1XdeIZ9dLGHbdokwnDEKXO+5qerE8VdQta57zhbRv7YZOYR0j+W4u Sit8/+//FJcc9iQoXPY2QsgtaKVIkSBPA5RJAFY7FvQGQ7Et96FDyIgFfC0DufI284o9 U+SBNLVbcBnuOp0gXRqECkY2Ffnkidf8IEIX8svGeN70jg5wrsPNqJVl5TiTWVNrJCuh dEaZle/kNXu/pZYXSv/i8qV0HiZXXYFjeNHJAK1+XE8IP4jMqxYLB1uwT/4vde1xabMN C+9w== X-Gm-Message-State: AOJu0YxK/RLs4Mc7h/LTfVlzVUXaNvpU62lyl+UuwsWMKHAgO5y3bZde Ur7R5TDzWxiekMdMJi7ocL4def+UYAo= X-Google-Smtp-Source: AGHT+IEQyzrX4pqn8PTZYG+0dGN0WVMtzuH+hEQ610+sX5ZmCz9heI6iwhfQTfSb6CRLI5ZsL0j7wA== X-Received: by 2002:a05:600c:34ce:b0:401:bb6e:453a with SMTP id d14-20020a05600c34ce00b00401bb6e453amr3280276wmq.0.1694645536274; Wed, 13 Sep 2023 15:52:16 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id z25-20020a1c4c19000000b003fe29f6b61bsm291851wmf.46.2023.09.13.15.52.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Sep 2023 15:52:15 -0700 (PDT) From: Simon Tournier To: Ludovic =?utf-8?Q?Court=C3=A8s?= , wolf Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much In-Reply-To: <874jjylza9.fsf@gnu.org> References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> <87pm2vvibo.fsf@jpoiret.xyz> <87pm2s385m.fsf@gnu.org> <874jjylza9.fsf@gnu.org> Date: Thu, 14 Sep 2023 00:36:17 +0200 Message-ID: <86o7i5wvj2.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65720 Cc: Josselin Poiret , 65720@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludo, On Wed, 13 Sep 2023 at 20:10, Ludovic Court=C3=A8s wrote: > =E2=80=98get-internal-run-time=E2=80=99 returns =E2=80=9Cunits of process= or time=E2=80=9D used by the > current process (info "(guile) Time"). When shelling out, the process > calls waitpid(2) and does nothing, so naturally its processor time is > close to zero. > > =E2=80=98get-internal-real-time=E2=80=99 should give something closer to = elapsed time. Well, let avoid to mix unrelated discussion. :-) For discussing that specific part, I reported on guix-devel my timing using ,time. comparing commit-relation using Scheme+libgit2 vs shellout plumbing= Git Simon Tournier Tue, 12 Sep 2023 00:48:30 +0200 id:865y4gz5q9.fsf@gmail.com https://lists.gnu.org/archive/html/guix-devel/2023-09 https://yhetil.org/guix/865y4gz5q9.fsf@gmail.com The result is still significantly less and discussion is welcome overthere. :-) Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 18 18:35:48 2023 Received: (at 65720) by debbugs.gnu.org; 18 Sep 2023 22:35:48 +0000 Received: from localhost ([127.0.0.1]:54873 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qiMqJ-0004FE-KV for submit@debbugs.gnu.org; Mon, 18 Sep 2023 18:35:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52356) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qiMqH-0004Ez-6S for 65720@debbugs.gnu.org; Mon, 18 Sep 2023 18:35:46 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qiMq3-0007GS-Aj for 65720@debbugs.gnu.org; Mon, 18 Sep 2023 18:35:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=VnCdo5ERqnjAzkB8ATdJaeLx+YXVmKXugHDB0d3lVWU=; b=QVjPC0vXxfleSW5DPSJ/ bBSDF5kCSrl4m08GiI+NBRGYpE+fjV3E1lBB9/7uWQUm5j/9hzw6+hA1tBoVre8fgAdLAVZhXqi1d +o2S9nd40TLAzVrAsYWIc12t51Cu7I+vuZOa0yqWagxtzZ/J4GbznfE5waXrbAgXuqXRg4TJf2PR3 d3WIQU5jfQ3h7ILznZqg8G/zUkf24ZmFd4prTDjjNDRJWWLyU9DnR3SpiDUGnp4/zT+9ngft3l4rl B2KAsW5eEUYZl+LevHgLXeWIvtnR6Zhmi1pJDw15iHlM7oY/mPlZyOPVVgSzRvvdZrGr7Ifek2hHN FUxmYlI5Mx/D2g==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 65720@debbugs.gnu.org Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much References: <87bkejc7go.fsf@inria.fr> Date: Tue, 19 Sep 2023 00:35:28 +0200 In-Reply-To: <87bkejc7go.fsf@inria.fr> ("Ludovic =?utf-8?Q?Court=C3=A8s=22?= =?utf-8?Q?'s?= message of "Sun, 03 Sep 2023 22:44:39 +0200") Message-ID: <87jzsnf6tr.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > As reported by Tobias on IRC (in the context of =E2=80=98hpcguix-web=E2= =80=99), > checkouts managed by Guile-Git appear to grow beyond reason. As an > example, here=E2=80=99s the same =E2=80=98.git=E2=80=99 managed with Guil= e-Git and with Git: > > $ du -hs ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27= shj7apsnalwq > 6.7G /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlay= wmisyr27shj7apsnalwq > $ du -hs .git > 517M .git More data=E2=80=A6 The biggest file in that repo is a pack that was created when that repo was first cloned (Aug. 2021): --8<---------------cut here---------------start------------->8--- $ du /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisy= r27shj7apsnalwq/.git/objects/pack/* |sort -k1 -n| tail -3 44272 /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmis= yr27shj7apsnalwq/.git/objects/pack/pack-3c2f1857501b01c321bc67ba1f30704deb9= e18e9.pack 47272 /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmis= yr27shj7apsnalwq/.git/objects/pack/pack-30d5b35ad14a8398464e49e224811b162f6= 73d66.pack 191492 /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmi= syr27shj7apsnalwq/.git/objects/pack/pack-d39507858782209d1ad87e389e4dffd4b6= ff7ea2.pack $ ls -l /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywm= isyr27shj7apsnalwq/.git/objects/pack/pack-d39507858782209d1ad87e389e4dffd4b= 6ff7ea2.pack -r--r--r-- 1 ludo users 196079671 Aug 9 2021 /home/ludo/.cache/guix/check= outs/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack= /pack-d39507858782209d1ad87e389e4dffd4b6ff7ea2.pack $ ls -ld /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlayw= misyr27shj7apsnalwq/.git/config -rw-r--r-- 1 ludo users 266 Aug 9 2021 /home/ludo/.cache/guix/checkouts/p= jmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/config --8<---------------cut here---------------end--------------->8--- The pack starts with things from Aug. 2021: --8<---------------cut here---------------start------------->8--- $ git show-index < pack-d39507858782209d1ad87e389e4dffd4b6ff7ea2.idx|sort -= k1 -n|head -3 12 30289f4d4638452520f52c1a36240220d0d940ff (852d8cb3) 927 d7ffc535c52f49177a8e5553569cdb1e321b5bc6 (2007c5d0) 1800 0a379de3249d5e9ff66fb404f7e5aa8ce2cb3d24 (b1e69aa4) $ git show 30289f4d4638452520f52c1a36240220d0d940ff commit 30289f4d4638452520f52c1a36240220d0d940ff Author: Milkey Mouse Date: Sun Aug 8 22:15:40 2021 -0700 [=E2=80=A6] --8<---------------cut here---------------end--------------->8--- =E2=80=A6 and at the bottom (large offsets) it contains very old blogs from= the Nix repo that somehow made it here. I figured we still had a =E2=80=98nix=E2=80=99 branch from the early days, = that contains the history of Nix. I=E2=80=99ve now removed it, which helps a bit: --8<---------------cut here---------------start------------->8--- scheme@(guile-user)> ,use(git) scheme@(guile-user)> ,t (clone "https://git.savannah.gnu.org/git/guix.git" = "/tmp/guix") $5 =3D # ;; 600.534529s real time, 435.260926s run time. 0.000000s spent in GC. scheme@(guile-user)> ,t (clone "https://git.savannah.gnu.org/git/guix.git" = "/tmp/guix-after-removing-nix-branch") $6 =3D # ;; 420.321511s real time, 398.772963s run time. 0.000000s spent in GC. --8<---------------cut here---------------end--------------->8--- =E2=80=A6 and more importantly: --8<---------------cut here---------------start------------->8--- $ du -hs /tmp/guix/.git 373M /tmp/guix/.git $ du -hs /tmp/guix-after-removing-nix-branch/.git 362M /tmp/guix-after-removing-nix-branch/.git --8<---------------cut here---------------end--------------->8--- Anyway, what seems to happen is that every pull (every call to =E2=80=98remote-fetch=E2=80=99) creates a new pack (see =E2=80=98git_fetch_= download_pack=E2=80=99 in libgit2), which becomes inefficient in the long run (lots of small poorly-compressed packs). That=E2=80=99s at least one possible explanation. To be continued=E2=80=A6 Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 19 04:09:31 2023 Received: (at 65720) by debbugs.gnu.org; 19 Sep 2023 08:09:31 +0000 Received: from localhost ([127.0.0.1]:55311 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qiVnW-0003Lx-Nk for submit@debbugs.gnu.org; Tue, 19 Sep 2023 04:09:31 -0400 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]:37893) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qiVnT-0003Lb-R9 for 65720@debbugs.gnu.org; Tue, 19 Sep 2023 04:09:29 -0400 Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-40472fef044so10907625e9.0 for <65720@debbugs.gnu.org>; Tue, 19 Sep 2023 01:09:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695110953; x=1695715753; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=WHYfevDu/6ukObYLZtp3ZBrsavJCnLK/TRMgi/UuzXo=; b=mLxysicECnZwPFa1GRYgdglf3esMI1DI9XXu6Dq9T9X6UI1NEQ+mIGnKeCHBOZvjuk J/4hjso0/OgCnqYFL2+z6e9c9DkRDwPy6FTZ+qgj3goKVb0hNohiH5X6pcrU6cS5K9u+ gdDSUtaABf07uAxg1aawbIf+qfTdVZwbYz5YHHklhEJELSgWReYwusWql+gEMysTLtWd LG1Du0hKHAwPqmtsAs15wV+UpDHUMAsVuWBKLX/ghbxDjnfcROI7sfhTt/m7caBf6OB+ feqMttKWczN3OyCj3w8tnUi/OPzs0L/WRAc3F0NW9BaVGmiGAz25QRb+GcgwOQN0m1KR wDnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695110953; x=1695715753; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:to:from:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=WHYfevDu/6ukObYLZtp3ZBrsavJCnLK/TRMgi/UuzXo=; b=PSKedUlxufvFagZ/tJG7b2igEaIb5OcVU714WlIa446dCo2Lqy/H5/BMlekrSERYN7 Jc4nhW1R/zdKvgDPicJviwUAxkp2HuJglf1GA04NMzGpIGWqLHKZm96gfZ2b1Mf8R0sV 0O4DPSZ4v+PhHZIKxoDIGsAOG+8bc9Qcde/3h1KdplMlOixXsCH7+6PAX63WADTh6eWA 6K1nzqMZimY6Eil6OiM8/KyJe95M2k5FS8nS/Y8NAxoGLnd52niOBLsyD6h6cUTagcbS abh5iJyaIDuTiDnCmc3SNmrjbDEEZf/oJw/tC5wwWtwNm5VuMvvVq4UYfcOe84ogBEBg IxEA== X-Gm-Message-State: AOJu0YwdX6I3/Vyuef1mX9K4a2dfiFukzjI0irsxFUxmj2n+1bjyf1Gd NyU//xOc6L+9e+Wr1BOKrIpLC2sk+jE= X-Google-Smtp-Source: AGHT+IHZINXRqmB1dHsH7sixJCNHfhC70+ZTL7pErGaW43Ao7p1NSxFlmFWJ6z6XuUC2V45N38npcw== X-Received: by 2002:adf:ec4f:0:b0:31f:a717:f1cc with SMTP id w15-20020adfec4f000000b0031fa717f1ccmr8798964wrn.5.1695110953126; Tue, 19 Sep 2023 01:09:13 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id c13-20020a5d4ccd000000b0030fd03e3d25sm14846313wrt.75.2023.09.19.01.09.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 01:09:12 -0700 (PDT) From: Simon Tournier To: Ludovic =?utf-8?Q?Court=C3=A8s?= , 65720@debbugs.gnu.org Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much In-Reply-To: <87jzsnf6tr.fsf@gnu.org> References: <87bkejc7go.fsf@inria.fr> <87jzsnf6tr.fsf@gnu.org> Date: Tue, 19 Sep 2023 09:19:44 +0200 Message-ID: <86wmwmlje7.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65720 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludo. On Tue, 19 Sep 2023 at 00:35, Ludovic Court=C3=A8s wrote: > --8<---------------cut here---------------start------------->8--- > scheme@(guile-user)> ,use(git) > scheme@(guile-user)> ,t (clone "https://git.savannah.gnu.org/git/guix.git= " "/tmp/guix") > $5 =3D # > ;; 600.534529s real time, 435.260926s run time. 0.000000s spent in GC. > scheme@(guile-user)> ,t (clone "https://git.savannah.gnu.org/git/guix.git= " "/tmp/guix-after-removing-nix-branch") > $6 =3D # > ;; 420.321511s real time, 398.772963s run time. 0.000000s spent in GC. > --8<---------------cut here---------------end--------------->8--- [...] > --8<---------------cut here---------------start------------->8--- > $ du -hs /tmp/guix/.git > 373M /tmp/guix/.git > $ du -hs /tmp/guix-after-removing-nix-branch/.git > 362M /tmp/guix-after-removing-nix-branch/.git > --8<---------------cut here---------------end--------------->8--- Just to also point [1] that using shallow clone and restrict to the oldest reachable commit by the time-machine, it saves 25% of bits to download, and similarly on disk. --8<---------------cut here---------------start------------->8--- scheme@(guix-user)> ,t (clone "https://git.savannah.gnu.org/git/guix.git" "= /tmp/guix-guile") $1 =3D # ;; 383.186818s real time, 278.060733s run time. 0.000000s spent in GC. $ time git clone https://git.savannah.gnu.org/git/guix.git guix-full Receiving objects: 100% (693699/693699), 342.14 MiB | 2.87 MiB/s, done. real 2m40,830s user 3m4,683s sys 0m8,189s $ time git clone --shallow-since=3D2019-04-30 https://git.savannah.gnu.org/= git/guix.git guix-oldest Receiving objects: 100% (428646/428646), 259.41 MiB | 3.87 MiB/s, done. real 1m45,604s user 2m32,370s sys 0m5,916s $ du -sh guix-*/.git 362M guix-full/.git 362M guix-guile/.git 272M guix-oldest/.git --8<---------------cut here---------------end--------------->8--- Cheers, simon 1: Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in bui= lder for Git checkouts) Simon Tournier Mon, 11 Sep 2023 19:52:34 +0200 id:871qf4ha1p.fsf@gmail.com https://lists.gnu.org/archive/html/guix-devel/2023-09 https://yhetil.org/guix/871qf4ha1p.fsf@gmail.com From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 20 12:16:09 2023 Received: (at 65720) by debbugs.gnu.org; 20 Oct 2023 16:16:10 +0000 Received: from localhost ([127.0.0.1]:41301 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtsAT-0004fY-BD for submit@debbugs.gnu.org; Fri, 20 Oct 2023 12:16:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37666) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtsAP-0004eq-NV for 65720@debbugs.gnu.org; Fri, 20 Oct 2023 12:16:07 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qts9r-0004Rf-PD; Fri, 20 Oct 2023 12:15:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=guykA3Jje4B/fdh8d1YKTVE8F+9iXXIl3FUKarfygvY=; b=jF0iDL/GbSiYleynVOTG /8r/H3EkXts9gzZHBgd9R8nNRQDQE9k6G5S40965qhyaaKSTNmvZmEP05W378rPNL287CkVM+Ibc7 7E32AmsdJM3Am6miqMoPX9YkU5o+Q5z+orHS8Uflfk+Q6GcnHJFgV54Bg6XcAw2I1w6n8BhwLD8XH Xc6+syD3NXx+2uTgwx7sgIiSqx8Q8TyYiO+e+VCDOrgg9PMq1m8eYZ6M32W5h/fNQYw3CoJwQsjTq N7T/JL+bxKLs4dAkgwLr8izbta894kjMEC5GTNjZ2vBkHDJJ991iz+BbSwyZrUfYucrv//2aKtBun iam1wsZbcvK/hA==; From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: guix-patches@gnu.org Subject: [PATCH] =?UTF-8?q?git:=20Shell=20out=20to=20=E2=80=98git=20gc?= =?UTF-8?q?=E2=80=99=20when=20necessary.?= Date: Fri, 20 Oct 2023 18:15:12 +0200 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: <87jzswsrlt.fsf@gnu.org> References: <87jzswsrlt.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Debbugs-Cc: Christopher Baines , Josselin Poiret , Ludovic Courtès , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= , 65720@debbugs.gnu.org, Josselin Poiret , Simon Tournier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Fixes . This fixes a bug whereby libgit2-managed checkouts would keep growing as we fetch. * guix/git.scm (packs-in-git-repository, maybe-run-git-gc): New procedures. (update-cached-checkout): Use it. --- guix/git.scm | 39 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 36 insertions(+), 3 deletions(-) Hi! This is a radical fix/workaround for the unbounded Git checkout growth problem, shelling out to ‘git gc’ when it’s likely needed (“too many” pack files around). I thought we might be able to implement a ‘git gc’ approximation using the libgit2 “packbuilder” interface, but I haven’t got around to doing it: . Once again, shelling out is not my favorite option, but it’s a bug we should fix sooner rather than later, hence this compromise. Thoughts? Ludo’. diff --git a/guix/git.scm b/guix/git.scm index b7182305cf..d704b62333 100644 --- a/guix/git.scm +++ b/guix/git.scm @@ -1,6 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2017, 2020 Mathieu Othacehe -;;; Copyright © 2018-2022 Ludovic Courtès +;;; Copyright © 2018-2023 Ludovic Courtès ;;; Copyright © 2021 Kyle Meyer ;;; Copyright © 2021 Marius Bakke ;;; Copyright © 2022 Maxime Devos @@ -29,15 +29,16 @@ (define-module (guix git) #:use-module (guix cache) #:use-module (gcrypt hash) #:use-module ((guix build utils) - #:select (mkdir-p delete-file-recursively)) + #:select (mkdir-p delete-file-recursively invoke/quiet)) #:use-module (guix store) #:use-module (guix utils) #:use-module (guix records) #:use-module (guix gexp) #:autoload (guix git-download) (git-reference-url git-reference-commit git-reference-recursive?) + #:autoload (guix config) (%git) #:use-module (guix sets) - #:use-module ((guix diagnostics) #:select (leave warning)) + #:use-module ((guix diagnostics) #:select (leave warning info)) #:use-module (guix progress) #:autoload (guix swh) (swh-download commit-id?) #:use-module (rnrs bytevectors) @@ -428,6 +429,35 @@ (define (delete-checkout directory) (rename-file directory trashed) (delete-file-recursively trashed))) +(define (packs-in-git-repository directory) + "Return the number of pack files under DIRECTORY, a Git checkout." + (catch 'system-error + (lambda () + (let ((directory (opendir (in-vicinity directory ".git/objects/pack")))) + (let loop ((count 0)) + (match (readdir directory) + ((? eof-object?) + (closedir directory) + count) + (str + (loop (if (string-suffix? ".pack" str) + (+ 1 count) + count))))))) + (const 0))) + +(define (maybe-run-git-gc directory) + "Run 'git gc' in DIRECTORY if needed." + ;; XXX: As of libgit2 1.3.x (used by Guile-Git), there's no support for GC. + ;; Each time a checkout is pulled, a new pack is created, which eventually + ;; takes up a lot of space (lots of small, poorly-compressed packs). As a + ;; workaround, shell out to 'git gc' when the number of packs in a + ;; repository has become "too large", potentially wasting a lot of space. + ;; See . + (when (> (packs-in-git-repository directory) 25) + (info (G_ "compressing cached Git repository at '~a'...~%") + directory) + (invoke/quiet %git "-C" directory "gc"))) + (define* (update-cached-checkout url #:key (ref '()) @@ -515,6 +545,9 @@ (define* (update-cached-checkout url seconds seconds nanoseconds nanoseconds)))) + ;; Run 'git gc' if needed. + (maybe-run-git-gc cache-directory) + ;; When CACHE-DIRECTORY is a sub-directory of the default cache ;; directory, remove expired checkouts that are next to it. (let ((parent (dirname cache-directory))) base-commit: 6b0a32196982a0a2f4dbb59d35e55833a5545ac6 -- 2.41.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 23 08:18:22 2023 Received: (at submit) by debbugs.gnu.org; 23 Oct 2023 12:18:22 +0000 Received: from localhost ([127.0.0.1]:49288 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qutsz-0006mw-Hz for submit@debbugs.gnu.org; Mon, 23 Oct 2023 08:18:21 -0400 Received: from lists.gnu.org ([2001:470:142::17]:39518) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qutsu-0006lG-QD for submit@debbugs.gnu.org; Mon, 23 Oct 2023 08:18:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qutsM-0001qx-4A for guix-patches@gnu.org; Mon, 23 Oct 2023 08:17:42 -0400 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qutsK-0005KK-IS; Mon, 23 Oct 2023 08:17:41 -0400 Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-408cd9660b8so4432195e9.1; Mon, 23 Oct 2023 05:17:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698063458; x=1698668258; darn=gnu.org; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=Xt1z37opOQR08iqozxo1uMwugTW3bMHxKjeX7VAYwkI=; b=OGhQat8PKDm02pINrSYQnkhw+YfxH5yTqyTdXDgxKVhQAEYAgN8lRXfcsKWUrH265x xc1ZYHKzQcJKwvi5SuCGss2N8tvRN3bqDuAviASlGeRGy/k4DPG718x9Lv4ypBijD+MJ UPbqcQudTlJMvoc1YQLBYArKlPLOxyQ/6fSjbJkPMvwnXgllI/2qwU5lSOPIboDZdwHo eHKcSJna95CR8wSMewy42aAsZmxr7Tp3L1h+8/VzmldZTjZcSZ4jBjehnA9rhq/1t9gz ygbm9D4K1nAZYZlJNguVV0rTOnngV9dy/CCJTsmZbCD7vR2Q1puY1pVZBBi5b9pK1URw 4lIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698063458; x=1698668258; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Xt1z37opOQR08iqozxo1uMwugTW3bMHxKjeX7VAYwkI=; b=ARxaqgYekHcSLUNnZcy1hteSNrwUk791pRkxj6UQicNe6dHodCNgb1qFzskimVwWnx Mdqqv/VNvrfSE3uR9yL2z7GnF25gd/bxrONAUXBOBp3bcopRi05iOFJKMSA6mkelaTp0 Zs2xaUF1EBk/pXk6UFykSOu8jUFE4lNndiPJcnCUQSMu74GFnU9/BK/ABTyQKsWeVb4R J59cp+BfUASnWyiFMWAF9Ehpvw98i8ZSNwogaJGmp74MHKZKk84gXxSycfmd1FtNup/T ArJm9mhTyqyet6nwuaRHKnPJOVY6Bgc/YGo7oYRV6UCB98S+flniauBuVs+EV/yY9own AtEw== X-Gm-Message-State: AOJu0YyAskd4/twampaLSsLqgBkbB6ZU5NL3GqYp8g7z66etcFEfZEs8 6hhRQJG7wTcINviffuNXP90= X-Google-Smtp-Source: AGHT+IEUzcGeauVNAvtPZyCJG5W/D/IQc51htixeCW/va1rc4v1+V80hfJuzOx11Em45vywMXPcnmA== X-Received: by 2002:a05:600c:3b23:b0:401:c07f:72bd with SMTP id m35-20020a05600c3b2300b00401c07f72bdmr6846951wms.4.1698063457671; Mon, 23 Oct 2023 05:17:37 -0700 (PDT) Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id g17-20020a5d5551000000b0032cc35c2ef7sm7674108wrw.29.2023.10.23.05.17.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 05:17:37 -0700 (PDT) From: Simon Tournier To: Ludovic =?utf-8?Q?Court=C3=A8s?= , guix-patches@gnu.org Subject: Re: bug#65720: [PATCH] git: Shell out to =?utf-8?B?4oCYZ2l0IGdj?= =?utf-8?B?4oCZ?= when necessary. In-Reply-To: References: <87jzswsrlt.fsf@gnu.org> Date: Mon, 23 Oct 2023 12:08:07 +0200 Message-ID: <87il6xlkhk.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::32b; envelope-from=zimon.toutoune@gmail.com; helo=mail-wm1-x32b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 4.0 (++++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Ludo, On Fri, 20 Oct 2023 at 18:15, Ludovic Courtès wrote: > * guix/git.scm (packs-in-git-repository, maybe-run-git-gc): New > procedures. > (update-cached-checkout): Use it. > --- > guix/git.scm | 39 ++++++++++++++++++++++++++++++++++++--- > 1 file changed, [...] Content analysis details: (4.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.0 MANY_TO_CC Sent to 10+ recipients 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (zimon.toutoune[at]gmail.com) -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) X-Debbugs-Envelope-To: submit Cc: Josselin Poiret , 65720@debbugs.gnu.org, Mathieu Othacehe , Ludovic =?utf-8?Q?Court=C3=A8s?= , Tobias Geerinckx-Rice , Ricardo Wurmus , Christopher Baines X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi Ludo, On Fri, 20 Oct 2023 at 18:15, Ludovic Court=C3=A8s wrote: > * guix/git.scm (packs-in-git-repository, maybe-run-git-gc): New > procedures. > (update-cached-checkout): Use it. > --- > guix/git.scm | 39 ++++++++++++++++++++++++++++++++++++--- > 1 file changed, 36 insertions(+), 3 deletions(-) LGTM. Just two colors for the bikeshed. :-) > + (when (> (packs-in-git-repository directory) 25) Why 25? And not 10 or 50 or 100? > (define* (update-cached-checkout url > #:key > (ref '()) > @@ -515,6 +545,9 @@ (define* (update-cached-checkout url > seconds seconds > nanoseconds nanoseconds)))) >=20=20 > + ;; Run 'git gc' if needed. > + (maybe-run-git-gc cache-directory) Why not trigger it by =E2=80=9Cguix gc=E2=80=9D? Well, I expect =E2=80=9Cguix gc=E2=80=9D to take some time and I choose whe= n. However, I want =E2=80=9Cguix pull=E2=80=9D or =E2=80=9Cguix time-machine=E2=80=9D t= o be as fast as possible and here some extra time is added, and I cannot control exactly when. Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 23 18:28:27 2023 Received: (at submit) by debbugs.gnu.org; 23 Oct 2023 22:28:27 +0000 Received: from localhost ([127.0.0.1]:52477 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qv3PP-00018q-B0 for submit@debbugs.gnu.org; Mon, 23 Oct 2023 18:28:27 -0400 Received: from lists.gnu.org ([2001:470:142::17]:52834) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qv3PN-00018a-6R for submit@debbugs.gnu.org; Mon, 23 Oct 2023 18:28:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv3Oo-0005JW-EX for guix-patches@gnu.org; Mon, 23 Oct 2023 18:27:50 -0400 Received: from tobias.gr ([80.241.217.52]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qv3Om-0005jI-J0; Mon, 23 Oct 2023 18:27:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=2018; bh=JKRSkzmIbi5Bp 0jvsU+P4SibPxZlEzDUzjqTJHcbKi8=; h=references:in-reply-to:subject:cc: to:from:date; d=tobias.gr; b=EsbQ+bTiXmD7L2+b+LY0EcMuvBjwps+68kFE/Ho33 WNr16DQIGR36DdnTuT6vZo6gDG5vTVkYIhBV4S8bnURqb0OdmOsnQNh8WQAPHvf4lk0pKQ 58diYe1ZDsYhlDBScU9NTUBvDLAhQjaXZNFLwL0gk2fdCp/775auM2jE3mEzWW1pXCYH6c sFEQcxiBbS1fKLBaQVa23/IzKD0fp5/nR9e/6vBa3aCrH4xcOsCJ6W1ll30M6ahu8isnfD lDCQpsi/8EoNZDW/73AaAsUIgVqjaaZ+9xjbxjS1jsEExySBQdPEL7XUxplAhF52Mj4cyH WeETERqxKVKUfQw8iehWg== Received: by submission.tobias.gr (OpenSMTPD) with ESMTPSA id 796a07ec (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 23 Oct 2023 22:27:42 +0000 (UTC) Date: Mon, 23 Oct 2023 22:27:39 +0000 From: Tobias Geerinckx-Rice To: Simon Tournier , =?ISO-8859-1?Q?Ludovic_Court=E8s?= , guix-patches@gnu.org Subject: =?UTF-8?Q?Re=3A_bug=2365720=3A_=5BPATCH=5D_git=3A_Shel?= =?UTF-8?Q?l_out_to_=E2=80=98git_gc=E2=80=99_when_necessary=2E?= In-Reply-To: <87il6xlkhk.fsf@gmail.com> References: <87jzswsrlt.fsf@gnu.org> <87il6xlkhk.fsf@gmail.com> Message-ID: <8A262178-2BFF-41EE-BEF2-5DC3270EF9C5@tobias.gr> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Autocrypt: addr=me@tobias.gr; prefer-encrypt=mutual; keydata= mQINBFVks2ABEACjGBPhWf/qx0L9OhEIrAFTimo5dHa1FLy0AHaHvxmwYSIdJmERYGiNle1rcOvw cFRtu8KJUsrs27Vgoso3qHJpghVitUUf0v3ZuXQT9kfuQLz1Y8pyMzHwVFMLiJVj4Z3y7CJk+xyZ cpSAMbyPINbFVEhsK+z+8ojVGuaiucZkib6b67ySG6Pp1bon8xVvosj71ZRjfXh1t4X8laWO7fQq itT9lmc6DxbE/4vIhR+Vb2MblaA+DyHoNHGGao89h4CO99lfzWzsux41DnEG9d317sJRQTig6Wja EKHXZRA9FbfogD4SDa2uQYCpTJpsVjAyZyu2fuJ+t0zJJ+Ai9qDY87P6hOyd+/n8Eh2Y4TbxJiDo XUT72XY/RfPH1qrMIP3EI/NNL4LQeGG1n+625k3OVWcRVXG2vRrB6qurLmGkLEmjXWCFD9cCRGfH LeajLm9sM+t/nZPZ3btetcmK9tM2EwivyLUNhrTk73UUnI4CSAzdO2cISqo9zSMtFgj2alqd2fOR s7CKfEn+5PquruDbp/Ej7dOOrjgWSCXLDDYXRrtaKrLz/dhqq5ftFYi9tUTTQecFotM08fPtu+Kw JMP2ySHCkUqp0GvrUCeSRPAJZsmJrd535y+LlRhnqb0mbG4dgMa8A6xhkFYugnqldy/q7kX1EmRI 686N7bA6fh1MCQARAQABtCRUb2JpYXMgR2VlcmluY2t4LVJpY2UgPG1lQHRvYmlhcy5ncj6JAlIE EwEKADwCHgECF4ACGQECGwMWIQT1vFU0w28Ah7OdNu8cncT+udt8SwUCXpe0rAYLCQgHCgQFFQoJ CAsFFgMCAQAACgkQHJ3E/rnbfEu5IhAAk+0BW/twLmx1xMmeXn+I7Ne6SG3++0TRBduEaGWV3n59 lX6XPZUQdAPpS4uy0H+c90Owkw+aWUEwfyOWphrxZRtR2cCOP/3Pxj3Vgtz5RkY4u27lMj15jqa/ p7l2l256ZKJOegr9TvOWtkhMp5lxeVHT6f/44Kv/r/8mMCgSnLXYrEWPE462xI+mIJOanHLJb6No f2xLRCvXoLLp7Yejjv1dwOO71R9PMRhtNy46pZM1ylQ++UTkeSocJw4aNtiu0DHOkX9AlNBkutIx x07RpO+MqJKlzzLeQiC/fE5+dR2itRONopwXAqN3MuT7MonQo5XifBn+VK8i9xZWTXZDkWItWtCC 8oIj4zwxwFWiTmMwwSbI3Wdd/11Zw3CLc4Gd0M6NVgvAnuErQXSgr4lrWhZcncvi4L6EJTc9AUSa 8UWPF+S9t+CHTukpJmcYnsccMkOBhT7OZlmWBsylrYK/JTRWqgWSHWdSKmOuLK+MGDneOZEHkEcf jeXRWvmG7MSU5tE/p7NDLIg9vkvhQV9b0q4OtY65uNWbRe2QRJaYMDcYUAeSZzivRa8VaoVen6tb FvH44zpCxubn23ABl9YIzwvJC++r+H2qLdLpy0cfITiZadZ74Ae0aosNw7XARS6OY+A03BfXyPiI 2oW0jf/PdH9sh2mQrQxIQJ5cZz6Z3X20MFRvYmlhcyBHZWVyaW5ja3gtUmljZSAodG9iaWFzLmdy KSA8bWVAdG9iaWFzLmdyPokCNgQwAQoAIBYhBPW8VTTDbwCHs5027xydxP6523xLBQJZUuJkAh0g AAoJEBydxP6523xLwEoP/iaVWdICefmaZ+bkdUg6hwTKxSPlYdLoaCxYI5V2NnZYXG0dy47E7WJP 4EmTDldcne4UG5NyCgXPSMZDNMqcddEMNbmlssTcLKyu6R/VvTcuL3pEhNoLzm9t2IHR/YGAHJIc pyfmJ3amGSlsbo6s6Vv+mbrJTRPIDpLUst/PDoGYihcSKTxoIw8RtNAy+FAliIdJ8ZuPA3BTxQei sQzgWJo68UInvnFp4juq0zupNDqKddemI8kZCRK/fFPBDD71nqiSJjx2tvbjkTiPU9gmTp7Nulgj ZATZIt0pUhn87w15ItzrsiEjTYCMTmHY5wnvyLarFKZLzQjgUJfsbv4WRJRnX0/EEJWMvMtSdtXL fjwTLrSGAr4vh13PMfE6uVkGkeCV/8BfGJqRE4zkI+QOVILw6VSAP6sdtGSGb52bcCuPj7p9zGIB IuKFudT+DzUb5uV2jbahSjJ8X4Ak/ruG2Hv3iICZawDpHV1JDNgY910KNi5T6gFrjFA2G6aiXDCO SlToLPnPBYH85UVOsp2VykE9AREKS7a7o76cibjwBNN+4xJqJRxhs9YH4JKzE3w/ptVnx49QFBGc pjPkzMTs7DtanD8iqL4J6afnycv9KqIlNJONaMHvzxeunV7QCsoF6DdusZJKFMexiDBaOFUmGFFS XfzK2oNjknx1QApyTPvhiQJpBBMBCgBTAhsvAh4BAheAAhkBBAsJCAcFFQoJCAsFFgMCAQASGGhr cDovL3BncC5taXQuZWR1FiEE9bxVNMNvAIeznTbvHJ3E/rnbfEsFAllS4TwFCRa6MNwACgkQHJ3E /rnbfEuMFg//csSmcerqM+h1jwlKPJ4UiLXj3kFxSuSr0DrJzgLIpjuMdB2bhJ3YZwJTrZI9P7DP v+ebTKoeCjxoi5zJcINakZ9/FBn3cOCsYmpdHdaOPm34pHh9c0y9+NAvGJvugBRWIWK3qYYJzjff YL1r5lGCM4FXuhTcjVTTs5iCvYlAhtnHEAM6+c+mftdwrCJaqCiz3G13OCtpujn8fOn/B2nv6ZdT hen3Xqh3xBU2PnxPhVtyOulsJ318SKbUAk0h3tbLLkzFIS+7XvpHuwB72NkZw233IAN1Uq/nmJK/ YIsi+icNGpI60+PnLFiqYrSmtD6BCf8pOPq+isscu5uTuhfHt8fjLZPiqyf12CLoNKIp94sUi/1n M75GKz9MPWs6vqosq+E1xs9wVBw0gwUluejWgg3mIE3754W6xQ6tnpspdrt+x5kykfPMD3r1T8uq +z4ZUtE3A6QIjjxzIN0Xk8ZpC5Mg3yUzhyM1qF4e8pM2U7/tJC9BO5CNWJmKBqmGZ5sYw85wjEZ+ WnfU770tgCGZjcjIYjqYuoCgHMd2J8pkXdf76CINWK4+fCoKHr/4kWfU3e8ikFStTBer/Bu5tXa5 RPh90F6l2DNRitUdv+7vAiFnAuP61ZSUSqRbOiE1SPTv4cGBOCgB7xIOU6+rKzlweR/vO0otLZc+ TyFnSKPrfzg= Received-SPF: pass client-ip=80.241.217.52; envelope-from=me@tobias.gr; helo=tobias.gr X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.7 (/) X-Debbugs-Envelope-To: submit Cc: Ricardo Wurmus , Christopher Baines , Josselin Poiret , 65720@debbugs.gnu.org, Mathieu Othacehe X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.3 (/) >Why not trigger it by =E2=80=9Cguix gc=E2=80=9D? Unless there's a new option I missed, guix gc doesn't handle this=2E >Well, I expect =E2=80=9Cguix gc=E2=80=9D to take some time and I choose w= hen=2E However, >I want =E2=80=9Cguix pull=E2=80=9D or =E2=80=9Cguix time-machine=E2=80=9D= to be as fast as possible I don't think that things should be pushed into guix gc merely because the= y are slow=2E This is not a great post (I'd look at the git code if I were at a computer= ) but I remember git printing something like 'optimising repository in the = background'=2E Maybe something similar would be appropriate here, to bette= r hide such housekeeping from the user=2E Kind regards, T G-R Sent on the go=2E Excuse or enjoy my brevity=2E From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 24 05:01:50 2023 Received: (at submit) by debbugs.gnu.org; 24 Oct 2023 09:01:50 +0000 Received: from localhost ([127.0.0.1]:53092 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qvDIL-00072Y-Nn for submit@debbugs.gnu.org; Tue, 24 Oct 2023 05:01:50 -0400 Received: from lists.gnu.org ([2001:470:142::17]:60482) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qvDIJ-00072A-Ap for submit@debbugs.gnu.org; Tue, 24 Oct 2023 05:01:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qvDHf-0007Dg-52 for guix-patches@gnu.org; Tue, 24 Oct 2023 05:01:08 -0400 Received: from mail-wr1-x42f.google.com ([2a00:1450:4864:20::42f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qvDHb-0008OR-Bs; Tue, 24 Oct 2023 05:01:06 -0400 Received: by mail-wr1-x42f.google.com with SMTP id ffacd0b85a97d-32da4180ca2so888354f8f.1; Tue, 24 Oct 2023 02:01:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1698138059; x=1698742859; darn=gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=aw5Nh/VLUNvmWYXBFYb5tijmYPqxdfOEBWvJMDoke74=; b=HrCR8075kkcPjUcS/Za+LQlrGapttkCNWE4RpilTaoVEqgXbMJ3jibkBEOFRQSPJhm Sz73afqF0dpTl23bzxJ7XXFa1VesobmB7DlHyV6mWt3B/5yNExn0vydckrboihxF/hF6 0NPebBiJbgj7LD3Ejhn3+VWDPvSUDQa+bVl3sqeteUecmpDoJMZIhiCe6lnllBOY2gVr iA+RAq3yn5Y3SML4nonkxvKnb55jmX14UI5BYnLwnDCCwAsrcMmSTZrRgO5gHocDYJm7 rerLvSZ/Wvqdw+KtYbvMwQesk5TeehKmB5areFCMNfou1KE1strbTHs1LdOeC3kjh4VS tziA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698138059; x=1698742859; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=aw5Nh/VLUNvmWYXBFYb5tijmYPqxdfOEBWvJMDoke74=; b=vqFr06d+SRvmHE8cCE1q7HzDYUD6I/JL/TDsJTgNfBfWH4cn0m2CjmOR6mdUA2idFl xktzPto9/0hvDlUMdU/Gi8ewZytKP4Em/hYCYVRFF/O4g1ngD004kXAEuI0ztxCWTn8r LOT9Kbx/aCD3IZ9tIYLGbpeuirIcy95fR1uiFpijuJhfMCdoaoLuva+lqrZwZ7WJRek9 Qtgp+RGX02jeMxAKlO9F9d54Pd6Jo6KoRRaH7cwnpVmvcqtxuQICWIpGGRdA1GqQhe6b 0B8aKPSOl43Me8ty+6iCv4XXJdrxmEI8e579Mvu/Vg31a33gpeUSH/m2RN33y+rmglFS lZ8Q== X-Gm-Message-State: AOJu0Yx+H1jOLA4Xf8HBt2PoPttCsemBEc3NYtXwGt7zUPg97nXeFFnd fNHfv+JV+PNJp264W0lPsuAlbV+h2YI= X-Google-Smtp-Source: AGHT+IF+Ahwo5V154BiQy09zZUiILlaC8eqH4OJq7FVNNv39G7L84Er++dEJK1WK83EyJlec8ijTxg== X-Received: by 2002:adf:a455:0:b0:32d:d8ed:d6e0 with SMTP id e21-20020adfa455000000b0032dd8edd6e0mr7147722wra.0.1698138059506; Tue, 24 Oct 2023 02:00:59 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id g17-20020a5d5551000000b0032cc35c2ef7sm9490207wrw.29.2023.10.24.02.00.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Oct 2023 02:00:59 -0700 (PDT) From: Simon Tournier To: Tobias Geerinckx-Rice Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much References: <87jzswsrlt.fsf@gnu.org> <87il6xlkhk.fsf@gmail.com> <8A262178-2BFF-41EE-BEF2-5DC3270EF9C5@tobias.gr> Date: Tue, 24 Oct 2023 01:28:12 +0200 In-Reply-To: <8A262178-2BFF-41EE-BEF2-5DC3270EF9C5@tobias.gr> (Tobias Geerinckx-Rice's message of "Mon, 23 Oct 2023 22:27:39 +0000") Message-ID: <86r0lkvrzn.fsf_-_@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::42f; envelope-from=zimon.toutoune@gmail.com; helo=mail-wr1-x42f.google.com X-Spam_score_int: -5 X-Spam_score: -0.6 X-Spam_bar: / X-Spam_report: (-0.6 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_06_12=1.543, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi, On Mon, 23 Oct 2023 at 22:27, Tobias Geerinckx-Rice wrote: >>Why not trigger it by “guix gc”? > > Unless there's a new option I missed, guix gc doesn't handle this. Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: tobias.gr] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (zimon.toutoune[at]gmail.com) -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 1.1 DATE_IN_PAST_06_12 Date: is 6 to 12 hours before Received: date X-Debbugs-Envelope-To: submit Cc: Josselin Poiret , 65720@debbugs.gnu.org, Mathieu Othacehe , Ludovic =?utf-8?Q?Court=C3=A8s?= , Ricardo Wurmus , Christopher Baines , guix-patches@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.1 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi, On Mon, 23 Oct 2023 at 22:27, Tobias Geerinckx-Rice wrote: >>Why not trigger it by “guix gc”? > > Unless there's a new option I missed, guix gc doesn't handle this. Content analysis details: (1.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: tobias.gr] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (zimon.toutoune[at]gmail.com) -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 1.1 DATE_IN_PAST_06_12 Date: is 6 to 12 hours before Received: date -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager Hi, On Mon, 23 Oct 2023 at 22:27, Tobias Geerinckx-Rice wrote: >>Why not trigger it by =E2=80=9Cguix gc=E2=80=9D? > > Unless there's a new option I missed, guix gc doesn't handle this. Maybe I missed something but =E2=80=9Cguix gc=E2=80=9D handles what we impl= ement, no? :-) Well, I run =E2=80=9Cguix gc=E2=80=9D when I need some space. And this =E2=80=9Cmaybe-run-git-gc=E2=80=9D does exactly that: collect some spaces w= hen I need them. For me, they are part of =E2=80=9Cguix gc=E2=80=9D and not part of some upd= ate. Aside, re-thinking about other features, I am consistent with other comments I made when introducing =E2=80=99maybe-remove-expired-cache-entrie= s=E2=80=99; see . And consistent because most probably I still think the same: cache cleanup should be handled by =E2=80=9Cguix gc=E2=80=9D and not by the commands themselves. And maybe we= are having the same discussion. ;-) >>Well, I expect =E2=80=9Cguix gc=E2=80=9D to take some time and I choose w= hen. However, >>I want =E2=80=9Cguix pull=E2=80=9D or =E2=80=9Cguix time-machine=E2=80=9D= to be as fast as possible > > I don't think that things should be pushed into guix gc merely because > they are slow. Maybe I misread, somehow it appears to me that you miss the key part: I choose when some extra work is done and I keep =E2=80=9Cguix pull=E2=80=9D = and =E2=80=9Cguix time-machine=E2=80=9D as fast as possible. Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 30 08:04:59 2023 Received: (at 65720) by debbugs.gnu.org; 30 Oct 2023 12:04:59 +0000 Received: from localhost ([127.0.0.1]:44440 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qxR0s-00085U-RF for submit@debbugs.gnu.org; Mon, 30 Oct 2023 08:04:59 -0400 Received: from mira.cbaines.net ([212.71.252.8]:42990) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qxR0p-00085H-Nd; Mon, 30 Oct 2023 08:04:56 -0400 Received: from localhost (pd95b13b0.dip0.t-ipconnect.de [217.91.19.176]) by mira.cbaines.net (Postfix) with ESMTPSA id 5E33E27BBE2; Mon, 30 Oct 2023 12:04:21 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 955346f3; Mon, 30 Oct 2023 12:04:18 +0000 (UTC) References: <87jzswsrlt.fsf@gnu.org> User-agent: mu4e 1.10.5; emacs 28.2 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#66650] [PATCH] git: Shell out to =?utf-8?B?4oCYZ2l0IGdj?= =?utf-8?B?4oCZ?= when necessary. Date: Mon, 30 Oct 2023 12:02:59 +0000 In-reply-to: Message-ID: <87sf5swc3j.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65720 Cc: 65720@debbugs.gnu.org, 66650@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Fixes . > > This fixes a bug whereby libgit2-managed checkouts would keep growing as > we fetch. > > * guix/git.scm (packs-in-git-repository, maybe-run-git-gc): New > procedures. > (update-cached-checkout): Use it. > --- > guix/git.scm | 39 ++++++++++++++++++++++++++++++++++++--- > 1 file changed, 36 insertions(+), 3 deletions(-) > > Hi! > > This is a radical fix/workaround for the unbounded Git checkout growth > problem, shelling out to =E2=80=98git gc=E2=80=99 when it=E2=80=99s likel= y needed (=E2=80=9Ctoo many=E2=80=9D > pack files around). > > I thought we might be able to implement a =E2=80=98git gc=E2=80=99 approx= imation using > the libgit2 =E2=80=9Cpackbuilder=E2=80=9D interface, but I haven=E2=80=99= t got around to doing > it: . > > Once again, shelling out is not my favorite option, but it=E2=80=99s a bu= g we > should fix sooner rather than later, hence this compromise. > > Thoughts? This sounds good to me, the data service has this problem as well of cached checkouts that grow to be too large and this sounds like it'll address it. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmU/m8BfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xcx2A//f3qqeG4+4J5uvVPcP/26SMtpzvNCUsRs 5hzUgJherxg4U25tlUdXlpjSVTNDSc0qN7RpUQWy6rca/S/ro2NL/KcR7VAdyumu v39ldNwbq0W2YC/eZ9fxS7SzeCnWV2oOpO5X5sy69TGvjE/plWNStttvOF/HFy6Y CtH9GNXZE9xwL5PbRK1Lxun5JmamP3Lxk+oivN3ZC7AnhoYxGxJ4xHD9HfN970b/ 6kBbc0Vcf0wHtHRVIEuhBw01JkklchBhTTzbYoi3SUFZeovbkm4Ys5g3s7nDVVZD 5XNMkdp2YcgdUtsfZN1jhgFXTsa6XyfFnQS/1qMfPg3U1niAj0nKIqjEfmUNnGD1 PpQbQ5WvOZm5S70HHDG9Cg58BVIcH0hrHqfVYyghhttf2yUvdKp6CtqNyuBzsr7D 276K8EAeTMcfQwtArxKaFfFG/ggInMvPy1UA1FoN2j0EIIxeND/7vcejqqIssjZm jsU716+s9bP1JCf0s/gJPWSw7Iph7gOs4CKFUdQSeEqNawyXyetxc5PjI6K7NKzq QSa7SJlTe0Lv8maRIZ7LV8t08n3PPFO0sFC7MQMVTCFbkwqFwaVGGdeFiF8dYXfN m4eigk0nl9Poq7gQ79r0igy/rfkZW8mVKRucqVdUJ/znykJfiyBbadSmPcCZP6MS XzvO0Cgwki8= =sesm -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Nov 14 04:19:56 2023 Received: (at 65720) by debbugs.gnu.org; 14 Nov 2023 09:19:56 +0000 Received: from localhost ([127.0.0.1]:60021 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r2paO-0000lJ-Dj for submit@debbugs.gnu.org; Tue, 14 Nov 2023 04:19:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39730) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r2paL-0000kr-HX; Tue, 14 Nov 2023 04:19:54 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r2pZY-0004TY-Te; Tue, 14 Nov 2023 04:19:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=JZDU1hOSj2ITSd6DjETVBVkYN5+QtFH0XZM216c9Cuc=; b=kdEbrjIYkhUUgzl1O0BG z4mkDThAEOWVKdLJu5dwi6F0U0RBk6hj78YdTDQqIERlIJXAW6DRj9+d0Cio1OrcxTvQiAbOdQnPG ret1TpZDdGdeUS0uRurMQqjzddTomVBoMEStUEpTLBDX64ibAgXqimdmMkPnUDsndrKbPiLS43wOi 8JiRM3pHJr1hR1NT762nRI3csilJ3L7OUtI87viVA/P5DaKWZiGlANo9Lz5Cm7gFRAVLqJEZpiwud +l6U3JGnsQiCMFH6CxaGDPeX9ING1cJHWPlzUr/eGPQdsfh5j6dJ0GZspypgY5uktRkbOBxHQkGPY VDKqnJgU+seV6A==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: [bug#66650] [PATCH] git: Shell out to =?utf-8?B?4oCYZ2l0IGdj?= =?utf-8?B?4oCZ?= when necessary. In-Reply-To: <87sf5swc3j.fsf@cbaines.net> (Christopher Baines's message of "Mon, 30 Oct 2023 12:02:59 +0000") References: <87jzswsrlt.fsf@gnu.org> <87sf5swc3j.fsf@cbaines.net> Date: Tue, 14 Nov 2023 10:19:01 +0100 Message-ID: <87o7fwae0q.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 Cc: Josselin Poiret , 65720@debbugs.gnu.org, 66650@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, Christopher Baines skribis: > Ludovic Court=C3=A8s writes: > >> Fixes . >> >> This fixes a bug whereby libgit2-managed checkouts would keep growing as >> we fetch. [...] > This sounds good to me, the data service has this problem as well of > cached checkouts that grow to be too large and this sounds like it'll > address it. Thanks for your input, Chris. Any other comments? I=E2=80=99d like to push the patch within a few days if there are no objections. https://issues.guix.gnu.org/66650 Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Nov 14 07:04:26 2023 Received: (at 65720) by debbugs.gnu.org; 14 Nov 2023 12:04:26 +0000 Received: from localhost ([127.0.0.1]:60188 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r2s9a-0005LV-7C for submit@debbugs.gnu.org; Tue, 14 Nov 2023 07:04:26 -0500 Received: from mail-wr1-x42c.google.com ([2a00:1450:4864:20::42c]:39148) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r2s9X-0005Kr-ND; Tue, 14 Nov 2023 07:04:24 -0500 Received: by mail-wr1-x42c.google.com with SMTP id ffacd0b85a97d-32f831087c6so1028145f8f.0; Tue, 14 Nov 2023 04:03:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699963416; x=1700568216; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=ylTF2Qwpjb3oXQkbeS9e3AnLlKkJ1OUyQRtwgU+E1+Q=; b=AwP+SsOPMQHvKYvI/o4aU2byBlVjQ/CaHhUssWI6cpQiahhd1zCpvl6U2/utawpQBx ZTZvA+Vk2sSqMnvz/OHFVRQsTUHRPNqJ0Iyf7sQVS7FbXNtxnL5f3zmXfSnqv8fw7uRO 9S8lXYSKh66vvSv7jcDKKU4XqDUTUoms6CoyLcIx3xyXhCY2eZ3azAt8b9Y5EtTuLEUz A/zdHK2nJ5BWAxqjUccSJ5LQz3yvypZcd2bZ1pjHB/3Yf9cxz3PW7AdYrhdqIzlBtWTN iX3kqQ9rD8TgASpm7v/AZBgUPeOemaIPBgrEHVwTajcVa0ktjskYEx2SY0CAqPoWjSwc 38iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699963416; x=1700568216; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ylTF2Qwpjb3oXQkbeS9e3AnLlKkJ1OUyQRtwgU+E1+Q=; b=lH24OIockUIOGanmqFgbxrGRc7zttqbK3ESKamxrsLW7PnpTCsdHXI0phtt6zvCLml 8yBP7XfUHD4onbzgiItQW/c7qSXWqsVg4mYvFbVf+GNl5BYpADFPkhwsGx5AgC+L01Iy rCkmsMgmlVhHCZxn1EeZ5fyBNlP88KEXX9n84Na/mKgS/y29/dJeYMQQ+NOyRq1UbwHR inOwBbax1uVPo/df/xqlNEXppNOm2756cw9ZyG+0fvYL6ixJEkhHZBINOuEHdJC88ZBp UO5PPoi8tLHIOxQLUtDpdf6MjbJlb5yWVCH+/JGyTytulOszcRIm6qUqQVwQmotqrNTK dS8w== X-Gm-Message-State: AOJu0YxmDycvu0Kzn756BoKs/IPx6TdQsZTlu7/M2iYMK2IL5k9fZlHf sMvVouVj9jniSrUgVTcBwkM2c3I4ZV4= X-Google-Smtp-Source: AGHT+IGxrX8kNQ98k5MVjas+2EFIinZ9YvkPaQ15MARAjwxnTfVMWW20wlN2YlpNmNhmPTWnsvkRBQ== X-Received: by 2002:a5d:648c:0:b0:331:3cec:214f with SMTP id o12-20020a5d648c000000b003313cec214fmr1209144wri.7.1699963415483; Tue, 14 Nov 2023 04:03:35 -0800 (PST) Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id u30-20020adfa19e000000b00318147fd2d3sm7718993wru.41.2023.11.14.04.03.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Nov 2023 04:03:35 -0800 (PST) From: Simon Tournier To: Ludovic =?utf-8?Q?Court=C3=A8s?= , Christopher Baines Subject: Re: bug#65720: [bug#66650] [PATCH] git: Shell out to =?utf-8?B?4oCYZ2l0IGdj4oCZ?= when necessary. In-Reply-To: <87o7fwae0q.fsf@gnu.org> References: <87jzswsrlt.fsf@gnu.org> <87sf5swc3j.fsf@cbaines.net> <87o7fwae0q.fsf@gnu.org> Date: Tue, 14 Nov 2023 10:32:41 +0100 Message-ID: <87v8a4el3a.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65720 Cc: Josselin Poiret , 65720@debbugs.gnu.org, 66650@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, On Tue, 14 Nov 2023 at 10:19, Ludovic Court=C3=A8s wrote: > Any other comments? I=E2=80=99d like to push the patch within a few days= if > there are no objections. As mentioned in [1], >> * guix/git.scm (packs-in-git-repository, maybe-run-git-gc): New >> procedures. >> (update-cached-checkout): Use it. >> --- >> guix/git.scm | 39 ++++++++++++++++++++++++++++++++++++--- >> 1 file changed, 36 insertions(+), 3 deletions(-) LGTM. Just two colors for the bikeshed. :-) >> + (when (> (packs-in-git-repository directory) 25) Why 25? And not 10 or 50 or 100? >> (define* (update-cached-checkout url >> #:key >> (ref '()) >> @@ -515,6 +545,9 @@ (define* (update-cached-checkout url >> seconds seconds >> nanoseconds nanoseconds)))) >>=20=20 >> + ;; Run 'git gc' if needed. >> + (maybe-run-git-gc cache-directory) Why not trigger it by =E2=80=9Cguix gc=E2=80=9D? Well, I expect =E2=80=9Cguix gc=E2=80=9D to take some time and I choose whe= n. However, I want =E2=80=9Cguix pull=E2=80=9D or =E2=80=9Cguix time-machine=E2=80=9D t= o be as fast as possible and here some extra time is added, and I cannot control exactly when. Cheers, simon 1: bug#65720: [PATCH] git: Shell out to =E2=80=98git gc=E2=80=99 when neces= sary. Simon Tournier Mon, 23 Oct 2023 12:08:07 +0200 id:87il6xlkhk.fsf@gmail.com https://issues.guix.gnu.org/65720 https://issues.guix.gnu.org/msgid/87il6xlkhk.fsf@gmail.com https://yhetil.org/guix/87il6xlkhk.fsf@gmail.com From debbugs-submit-bounces@debbugs.gnu.org Thu Nov 16 07:12:35 2023 Received: (at 65720) by debbugs.gnu.org; 16 Nov 2023 12:12:35 +0000 Received: from localhost ([127.0.0.1]:54658 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r3bEY-0006xi-Sg for submit@debbugs.gnu.org; Thu, 16 Nov 2023 07:12:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50414) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r3bEU-0006x9-HC; Thu, 16 Nov 2023 07:12:30 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r3bEO-0001BF-E9; Thu, 16 Nov 2023 07:12:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=ZEuFNinQlxo+OxQc1igD8XwkMi/joiRI0BI6c1SFqec=; b=ZFjlK3F4OWU+fez3iiWz IxsLp6ZtJRjErJj/o4gtgt1JCY7E1PHVTZ4ZDEmq8mORUCPR4y6rB/DAREMu8mI6i/s/p1QDwfE3w NNmVmuW0BNR6G2inIgfnUCeavacFkv5ww/W9Q2AcbcV1oFsh5VAOgD9AoA20LjlgDhS6jADhiRDrk kdY88EHA5DJsmDNCcPAAVz6qiiCcWG5Q6Ezg8GueQzuKdHm1pBap47ThK44XZK4jbniweKJ6S6JYX UKIptuWX6hh36Zb+mowvorzPnBB7SUWV1hRc3D9Q/u4D5B1oYB3H9qR+XrFKfbu9+wZfef7YUb/uB qbKsTlXJU438Cw==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Simon Tournier Subject: Re: bug#65720: [bug#66650] [PATCH] git: Shell out to =?utf-8?B?4oCYZ2l0IGdj4oCZ?= when necessary. In-Reply-To: <87v8a4el3a.fsf@gmail.com> (Simon Tournier's message of "Tue, 14 Nov 2023 10:32:41 +0100") References: <87jzswsrlt.fsf@gnu.org> <87sf5swc3j.fsf@cbaines.net> <87o7fwae0q.fsf@gnu.org> <87v8a4el3a.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Sextidi 26 Brumaire an 232 de la =?utf-8?Q?R=C3=A9vo?= =?utf-8?Q?lution=2C?= jour de la Pistache X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 16 Nov 2023 13:12:22 +0100 Message-ID: <87h6ll28yh.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 Cc: Josselin Poiret , Christopher Baines , 65720@debbugs.gnu.org, 66650@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Simon Tournier skribis: >>> * guix/git.scm (packs-in-git-repository, maybe-run-git-gc): New >>> procedures. >>> (update-cached-checkout): Use it. >>> --- >>> guix/git.scm | 39 ++++++++++++++++++++++++++++++++++++--- >>> 1 file changed, 36 insertions(+), 3 deletions(-) > > LGTM. Thanks! > Just two colors for the bikeshed. :-) > > >>> + (when (> (packs-in-git-repository directory) 25) > > Why 25? And not 10 or 50 or 100? Totally arbitrary. :-) I sampled the checkouts I had on my laptop and that seems like a reasonable heuristic. In particular, it seems that Git-managed checkouts never have this many packs; only libgit2-managed checkouts do, precisely because libgit2 doesn=E2=80=99t repack/GC. >>> + ;; Run 'git gc' if needed. >>> + (maybe-run-git-gc cache-directory) > > Why not trigger it by =E2=80=9Cguix gc=E2=80=9D? Because so far the idea is that ~/.cache/guix/checkouts is automatically managed without user intervention; it=E2=80=99s really a cache in that sens= e. > Well, I expect =E2=80=9Cguix gc=E2=80=9D to take some time and I choose w= hen. However, > I want =E2=80=9Cguix pull=E2=80=9D or =E2=80=9Cguix time-machine=E2=80=9D= to be as fast as possible and > here some extra time is added, and I cannot control exactly when. Yes, I see. The thing is =E2=80=98maybe-run-git-gc=E2=80=99 is only called= on the slow path; so for example, it=E2=80=99s not called on a =E2=80=98time-machine=E2= =80=99 cache hit, but only on a cache miss, which is already expensive anyway. Does that make sense? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Nov 16 08:25:10 2023 Received: (at 65720) by debbugs.gnu.org; 16 Nov 2023 13:25:10 +0000 Received: from localhost ([127.0.0.1]:54775 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r3cMn-0003LZ-SP for submit@debbugs.gnu.org; Thu, 16 Nov 2023 08:25:10 -0500 Received: from mail-vk1-xa32.google.com ([2607:f8b0:4864:20::a32]:41402) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r3cMm-0003LE-4P; Thu, 16 Nov 2023 08:25:09 -0500 Received: by mail-vk1-xa32.google.com with SMTP id 71dfb90a1353d-4abf066fa12so164595e0c.1; Thu, 16 Nov 2023 05:25:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700141103; x=1700745903; darn=debbugs.gnu.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=iT1oPeGFA7W1gIoMlp6My9hhZFQktk1eIxQtFZfDzbg=; b=Ed4hVnf0O55lV44Z9GY3Rhfpwgg/x23hJrRdeBg5mr1sIhK1fz9yM8RgeBDxFUq4D2 wXDNRqlb2BGJ41kaAtyzOWEWb/lVKPf8KCT6VRkxl5ew29kynlle159Y4hxgUYLsYq1A ZiRxaHANTDrZaj2Ncvi7wn2Ogql2He7f3aJ8cDvjA96vhiCKZRBBcCsGDZsdR8kUheHH apZNU8vr5Bl591napvUQ4WcfTDfIYcTXKSzqV4toZdbGayWYL2F8p/3KZvujLQlCyiTk cCpkX/rAjVBOufaxJRfBFL2nlH3eAZZDyJFiIyzVOLEyoemh6CSBHDNLvR0MjPPYp31A K8Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700141103; x=1700745903; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iT1oPeGFA7W1gIoMlp6My9hhZFQktk1eIxQtFZfDzbg=; b=affSzyL/wkSJFlt/npcguRlsap6WAEwtWz95L0cikApyA21gC9/tMNZTsiKzuhIwWo BKmlaz+8CAG+km1ucV6g9X+tjhyumwCYUuUHLeylg7lZfJ0UZQqxxl3qRXcI4bJuhPxr ePNK3VVX3azwqzsFHzBUYg8MKjHmX/rzzBa4pHhsKYWRROlC8Xon5YZEZF4SsMCQvhWP N+99DfE6gL5PZj6w03gQ0ljW9i3pnwuO1CqxxPDKh6aJ4PRMHQHSTlm1r0S2Ru9dV84e 4r4oc2NTlZgQzeyf+msvfjBiNH1FHOF9hkMezesUgo8Ggp90bPJSvs+yV32/GtvbmzNy 4/HQ== X-Gm-Message-State: AOJu0YxcyJxcEoSO9np/VcosNj8L+rn3dKmP3VxwEffwU5BVQjb2Tb5K gXxBlkmPn71ofWU+nCrWdYehxqFyZ81TLla4gyk= X-Google-Smtp-Source: AGHT+IERgt2ppQkKfWIZOIy7XSrP92IpnO9osFJyZ726t8riBobcc325dQ1HELzZa60bxTifVgexAjqSZNDmoWScr8g= X-Received: by 2002:a05:6102:2c02:b0:460:f279:ae0e with SMTP id ie2-20020a0561022c0200b00460f279ae0emr8825695vsb.1.1700141102724; Thu, 16 Nov 2023 05:25:02 -0800 (PST) MIME-Version: 1.0 References: <87jzswsrlt.fsf@gnu.org> <87sf5swc3j.fsf@cbaines.net> <87o7fwae0q.fsf@gnu.org> <87v8a4el3a.fsf@gmail.com> <87h6ll28yh.fsf@gnu.org> In-Reply-To: <87h6ll28yh.fsf@gnu.org> From: Simon Tournier Date: Thu, 16 Nov 2023 14:24:51 +0100 Message-ID: Subject: =?UTF-8?B?UmU6IGJ1ZyM2NTcyMDogW2J1ZyM2NjY1MF0gW1BBVENIXSBnaXQ6IFNoZWxsIG91dCB0bw==?= =?UTF-8?B?IOKAmGdpdCBnY+KAmSB3aGVuIG5lY2Vzc2FyeS4=?= To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65720 Cc: Josselin Poiret , Christopher Baines , 65720@debbugs.gnu.org, 66650@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, On Thu, 16 Nov 2023 at 13:12, Ludovic Court=C3=A8s wrote: > > Well, I expect =E2=80=9Cguix gc=E2=80=9D to take some time and I choose= when. However, > > I want =E2=80=9Cguix pull=E2=80=9D or =E2=80=9Cguix time-machine=E2=80= =9D to be as fast as possible and > > here some extra time is added, and I cannot control exactly when. > > Yes, I see. The thing is =E2=80=98maybe-run-git-gc=E2=80=99 is only call= ed on the slow > path; so for example, it=E2=80=99s not called on a =E2=80=98time-machine= =E2=80=99 cache hit, but > only on a cache miss, which is already expensive anyway. What you mean as "only called on the slow path" is each time 'update-cached-checkout' is called, right? So, somehow when 'maybe-run-git-gc' is called appears to me "unpredictable". But anyway. :-) Let move it elsewhere if I am really annoyed. Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 22 06:18:10 2023 Received: (at 65720) by debbugs.gnu.org; 22 Nov 2023 11:18:11 +0000 Received: from localhost ([127.0.0.1]:58126 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r5lFC-0001u6-HM for submit@debbugs.gnu.org; Wed, 22 Nov 2023 06:18:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:34650) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r5lFA-0001tn-1U; Wed, 22 Nov 2023 06:18:08 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r5lF1-000317-3G; Wed, 22 Nov 2023 06:17:59 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=MWXC4TYqLrnQXs1ngFX4mcOjA6MZowvoq66b2g7zDa4=; b=gF9gJtmSMNyKdkRRimV/ dTSlbTvLr6f0VYtMS92CkyrCM7diHPtITK475CkSmNnAmirZ9LGIeYYdDyXEQAcbGNLu+BS4JV6m6 WcBDswGI0iJ4BOjKd+UE22FPLLR0pe/H5UQW46kdiyyZE5n69FEUDn8RL7mNTlNUXh634I4icFBVM QCOJYklIXJt283yHPrjdkABtUzjE5JYXgPpd7bbYm//3ihyVxaRqVekexHIQ9X3hy7SNU9ANuhehh sSicMTKJigaLvwV3++y7oCsHPEoGpRfOaNDncctftsdVigWlbSzknQkL4332zOieKctvlX9ZRa0Ha QQU+NzvHAmMzUQ==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Simon Tournier Subject: Re: [bug#66650] bug#65720: [bug#66650] [PATCH] git: Shell out to =?utf-8?B?4oCYZ2l0IGdj4oCZ?= when necessary. In-Reply-To: (Simon Tournier's message of "Thu, 16 Nov 2023 14:24:51 +0100") References: <87jzswsrlt.fsf@gnu.org> <87sf5swc3j.fsf@cbaines.net> <87o7fwae0q.fsf@gnu.org> <87v8a4el3a.fsf@gmail.com> <87h6ll28yh.fsf@gnu.org> Date: Wed, 22 Nov 2023 12:17:55 +0100 Message-ID: <874jhem3z0.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720 Cc: Josselin Poiret , Christopher Baines , 65720@debbugs.gnu.org, 66650@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Simon Tournier skribis: > On Thu, 16 Nov 2023 at 13:12, Ludovic Court=C3=A8s wrote: > >> > Well, I expect =E2=80=9Cguix gc=E2=80=9D to take some time and I choos= e when. However, >> > I want =E2=80=9Cguix pull=E2=80=9D or =E2=80=9Cguix time-machine=E2=80= =9D to be as fast as possible and >> > here some extra time is added, and I cannot control exactly when. >> >> Yes, I see. The thing is =E2=80=98maybe-run-git-gc=E2=80=99 is only cal= led on the slow >> path; so for example, it=E2=80=99s not called on a =E2=80=98time-machine= =E2=80=99 cache hit, but >> only on a cache miss, which is already expensive anyway. > > What you mean as "only called on the slow path" is each time > 'update-cached-checkout' is called, right? Yes, which usually indicates we=E2=80=99re on a cache miss (for example a c= ache miss of =E2=80=98guix time-machine=E2=80=99) and thus are going to do poten= tially more work (updating a Git repo, building things, etc.). That=E2=80=99s why I th= ink it=E2=80=99s on the =E2=80=9Cslow path=E2=80=9D and shouldn=E2=80=99t make = much of a difference. More importantly, unless I=E2=80=99m mistaken, it=E2=80=99s rarely going to fire. > So, somehow when 'maybe-run-git-gc' is called appears to me > "unpredictable". But anyway. :-) Sure, but the way I see it, that=E2=80=99s the nature of caches. > Let move it elsewhere if I am really annoyed. :-/ Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 22 06:57:31 2023 Received: (at 65720) by debbugs.gnu.org; 22 Nov 2023 11:57:31 +0000 Received: from localhost ([127.0.0.1]:58137 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r5lrG-00034c-RX for submit@debbugs.gnu.org; Wed, 22 Nov 2023 06:57:31 -0500 Received: from mail-wm1-x32d.google.com ([2a00:1450:4864:20::32d]:40975) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r5lrD-00034I-By; Wed, 22 Nov 2023 06:57:28 -0500 Received: by mail-wm1-x32d.google.com with SMTP id 5b1f17b1804b1-408363c2918so7060895e9.0; Wed, 22 Nov 2023 03:57:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700654238; x=1701259038; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=dOZ3jbw+h+F/Dnxm5Tc+po224qPHyu2O+nU8on3+UEM=; b=LhLi/VFwMxItHl7cBd8eRK0y8/70OB5G4/zixHzY/wy1+jqXK7TpSkq0nZuVC232Y4 JGVagwBZMH/0ymMYNdG73cnsOa0j4sqTyEMuTG4NIcpDakj+0FAVumzxZEbgMeZ6Smhb wVaohdJNUqWT6h9r4ThdS1n131jxaCVPQzhnfoXTj9C3vmPwO1ZtlRvT49q1GtG2RVBC bV/SX+xvuAbc0PIarXRLOEzgllpy19r41ywCW/rr0m0PIHdJ1ZBENGjD+ddU0ZdaLUs+ SP98BeA/tdYAUXc2UN4b+4KGqUcLJDXu1SMGEZIpsZaUhxiRVnGasBZm8/CH7jBwhada iKww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700654238; x=1701259038; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=dOZ3jbw+h+F/Dnxm5Tc+po224qPHyu2O+nU8on3+UEM=; b=ERKCa1i/12vc+ugaHO6N2w3fbBduts60Dk1w+kt3C4eVY76G4bd9QTR2ZO/01J39CZ CNRaouwv0Xv/J6+w1KA8BCOSbN71MdKso3gPhMRiLuj6U1ubC+ZJGQ4b8aAuNIekTyi6 hzyQpEbFUdKejH2OmLDbQ3VwUrGD9Rl+uhnaTBHUeouMJymPgb5rDIuiwfpguUB9Iw0M rmhs3J+hE0cIowC9LpThWBdziJwvpBrwo+zTk1Zre+WS7OaZVGbIMcHIXuS2MYlS87Nb fioZXldJhl0LQPn+7NUurd1aT+MkmGOhAUDcofmA78J2pQTYO4WcmAEFjQYTQoLEB8XW sb3w== X-Gm-Message-State: AOJu0YzJUf3yJq0aLcoVuC1omT9KGYqmtci8MxxTiG+cnvAbr5gIUqFi f5ECQ5cNrH4CLtgYQbaINboXKbwsE3U= X-Google-Smtp-Source: AGHT+IFlRPsJJM81tcYy8kyNkmToAEmfISRsI1/v/tKT29cOZh6jY+BRhKq87dh9hC9IReMHZqKt9w== X-Received: by 2002:a05:600c:3104:b0:40b:2aaf:e066 with SMTP id g4-20020a05600c310400b0040b2aafe066mr1638948wmo.2.1700654238007; Wed, 22 Nov 2023 03:57:18 -0800 (PST) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id s21-20020a05600c45d500b0040a48430837sm2000668wmo.13.2023.11.22.03.57.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Nov 2023 03:57:17 -0800 (PST) From: Simon Tournier To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much References: <87jzswsrlt.fsf@gnu.org> <87sf5swc3j.fsf@cbaines.net> <87o7fwae0q.fsf@gnu.org> <87v8a4el3a.fsf@gmail.com> <87h6ll28yh.fsf@gnu.org> <874jhem3z0.fsf@gnu.org> Date: Wed, 22 Nov 2023 12:57:15 +0100 In-Reply-To: <874jhem3z0.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Wed, 22 Nov 2023 12:17:55 +0100") Message-ID: <86ttpehug4.fsf_-_@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65720 Cc: Josselin Poiret , Christopher Baines , 65720@debbugs.gnu.org, 66650@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludo, Thanks for explaining. On Wed, 22 Nov 2023 at 12:17, Ludovic Court=C3=A8s wrote: > it=E2=80=99s rarely going to fire. [...] >> Let move it elsewhere if I am really annoyed. > > :-/ Sorry, I poorly worded my last comment. :-) Somehow I was expressing: my view probably falls into the =E2=80=9CPremature optimization is the root of all evil=E2=80=9D category. Other said, I have= no objection and I will revisit the issue when I will be on fire, if I am, or annoyed for real. Cheers, simon PS: Aside this patch: >> So, somehow when 'maybe-run-git-gc' is called appears to me >> "unpredictable". But anyway. :-) > > Sure, but the way I see it, that=E2=80=99s the nature of caches. What makes cache unpredictable is their current state. However, this does not imply that *all* the actions modifying from one state to another must also be triggered in unpredictable moment. For instance, I choose when I wash family=E2=80=99s clothes and the wash-ma= chine does not start by itself when the unpredictable stack of family=E2=80=99s d= irty clothes is enough. Because, maybe today it=E2=80=99s rainy so drying is difficult and tomorrow will be sunny so it will be a better moment. :-) For me, =E2=80=9Cguix gc=E2=80=9D should be the driver for cleaning all the= various Guix caches. Anyway. :-D From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 22 11:00:37 2023 Received: (at 65720-done) by debbugs.gnu.org; 22 Nov 2023 16:00:37 +0000 Received: from localhost ([127.0.0.1]:59710 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r5peX-0006w5-HE for submit@debbugs.gnu.org; Wed, 22 Nov 2023 11:00:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43556) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r5peW-0006ve-9Q; Wed, 22 Nov 2023 11:00:36 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r5peM-0008H9-WA; Wed, 22 Nov 2023 11:00:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=gOeBvHBjrYk1lK6NZsKFKkZxBvOCYRIce4ZSgF8qBSw=; b=GvxX8mE6Uo1o2+3E9xyr aWfc3kwEgxI6nfGYOTlKEnORwhg1HBpyR8Lo5YRKbC9lWAk16oKhmfRgmHbBRv35JW4UMz66rs/f+ yFZ8ss4qCFXlMO54jYDRGHNaJaHKbYlKH7ATY1GGxBg5DHO33vBhZ38IVnYxt7EInE2Ec28oP2bKx uU6GYmEiFxVIsDAyG1PdFR0LZqhxZSZpI3yK61yCs5BsMhUu5aAz7Vy9aXpoCi8QjzflP+bDJYbQq wnHLpPZMc8t10EwT7b37CZlqDeEvUI2LBJvW7DmiO8by61f4HHfjE7gSH3ADH1kwgyVYYBEAZFVSZ hIiZYWIhjrw3qA==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Simon Tournier Subject: Re: [bug#66650] bug#65720: Guile-Git-managed checkouts grow way too much In-Reply-To: <86ttpehug4.fsf_-_@gmail.com> (Simon Tournier's message of "Wed, 22 Nov 2023 12:57:15 +0100") References: <87jzswsrlt.fsf@gnu.org> <87sf5swc3j.fsf@cbaines.net> <87o7fwae0q.fsf@gnu.org> <87v8a4el3a.fsf@gmail.com> <87h6ll28yh.fsf@gnu.org> <874jhem3z0.fsf@gnu.org> <86ttpehug4.fsf_-_@gmail.com> Date: Wed, 22 Nov 2023 17:00:07 +0100 Message-ID: <87h6ldkcc8.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720-done Cc: Josselin Poiret , Christopher Baines , 65720-done@debbugs.gnu.org, 66650-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Simon Tournier skribis: > Somehow I was expressing: my view probably falls into the =E2=80=9CPremat= ure > optimization is the root of all evil=E2=80=9D category. Other said, I ha= ve no > objection and I will revisit the issue when I will be on fire, if I am, > or annoyed for real. Alright! Pushed as b150c546b04c9ebb09de9f2c39789221054f5eea. Let=E2=80=99s see how it behaves and if there are problems we had overlooke= d=E2=80=A6 Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Nov 23 06:35:57 2023 Received: (at 65720-done) by debbugs.gnu.org; 23 Nov 2023 11:35:57 +0000 Received: from localhost ([127.0.0.1]:60773 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r67zw-00028c-DV for submit@debbugs.gnu.org; Thu, 23 Nov 2023 06:35:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:37530) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r67zo-00028E-AJ for 65720-done@debbugs.gnu.org; Thu, 23 Nov 2023 06:35:55 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r67zf-0002x0-2e for 65720-done@debbugs.gnu.org; Thu, 23 Nov 2023 06:35:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=8qhtZikQebzOiBNIiFXbjlyFmJgMY2wkkruOEd2W3y8=; b=hpw2MAZSpFLD8u1dXhXz 9kIUJi2J7AIBHdCKvYR9gq568YQeFQgay1xWcGUHyeiLXozWasehpf7zJWqC7HmiDE9E7vtPgxyQt lq8Z06X98JTxT0c4lGdVzxwVUTcly+CbgzuH+MmrVO9ecG57y/ajfvuTeEBiYAgLp9yY4Um5x2TnV GZIuhQlbFPBDnW+HQAByeQyGU9rtF74qpnynmauBHT0UnGwmo4c182VlK+3KDMgje49rlQaGHPLBu iantmmKX+85hwNJbDcE61aeM2zRx/irfXsjFmvT3xGKcO/6O86tRreDDrrsRw4A43dQKGEPMv4PWG WrG06VxN2Z1c5w==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: 65720-done@debbugs.gnu.org Subject: Re: bug#65720: Guile-Git-managed checkouts grow way too much In-Reply-To: <87bkejc7go.fsf@inria.fr> ("Ludovic =?utf-8?Q?Court=C3=A8s=22?= =?utf-8?Q?'s?= message of "Sun, 03 Sep 2023 22:44:39 +0200") References: <87bkejc7go.fsf@inria.fr> Date: Thu, 23 Nov 2023 12:35:36 +0100 Message-ID: <87ttpcd7nb.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65720-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s skribis: > As reported by Tobias on IRC (in the context of =E2=80=98hpcguix-web=E2= =80=99), > checkouts managed by Guile-Git appear to grow beyond reason. As an > example, here=E2=80=99s the same =E2=80=98.git=E2=80=99 managed with Guil= e-Git and with Git: > > $ du -hs ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27= shj7apsnalwq > 6.7G /home/ludo/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlay= wmisyr27shj7apsnalwq > $ du -hs .git > 517M .git Fixed by b150c546b04c9ebb09de9f2c39789221054f5eea. We still need to update the =E2=80=98guix=E2=80=99 package so that tools th= at rely on (guix git) such as the Data Service, hpcguix-web, and Cuirass, can benefit from this change. Ludo=E2=80=99. From unknown Mon Jun 16 17:18:35 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 21 Dec 2023 12:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator