Package: guix;
Reported by: Simon Tournier <zimon.toutoune <at> gmail.com>
Date: Wed, 6 Sep 2023 16:27:01 UTC
Severity: normal
Message #11 received at 65787 <at> debbugs.gnu.org (full text, mbox):
From: Simon Tournier <zimon.toutoune <at> gmail.com> To: Ludovic Courtès <ludo <at> gnu.org> Cc: 65787 <at> debbugs.gnu.org Subject: Re: bug#65787: time-machine is doing too much network requests Date: Mon, 11 Sep 2023 11:41:54 +0200
Hi Ludo,
> Maybe this is the purpose of your message:
> reducing Git remote accesses in those cases?
Yes. :-)
On Sun, 10 Sep 2023 at 22:10, Ludovic Courtès <ludo <at> gnu.org> wrote:
> It would probably be easier if you could come with
> an example where there’s Git-related network traffic where there
> shouldn’t.
Do we agree that the both Guile-Git procedures ’remote-fetch’ and
’branch-lookup’ are doing network traffic?
If yes, here two examples:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix pull -q -p /tmp/new
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
;;; (remote-fetch NETWORK)
;;; (branch-lookup NETWORK)
Authenticating channel 'guix', commits 9edb3f6 to 2eb6df5 (83 new commits)...
Building from this channel:
guix https://git.savannah.gnu.org/git/guix.git 2eb6df5
--8<---------------cut here---------------end--------------->8---
Well, that’s not perfect… it becomes much worse:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix time-machine -q -- describe
;;; (remote-fetch NETWORK)
;;; (branch-lookup NETWORK)
;;; (remote-fetch NETWORK)
;;; (branch-lookup NETWORK)
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
;;; (remote-fetch NETWORK)
;;; (branch-lookup NETWORK)
building /gnu/store/z8jwdgr7z6i8c00msdm2kzfv0n3zp25v-module-import-compiled.drv...
--8<---------------cut here---------------end--------------->8---
Do we agree that is suboptimal?
> ‘cached-channel-instance’ has 3 cases:
>
> 1. Obvious cache hit: This is when CHANNELS specifies the commit of
> each target channel (this happens for example when you type ‘guix
> time-machine -q --commit=a4c35c607cfd7d6b0bad90cfcc46188d489e1754)
> *and* the combination of channels is already in
> ~/.cache/guix/inferiors. This is the optimal case: the Git repo
> doesn’t even need to be opened.
Yes.
> 2. Cache hit: CHANNELS are pinned, but refer to tags (like “v1.2.0”)
> or short commit IDs (like “a4c35c6”). In that case,
> ‘channel-full-commit’ opens the Git repo to resolve the identifier.
> After that, we go to case #1 or #4.
That’s suboptimal. Currently, it reads:
--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 28 sept. 06 2023 14:54:50 (current)
guix 6113e05
repository URL: https://git.savannah.gnu.org/git/guix.git
commit: 6113e0529d61df7425f64e30a6bf77f7cfdfe5a5
$ ./pre-inst-env guix time-machine -q --commit=6113e05 -- describe
;;; (remote-fetch NETWORK)
;;; (remote-fetch NETWORK)
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
;;; (remote-fetch NETWORK)
Computing Guix derivation for 'x86_64-linux'... | C-c C-c
--8<---------------cut here---------------end--------------->8---
Using patch from #65352 [1], it removes these useless traffic:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix time-machine -q --commit=6113e05 -- describe
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Computing Guix derivation for 'x86_64-linux'... | C-c C-c
--8<---------------cut here---------------end--------------->8---
Using the proposed patch, it becomes optimal, IMHO. Well, let discuss
it in #65352 – comments are welcome. :-)
> 3. Cache hit: CHANNELS are not pinned—i.e., they refer to a branch,
> not a commit. In that case we first need to perform a ‘git fetch’
> and then we go to #1 or #4.
I hope that I am convincing you that this case is suboptimal: it does 3
’git fetch’ and more 3 others lookup requiring network. So it is about
6 network access when only one is necessary, IMHO.
> 4. Cache miss: ‘reference-available?’ returns #f for the channel
> commits, we got through ‘remote-fetch’ followed by
> ‘build-derivations’.
This case is part of #2 and discussed in #65352, IMHO.
Cheers,
simon
1: [bug#65352] [PATCH v2] DRAFT git: Avoid touching the network unless needed in 'reference-available?'.
Simon Tournier <zimon.toutoune <at> gmail.com>
Wed, 06 Sep 2023 16:17:08 +0200
id:32d3fb5066e0b20e200dabef0fba897634e21dda.1694009405.git.zimon.toutoune <at> gmail.com
https://yhetil.org/guix/32d3fb5066e0b20e200dabef0fba897634e21dda.1694009405.git.zimon.toutoune <at> gmail.com
https://issues.guix.gnu.org/msgid/32d3fb5066e0b20e200dabef0fba897634e21dda.1694009405.git.zimon.toutoune <at> gmail.com
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.