GNU bug report logs - #50960
[PATCH 00/10] Add 'guix shell' to subsume 'guix environment'

Previous Next

Full log

View this message in rfc822 format

From: Maxime Devos <maximedevos <at> telenet.be>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 50960 <at> debbugs.gnu.org
Subject: [bug#50960] [PATCH 10/10] shell: Maintain a profile cache.
Date: Sat, 02 Oct 2021 16:47:20 +0200

[Message part 1 (text/plain, inline)]

Ludovic Courtès schreef op za 02-10-2021 om 16:12 [+0200]:
[reordered]
> > So I think the cache should also check if these dependencies have been modified.
> > To keep track of the dependencies, something like the ‘compile-all,compile:
> > Keep track of dependencies of compiled modules.’ patch from
> > <https://issues.guix.gnu.org/50384> could be used, though the 'include', 'load',
> > 'include-from-path' and maybe 'use-modules' (if something like "guix shell -Lextra-modules-directory ..."
> > is done) macros would need to be replaced.
> 
> Problem is that any attempt to keep track of dependencies is always an
> approximation because macros can do anything—I can have my own macro
> that reads files at expansion time.  So I’m inclined to not even try.

I expect most people use 'include', 'load', 'include-from-path' or 'load-from-path'
instead of writing their own macro reading files at expansion time, and if they do
write their own macro, I expect it would be implemented in terms of the former anyway,
so I expect replacing these macros and 'use-modules' to be sufficient, especially
if the flag proposed below is available as ‘escape hatch’ for when the dependency
tracking is insufficient.

This is not merely an expansion-time problem, files loaded at 'load' or 'eval' time
are important as well.  E.g., consider the case where the manifest is something like

  (list
    (package
      (inherit stuff)
      (source (local-file "stuff" #:recursive? #t)))
    other-packages ...)

then all files inside 'stuff' are important as well.  I don't see how dependencies
could be tracked here without an excessive amount of 'stat' calls, maybe guix should
ignore these dependencies (possibly with a warning, and a reference to the manual
documenting which dependencies are tracked and which are not?). 

Ludovic Courtès schreef op za 02-10-2021 om 16:12 [+0200]:
> I’m not sure how to address it.  We’d need a cache key that’s more
> precise than inode/mtime, yet cheap to compute.

The mtime of the file and the mtime of every dependency (ignoring modules from
Guile, Guix and channels to reduce the number of calls to 'stat'). 

> Perhaps we need to live with that limitation, document it, and provide a
> flag to force a rebuild?

A documented flag to always consider the cache stale seems good, though I think
at least the dependencies made with the common macros and procedures 'include',
'load', 'include-from-path', 'load-from-path', 'use-modules' and non-recursive
'local-file' could be tracked, though this could be left as a TODO for later
I suppose.

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.