Ludovic Courtès schreef op ma 04-10-2021 om 10:19 [+0200]: > > A documented flag to always consider the cache stale seems good, though I think > > at least the dependencies made with the common macros and procedures 'include', > > 'load', 'include-from-path', 'load-from-path', 'use-modules' and non-recursive > > 'local-file' could be tracked, though this could be left as a TODO for later > > I suppose. > > Tracking those uses reliably is impossible: there could be same-named > bindings that do other things, there could be custom macros, there could > be “dynamic arguments” (whose value is not known statically), etc. You > have to expand + evaluate the code to get better results, and even then, > there might be different paths in the code so you can’t be sure you got > it right. I think there's a miscommunication here. From what I'm reading, what you have in mind is that, to determine the dependency information, "guix shell" would open "guix.scm", read it with the procedure 'read' and sniff for 'load', 'include-from-path', 'load-from-path', 'use-modules' and 'local-file' form -- something like 'module-file-dependencies' and 'extract-dependencies', but more general. However, my idea is to replace these macros, such that, when "guix shell" loads "guix.scm" or "manifest.scm", these macros inform "guix shell" that "guix.scm" or "manifest.scm" depend on certain files referred to by 'load', 'include-from-path', etc. forms, using a mechanism like the 'notice-dependency' defined in . Then, when "guix shell" puts the resulting profile in the cache, it includes the generated list of files. And when "guix shell" finds an entry in the cache, it will check if the files in this list (and guix.scm or manifest.scm of course) have been modified. If some are (or the forcing flag is set), guix.scm needs to be loaded and a new profile needs to be generated. If they are all unchanged, guix.scm will _not_ be read: the cached profile can be reused. It's not 100% reliable (e.g. the list of packages in the manifest could depend on the phase of the moon if (current-time) is used) (is that what you mean by ‘different paths’ and ‘dynamic arguments’?), but it should cover the vast majority of cases. I don't know a non-artifical situation where ‘custom macros’ are a problem -- do you know an example in the wild where this dependency tracking scheme would fail to work? > We could get an approximation for common uses by recognizing special > forms as you suggest. But it’s just that, an approximation. It's an approximation, sure, but it seems to be a quite accurate approximation to me. And it's not really recognising special forms that I'm suggesting, but rather modifying the macros behind these forms to inform "guix shell" of what the dependencies are. > In such situations, I err on the side of not even trying. The added > complexity for a flaky result doesn’t pay off to me. I prefer to be > upfront, document limitations, and let users handle them as they see > fit. About complexity: there's some extra code, sure, but it doesn't seem complex to me. To track dependencies in , I only needed to add 'notice-dependency' and some other code to (guix build compile), and some code to build-aux/compile-all.scm to save/load the dependency information to/from the builddir and to also check the mtime of dependencies. Does it still seem flaky to you, after my explanation on how the dependency information would be determined? Being upfront, documenting limitations, and having a ‘force rebuild flag’ (is that what you mean by ‘letting users handle them as they see fit’?) (possibly also documenting 'notice-dependency') is not incompatible with the automatic dependency tracking. More abstractly, this seems more like a ‘Perfect is the enemy of the good’ situation. While I would prefer 'perfect' above 'good', and the automatic dependency tracking certainly isn't 'perfect', it does seem 'good' to me, and perfection appears to be impossible and there's the ‘force rebuild flag’ as an escape hatch, so I believe 'good-but-not-perfect' to be acceptable here, as long as it is documented in the manual that there are some limitation on what dependencies are automatically tracked. Greetings, Maxime.