Subject: bug#72300: project.el: detect newly created project contained within another

From: Dmitry Gutov <[email-masked]>
Date: Mon, 30 Sep 2024 04:35:07 +0300

On 13/08/2024 16:31, Ship Mints wrote:
> A good step is awareness for users and invalidating the cache via 
> project-forget methods is a good idea. 

For posterity: making it a method of an object doesn't seem to work 
since the directory might have changed enough that it's not possible to 
recreate the project instance to call it on.

 > I'd also offer a direct function
 > to invoke to invalidate the cache for programmatic use.

project-forget-project is a Lisp function which could be used both 
interactively and from code. Attached is a PoC/WIP patch which adds 
cache invalidation this way. Not entirely happy with the approach yet, 
but I'd like to checkpoint it here.

> vc caching, longer term, may need to consider a few more complex use 
> cases such as git repos with both submodules that are considered 
> extensions of the base project and submodules which are not. A concrete 
> example I see often is a "mono repo" structure with core server and 
> library code but with web and mobile front end code in submodules that 
> are treated as part of the project proper BUT with submodules for 
> vendor/third-party code that are not. A question here would be which 
> parts of the tree belong to which cached vc root.

Very tangentially, we've just added support for speeding up file listing 
for projects using "sparse index". That one's for a different structure 
and repos above certain size, though.

> Another use case I see is working on many unrelated projects/repos 
> across a variety of clients all in the same Emacs session and with 
> perhaps 100+ buffers/files open (as I pretty much have right now), a 17- 
> element cache won't be sufficient? Should the cache be for parent 
> directories and not for file names? With files, it gets full fast. Mine 
> is full right now with files most of which share the same repo root and 
> some that don't.

These are all internal details, but:

* 17 is the default size, then it grows,
* project-try-vc caches only directories, using its own key,
* The rest of the data you're seeing is for files and their VC statuses, 
that's separate info.

But indeed it's a bit too much intermixing, project-vc's cache should 
move to its own structure first. Stay tuned.

> I have wondered whether an implementation would be 
> better as directory variables? Cache invalidation without timestamps 
> on .dir-locals.el files remain the same but directory variable treatment 
> might be more natural to Emacs users?

Directory variables are a thing, but not in the same sense that 
file-local variables are, in particular there is no object to save a 
"directory local" to at runtime.

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 599a350e5ce..9cc70335afc 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -193,6 +193,10 @@ project-find-functions
   'project-current-directory-override
   "29.1")
 
+(defvar project-forget-functions (list #'project-vc-forget-directory)
+  "Special hook to clear cache for a directory tree.
+Each function on this hook is passed a string, the directory file name.")
+
 (defvar project-current-directory-override nil
   "Value to use instead of `default-directory' when detecting the project.
 When it is non-nil, `project-current' will always skip prompting too.")
@@ -599,6 +603,13 @@ project-try-vc
           (vc-file-setprop dir 'project-vc project)
           project))))
 
+(defun project-vc-forget-directory (root)
+  (obarray-map
+   (lambda (sym)
+     (when (string-prefix-p root (symbol-name sym))
+       (vc-file-setprop (symbol-name sym) 'project-vc nil)))
+   vc-file-prop-obarray))
+
 (defun project--submodule-p (root)
   ;; XXX: We only support Git submodules for now.
   ;;
@@ -1875,6 +1886,7 @@ project-forget-project
 PROJECT-ROOT is the root directory of a known project listed in
 the project list."
   (interactive (list (funcall project-prompter)))
+  (run-hook-with-args 'project-forget-functions project-root)
   (project--remove-from-project-list
    project-root "Project `%s' removed from known projects"))
 
@@ -1999,6 +2011,8 @@ project-remember-projects-under
 projects."
   (interactive "DDirectory: \nP")
   (project--ensure-read-project-list)
+  ;; Clear the cache for DIR and below, in case of prior visit.
+  (run-hook-with-args 'project-forget-functions dir)
   (let ((dirs (if recursive
                   (directory-files-recursively dir "" t)
                 (directory-files dir t)))