Subject: bug#41242: Port feature/native-comp to Windows - Reduce the number of files probed when finding a lisp file.

From: Nicolas Bértolo <[email-masked]>
Date: Sun, 31 May 2020 19:41:36 -0300

Last night I came up with a fresh idea to properly solve the problem with slow
loads. Considering that it happens when `load-path` contains hundreds of
directories I thought that a caching mechanism would help.

This "cache" would consist of a mapping of files that would get probed when
(load "foo") runs. It would be implemented as a hash table. This could get
stored in `package-user-dir` and package.el would be in charge of updating it
when installing or removing packages, just like autoloads.

The contents of this hash table could be something like this:

The directory company-20200525.101 could have a file load-cache.el with:

company -> ("company-20200525.101/eln-hash/company.eln"
            "company-20200525.101/company.el"
            "company-20200525.101/company.elc")
[...]

The directory helm-20200517.509 could have a file load-cache.el with:

helm -> ("helm-20200517.509/eln-hash/helm.eln"
         "helm-20200517.509/helm.el"
         "helm-20200517.509/helm.elc")
[...]

When `load-path` changes we could update the in-memory hash table by loading all
the load-cache.el files.

Then, when (require 'foo) runs, the loading code could look at the hash
table and only fopen() the files associated with the feature we are loading.
This would reduce the number of calls to fopen() from thousands to ~3 in the
worst case.

Of course, this feature would be disabled by default.

What do you think?