Subject: bug#40971: Updating built-in packages that seq depends on is broken due to a bug in package.el

From: Clément Pit-Claudel <[email-masked]>
Date: Wed, 29 Apr 2020 15:47:14 -0400

Hi all,

This is a follow-up to the thread at https://lists.gnu.org/archive/html/emacs-devel/2020-04/msg01974.html.

`seq` and a few other packages are distributed through ELPA but also built-in.  In theory, this makes it possible to upgrade these packages after installing Emacs.

However, because implementation of package.el requires `seq', seq is loaded very early after starting Emacs — so early, in fact, that when package-initialize adds elpa/ directories to the load-path, seq is already loaded.

As a result, even if a newer version of seq is installed, it is never loaded, because the built-in seq library is already loaded.  Concretely, this means that it isn't possible to update seq.

I got bitten by this while developing a package that uses seq-sort-by, which isn't available in Emacs 25's built-in seq.  I added a dependency on seq 2.14, which does contain seq-sort-by, but that doesn't help: package.el does install the dependency, but it is never loaded.

I believe the bug lies in `package--list-loaded-files'.  Theoretically, after upgrading a package, it should be reloaded if it was already in the load history.  But the code to find previously loaded files uses file-name-sans-extension to match load-history entries to library file names.  This fails because seq.el is installed as seq.el.gz, so file-name-sans-extension returns seq.el, not seq.  

The correct approach would likely be to use get-load-suffixes, although it might also be enough to use find-library--from-load-history, which seems to already do most of the needed work.

Clément.