Subject: bug#76325: 30.0.93; package-vc-install fails with repositories that contain multiple single-file packages

From: Armin Darvish <[email-masked]>
Date: Sun, 16 Feb 2025 16:46:12 -0800

Hi Philip,

I am on emacs versio 30.0.93. and I tried an init file like this:
,----
| ;;; init.el --- -*- lexical-binding: t; -*-
| 
| (require 'package)
| (package-initialize)
| (package-vc-install
|   '(consult-omni :url "https://github.com/armindarvish/consult-omni"
|          :main-file "consult-omni.el"))
`----

> The issue we have to keep in mind is that since we add the repository
> directly to `load-path', all the files can be used no matter what
> "specific" file you might intend to use.  To retain usability and not
> have unexpected function calls fail, we rather install all the
> dependencies.

My understanding was that packages are either a single-file package, which won't have this problem, or a multi-file package, in which case, the "Package-Requires: " header should only be in the main file and not the extra lisp files. As far as I know, other package managing systems (like MELPA) don't use hte "Package-Requires: " header in the files other than the main lisp file either. Note that, we can still add all the files in the repo to load-path, but when automatically making the "define-package" declaration in a "package-pkg.el" file, the dependencies should be inferred form the main file and not others, otherwise there will be weird looped dependencies. For example, in case of embark with the following init file:

,----
| ;;; init.el --- -*- lexical-binding: t; -*-
| 
| (require 'package)
| ;; Adds the Melpa archive to the list of available repositories
|  (setq package-archives
|        '(("elpa" . "https://elpa.gnu.org/packages/")
|          ("melpa-stable" . "https://stable.melpa.org/packages/")
|          ("melpa" . "https://melpa.org/packages/")))
| ;; Initializes the package infrastructure
| (package-initialize)
| (package-refresh-contents)
| 
| (use-package embark
| :vc (:url "https://github.com/oantolin/embark"))
| 
`----

I can see multiple versions of embark being pulled from MELPA becuase of looped depenncies in the embark-pkg.el file automatically created by package-vc-install. Here is the contents of that file:

,----
| (define-package "embark" "1.1" "No description available." '((emacs "25.1") (embark "0.9") (avy "0.5") (emacs "27.1") (compat "29.1.4.0") (embark "1.0") (consult "1.0")) :kind vc :commit "195add1f1ccd1059472c9df7334c97c4d155425e")
`----

Note that while this installs just fine, it is pulling 3 different versions of embark from MELPA becuase of this inferred looped self-dependency from differnt *.el files in embark repo, which is not the intended behavior by the author of that package.

That said, I agree with the point that there can potentially be other solutions like terminating properly as you said or even better would be detecting and ignoring any self-dependency. I cannot think of any scenario where a package should depend on its own or on an older version of its own.

Philip Kaludercic @ 2025-02-16 18:53 :

> Armin Darvish <[email-masked]> writes:
>
>> Hello,
>>
>> I have noticed that with vc repositories that contain multiple related single-file
>> packages, package-vc-install creates looped self-dependency that can cause errors. 
>>
>> For example, trying to install the repository: 
>> <https://github.com/armindarvish/consult-omni> 
>> will result in "Lisp nesting exceeds ‘max-lisp-eval-depth’: 1601" error. This is 
>> because currently package-vc-install tries to read all ".el" files in the root 
>> directory to get the dependencies and build the "define-package" declaration in
>> consult-omni-pkg.el. This is not compatible with repositories that have multiple
>> single-file packages in the root directory. Instead, the dependencies should be 
>> inferred from the main lisp file only. This will be safe with multi-file packages
>> as well because the convention is to have the "Package-Requires:" header only in 
>> the main lisp file and not the additional lisp files.
>
> What version of Emacs are you using?  Installing your repository doesn't
> raise any error when I try to do so.
>
>> There are other packages that have multiple single-file packages as well, for 
>> example, <https://github.com/oantolin/embark> includes embark and embark-consult in
>> the root directory. Currently, installing embark with package-vc-install causes
>> several different versions of embark being downlaoded because the dependencies are
>> read from all of those files even though they are meant to be separate packages.
>
> The issue we have to keep in mind is that since we add the repository
> directly to `load-path', all the files can be used no matter what
> "specific" file you might intend to use.  To retain usability and not
> have unexpected function calls fail, we rather install all the
> dependencies.  The recursion error above hints at some programming
> issue, where we don't terminate properly.
>
>>
>> Best Regards,
>> *Armin Darvish*
>>
>> --
>> ------------------------------------------------------------------------
>> [www.armindarvish.com]
>>
>>
>> [www.armindarvish.com] <https://www.armindarvish.com/>

-- 

Best Regards,

*Armin Darvish*
------------------------------------------------------------------------
[www.armindarvish.com]

[www.armindarvish.com] <https://www.armindarvish.com/>