Hi Philip,
>To clarify the terminology, ELPA/MELPA are package archives that take
>usually take a source code checkout and prepare packages. For each
>package, they might specify or infer a main file or some other source of
>information to get the metadata like package dependencies, but they will
>also throw out all other files that are not part of the package.
Yes, what I meant was how MELPA builds. Wile it does put the files in load-path as well, but it ignores the "Package-Requires:" header for inferring package dependencies in all other files, but the main lisp file, and it can of course run into the problem you are trying to avoid but I would argue the MELPA's approach is better (see below).
>Package-vc is kind of like that, in that it is an independent
>implementation of ELPA's build scripts, but as the goal is to explicitly
>load the directory with the source code checkout we don't remove the
>"unrelated" files (that is one of the reasons we also advise against
>combining multiple packages in the same directory).
I understand the advice against multiple packages in the same repo, but a lot of the times a package has its own extensions (that may have a different set of dependencies) and it is much easier to have all the extensions in one repo for mmanaging the repo becuase it's easier to keep consistencies and also it is much easier to track issues and etc. when all of the related files are in the same repo.
I am not sure I follow your argument. The other files are still
installed and loadable, but broken since they are missing dependencies.
That is the state I wish to avoid.There is a bug in the dependency resolution, but that is a separate
issue IMO.
Yes, but this is exactly what MELPA does. It puts those files in the laod path but ignores the "Package-Requires:" header when it comes to installing package dependencies. The reason I think this makes more sense is because the dependencies are defined at the package level, and therefore they should be defined once in the main file, otherwise, the dependencies from all different files need to be compared at build time and the common required set of dependecies and versions need to be inferred. For example, as I showed with the embark example in my previous email, different files can require the same package but with a different minimum version. Right now package-vc-installl is pulling different versions (in different folders under "/elpa" directory) and I am not even sure in what order the different versions of embark are loaded. This can lead to much bigger problems than the issue you are trying to avoid (a.k.a. some featuers not working because the dependency is not available).
Of course you can argue that there is a bug in dependency resolution, but to me the easiest solution would be to do what MELPA does, inly use the main file for inferring dependencies. Alternatively, if we add features such as "include-files", "ignore-files", "clone-depth", etc, as options for the recipe and do a much smarter job in inferring dependencies (e.g. do not create self-dependency, only install the highest required version of any package, …), then there will be other ways to avoid this kind of issue.
Best,
Armin
Philip Kaludercic @ 2025-02-17 17:24 :
Armin Darvish <armindarvish@gmail.com> writes:
Hi Philip,
I am on emacs versio 30.0.93. and I tried an init file like this:
,-—
;;; init.el — -- lexical-binding: t; -- (require 'package) (package-initialize) (package-vc-install '(consult-omni :url "https://github.com/armindarvish/consult-omni" :main-file "consult-omni.el")) `-—
Yes, I can reproduce the issue now, thanks!
The issue we have to keep in mind is that since we add the repository
directly to `load-path', all the files can be used no matter what
"specific" file you might intend to use. To retain usability and not
have unexpected function calls fail, we rather install all the
dependencies.My understanding was that packages are either a single-file package,
which won't have this problem, or a multi-file package, in which case,
the "Package-Requires: " header should only be in the main file and
not the extra lisp files. As far as I know, other package managing
systems (like MELPA) don't use hte "Package-Requires: " header in the
files other than the main lisp file either.To clarify the terminology, ELPA/MELPA are package archives that take
usually take a source code checkout and prepare packages. For each
package, they might specify or infer a main file or some other source of
information to get the metadata like package dependencies, but they will
also throw out all other files that are not part of the package.Package-vc is kind of like that, in that it is an independent
implementation of ELPA's build scripts, but as the goal is to explicitly
load the directory with the source code checkout we don't remove the
"unrelated" files (that is one of the reasons we also advise against
combining multiple packages in the same directory).Note that, we can still
add all the files in the repo to load-path, but when automatically
making the "define-package" declaration in a "package-pkg.el" file,
the dependencies should be inferred form the main file and not others,
otherwise there will be weird looped dependencies.I am not sure I follow your argument. The other files are still
installed and loadable, but broken since they are missing dependencies.
That is the state I wish to avoid.There is a bug in the dependency resolution, but that is a separate
issue IMO.For example, in
case of embark with the following init file:,-—
;;; init.el — -- lexical-binding: t; -- (require 'package) ;; Adds the Melpa archive to the list of available repositories (setq package-archives '(("elpa" . "https://elpa.gnu.org/packages/") ("melpa-stable" . "https://stable.melpa.org/packages/") ("melpa" . "https://melpa.org/packages/"))) ;; Initializes the package infrastructure (package-initialize) (package-refresh-contents) (use-package embark :vc (:url "https://github.com/oantolin/embark")) `-—
I can see multiple versions of embark being pulled from MELPA becuase
of looped depenncies in the embark-pkg.el file automatically created
by package-vc-install. Here is the contents of that file:,-—
(define-package "embark" "1.1" "No description available." '((emacs "25.1") (embark "0.9") (avy "0.5") (emacs "27.1") (compat "29.1.4.0") (embark "1.0") (consult "1.0")) :kind vc :commit "195add1f1ccd1059472c9df7334c97c4d155425e") `-—
Note that while this installs just fine, it is pulling 3 different
versions of embark from MELPA becuase of this inferred looped
self-dependency from differnt *.el files in embark repo, which is not
the intended behavior by the author of that package.That said, I agree with the point that there can potentially be other
solutions like terminating properly as you said or even better would
be detecting and ignoring any self-dependency. I cannot think of any
scenario where a package should depend on its own or on an older
version of its own.I agree. I'll try to find a solution to the issue by fixing the
recursion issue and update this thread.>
Philip Kaludercic @ 2025-02-16 18:53 :
Armin Darvish <armindarvish@gmail.com> writes:
Hello,
I have noticed that with vc repositories that contain multiple related single-file
packages, package-vc-install creates looped self-dependency that can cause errors.For example, trying to install the repository:
https://github.com/armindarvish/consult-omni
will result in "Lisp nesting exceeds ‘max-lisp-eval-depth’: 1601" error. This is
because currently package-vc-install tries to read all ".el" files in the root
directory to get the dependencies and build the "define-package" declaration in
consult-omni-pkg.el. This is not compatible with repositories that have multiple
single-file packages in the root directory. Instead, the dependencies should be
inferred from the main lisp file only. This will be safe with multi-file packages
as well because the convention is to have the "Package-Requires:" header only in
the main lisp file and not the additional lisp files.What version of Emacs are you using? Installing your repository doesn't
raise any error when I try to do so.There are other packages that have multiple single-file packages as well, for
example, https://github.com/oantolin/embark includes embark and embark-consult in
the root directory. Currently, installing embark with package-vc-install causes
several different versions of embark being downlaoded because the dependencies are
read from all of those files even though they are meant to be separate packages.The issue we have to keep in mind is that since we add the repository
directly to `load-path', all the files can be used no matter what
"specific" file you might intend to use. To retain usability and not
have unexpected function calls fail, we rather install all the
dependencies. The recursion error above hints at some programming
issue, where we don't terminate properly.>
Best Regards,
Armin Darvish–
[www.armindarvish.com]
[www.armindarvish.com] https://www.armindarvish.com/
–
Best Regards,