GNU bug report logs - #56877
29.0.50; Built-in packages are not always package-installed-p

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 56877 in the body.
You can then email your comments to 56877 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Matt Armstrong <matt <at> rfc20.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.0.50; Built-in packages are not always package-installed-p
Date: Mon, 01 Aug 2022 13:15:25 -0700

Before `package--initialized' is set non-nil a call to
(package-installed-p PACKAGE) returns nil for symbols naming built-in
packages. After `package--initialized' is non-nil the same call returns
non-nil.

I discovered this by accident when I figured out this use-package form
caused Emacs to contact GNU ELPA during startup:

(use-package project
  :ensure t
  ;; stuff)

For the above use-package calls (package-installed-p 'project), which
returns nil.  Use-package then attempts to install it using package.el,
which causes network activity, etc., and then package.el decides that it
is already installed.  This repeats each time I start Emacs.

There is code in `package-installed-p' that consults
`package-activated-list' when `package--initalized' is non-nil.  This is
the code path I am exercising.  We also have this FIXME:

(defvar package-activated-list nil
  ;; FIXME: This should implicitly include all builtin packages.
  "List of the names of currently activated packages.")

I have a fix prepared, which I will send once i have a bug number.


Configured using:
 'configure --with-native-compilation --with-pgtk'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP
NOTIFY INOTIFY PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS XIM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: ELisp/l

Minor modes in effect:
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  auto-revert-mode: t
  bug-reference-prog-mode: t
  msb-mode: t
  display-time-mode: t
  shell-dirtrack-mode: t
  auto-insert-mode: t
  keyfreq-autosave-mode: t
  keyfreq-mode: t
  savehist-mode: t
  icomplete-vertical-mode: t
  icomplete-mode: t
  editorconfig-mode: t
  which-key-mode: t
  electric-pair-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t
  auto-save-visited-mode: t

Load-path shadows:
~/env/elisp/ol-notmuch hides /home/matt/.config/emacs/elpa/ol-notmuch-20220428.1337/ol-notmuch
/home/matt/.config/emacs/elpa/transient-20220717.1713/transient hides /home/matt/git/emacs-build/lisp/transient

Features:
(ert ewoc debug backtrace shadow sort mail-extr magit-extras face-remap
magit-submodule magit-obsolete magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log which-func imenu
magit-diff git-commit log-edit magit-core magit-autorevert magit-margin
magit-transient magit-process with-editor magit-mode transient magit-git
magit-base magit-section crm dash compat-27 compat-26 compat make-mode
emacsbug smerge-mode diff add-log vc-annotate vc-hg vc-bzr vc-src
vc-sccs vc-svn vc-cvs vc-rcs log-view pcvs-util whitespace vc sh-script
smie executable autorevert cl-print package-x dabbrev misearch
multi-isearch pulse color xref vc-git diff-mode vc-dispatcher
bug-reference shortdoc benchmark help-fns radix-tree textsec uni-scripts
idna-mapping ucs-normalize uni-confusable textsec-check protbuf msb time
copyright editorconfig-core editorconfig-core-handle
editorconfig-fnmatch cus-edit pp cus-start cus-load org-element avl-tree
generator ol-w3m ol-rmail ol-mhe ol-irc ol-info org-habit org-agenda
org-refile ol-gnus nnselect gnus-art mm-uu mml2015 mm-view mml-smime
smime dig gnus-sum gnus-group gnus-undo gnus-start gnus-dbus dbus
gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo parse-time
gnus-spec gnus-int gnus-range gnus-win ol-eww eww xdg url-queue
thingatpt shr pixel-fill kinsoku url-file url-dired svg xml dom mm-url
gnus nnheader range wid-edit ol-doi org-link-doi ol-docview doc-view
filenotify jka-compr image-mode exif ol-bibtex ol-bbdb server dirtrack
ob-shell shell ob-ruby ob-python python ob-dot org-protocol org ob
ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src
ob-comint org-pcomplete pcomplete comint ansi-color ring org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp
ob-core ob-eval org-table oc-basic bibtex iso8601 org-keys oc
org-loaddefs find-func cal-menu calendar cal-loaddefs ol-notmuch ol
org-compat org-macs format-spec skeleton autoinsert advice keyfreq
project finder-inf mm-archive message sendmail yank-media dired
dired-loaddefs rfc822 mml mml-sec epa derived gnus-util
text-property-search time-date mailabbrev gmm-utils mailheader mm-decode
mm-bodies mm-encode mail-utils gnutls network-stream url-cache url-http
url-auth mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums
mail-prsvr url-gw nsm puny epg rfc6068 epg-config edmacro kmacro
savehist icomplete comp comp-cstr warnings icons rx editorconfig
modus-operandi-theme modus-themes which-key package browse-url url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source eieio eieio-core password-cache json subr-x map byte-opt
url-vars cl-macs gv cl-extra help-mode cl-seq elec-pair use-package
use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core cl-loaddefs
cl-lib bytecomp byte-compile cconv info bazel-autoloads
clang-format+-autoloads clang-format-autoloads cmake-mode-autoloads
d-mode-autoloads debbugs-autoloads editorconfig-autoloads
eglot-autoloads elpy-autoloads company-autoloads
exec-path-from-shell-autoloads flymake-ruby-autoloads
flymake-easy-autoloads flymake-yamllint-autoloads go-mode-autoloads
google-c-style-autoloads graphviz-dot-mode-autoloads
highlight-indentation-autoloads magit-autoloads git-commit-autoloads
magit-section-autoloads dash-autoloads markdown-mode-autoloads
meson-mode-autoloads modus-themes-autoloads nixpkgs-fmt-autoloads
ol-notmuch-autoloads notmuch-autoloads orderless-autoloads
org-drill-autoloads persist-autoloads pylint-autoloads pyvenv-autoloads
s-autoloads shfmt-autoloads reformatter-autoloads transient-autoloads
use-package-autoloads bind-key-autoloads vertico-autoloads
which-key-autoloads with-editor-autoloads compat-autoloads
yaml-mode-autoloads yasnippet-autoloads rmc iso-transl tooltip eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/pgtk-win pgtk-win term/common-win pgtk-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
faces cus-face macroexp files window text-properties overlay sha1 md5
base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 1041704 138279)
 (symbols 48 41342 108)
 (strings 32 237540 12847)
 (string-bytes 1 6538892)
 (vectors 16 112331)
 (vector-slots 8 2502178 111733)
 (floats 8 749 1306)
 (intervals 56 26923 3648)
 (buffers 992 42))

Message #10 received at 56877 <at> debbugs.gnu.org (full text, mbox):

From: Matt Armstrong <matt <at> rfc20.org>
To: 56877 <at> debbugs.gnu.org
Subject: Re: bug#56877: 29.0.50; Built-in packages are not always
 package-installed-p
Date: Mon, 01 Aug 2022 14:02:00 -0700

[Message part 1 (text/plain, inline)]

Matt Armstrong <matt <at> rfc20.org> writes:

> I have a fix prepared, which I will send once i have a bug number.

Now attached:

[0001-Consider-built-in-packages-to-be-installed.patch (text/x-diff, inline)]

From b9431eee1bc141a036aa42541f68b0c07474bc40 Mon Sep 17 00:00:00 2001
From: Matt Armstrong <matt <at> rfc20.org>
Date: Mon, 1 Aug 2022 11:18:37 -0700
Subject: [PATCH] Consider built-in packages to be installed

With this change (package-installed-p PACKAGE nil) evaluates to the
same thing before and after package initialization.

* lisp/emacs-lisp/package.el (package-installed-p): Check for built-in
packages before initialization. (bug#56877)
---
 lisp/emacs-lisp/package.el            |  5 ++++-
 test/lisp/emacs-lisp/package-tests.el | 15 +++++++++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index df70f908da..482de52f85 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -2085,7 +2085,10 @@ package-installed-p
          package-activated-list)
     ;; We used the quickstart: make it possible to use package-installed-p
     ;; even before package is fully initialized.
-    (memq package package-activated-list))
+    (or
+     (memq package package-activated-list)
+     ;; Also check built-in packages.
+     (package-built-in-p package min-version)))
    (t
     (or
      (let ((pkg-descs (cdr (assq package (package--alist)))))
diff --git a/test/lisp/emacs-lisp/package-tests.el b/test/lisp/emacs-lisp/package-tests.el
index d7a55998c2..75c3f061ab 100644
--- a/test/lisp/emacs-lisp/package-tests.el
+++ b/test/lisp/emacs-lisp/package-tests.el
@@ -638,6 +638,21 @@ package-test-update-archives/ignore-nil-entry
       (package-refresh-contents)
       (should (equal (length package-archive-contents) 2)))))
 
+(ert-deftest package-test-package-installed-p ()
+  "Foo"
+  (with-package-test ()
+    ;; Verify that `package-installed-p' evaluates true for a built-in
+    ;; package, in this case `project', before package initialization.
+    (should (not package--initialized))
+    (should (package-installed-p 'project nil))
+    (should (not (package-installed-p 'imaginary-package nil)))
+
+    ;; The results don't change after package initialization.
+    (package-initialize)
+    (should package--initialized)
+    (should (package-installed-p 'project nil))
+    (should (not (package-installed-p 'imaginary-package nil)))))
+
 (ert-deftest package-test-describe-package ()
   "Test displaying help for a package."
 
-- 
2.35.1

Message #13 received at 56877 <at> debbugs.gnu.org (full text, mbox):

From: Matt Armstrong <matt <at> rfc20.org>
To: 56877 <at> debbugs.gnu.org
Subject: Re: bug#56877: 29.0.50; Built-in packages are not always
 package-installed-p
Date: Mon, 01 Aug 2022 14:26:57 -0700

[Message part 1 (text/plain, inline)]

Matt Armstrong <matt <at> rfc20.org> writes:

> Matt Armstrong <matt <at> rfc20.org> writes:
>
>> I have a fix prepared, which I will send once i have a bug number.
>
> Now attached:

Second try.  Replaced "Foo" with a real docstring!

[0001-Consider-built-in-packages-to-be-installed.patch (text/x-diff, inline)]

From f66a0e6bf84f10d2688b2892a08e2d090e5bd6c7 Mon Sep 17 00:00:00 2001
From: Matt Armstrong <matt <at> rfc20.org>
Date: Mon, 1 Aug 2022 11:18:37 -0700
Subject: [PATCH] Consider built-in packages to be installed

With this change (package-installed-p PACKAGE nil) evaluates to the
same thing before and after package initialization.

* lisp/emacs-lisp/package.el (package-installed-p): Check for built-in
packages before initialization. (bug#56877)
---
 lisp/emacs-lisp/package.el            |  5 ++++-
 test/lisp/emacs-lisp/package-tests.el | 15 +++++++++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index df70f908da..482de52f85 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -2085,7 +2085,10 @@ package-installed-p
          package-activated-list)
     ;; We used the quickstart: make it possible to use package-installed-p
     ;; even before package is fully initialized.
-    (memq package package-activated-list))
+    (or
+     (memq package package-activated-list)
+     ;; Also check built-in packages.
+     (package-built-in-p package min-version)))
    (t
     (or
      (let ((pkg-descs (cdr (assq package (package--alist)))))
diff --git a/test/lisp/emacs-lisp/package-tests.el b/test/lisp/emacs-lisp/package-tests.el
index d7a55998c2..b903cd781b 100644
--- a/test/lisp/emacs-lisp/package-tests.el
+++ b/test/lisp/emacs-lisp/package-tests.el
@@ -638,6 +638,21 @@ package-test-update-archives/ignore-nil-entry
       (package-refresh-contents)
       (should (equal (length package-archive-contents) 2)))))
 
+(ert-deftest package-test-package-installed-p ()
+  "Test package-installed-p before and after package initialization."
+  (with-package-test ()
+    ;; Verify that `package-installed-p' evaluates true for a built-in
+    ;; package, in this case `project', before package initialization.
+    (should (not package--initialized))
+    (should (package-installed-p 'project nil))
+    (should (not (package-installed-p 'imaginary-package nil)))
+
+    ;; The results don't change after package initialization.
+    (package-initialize)
+    (should package--initialized)
+    (should (package-installed-p 'project nil))
+    (should (not (package-installed-p 'imaginary-package nil)))))
+
 (ert-deftest package-test-describe-package ()
   "Test displaying help for a package."
 
-- 
2.35.1

Message #16 received at 56877 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Matt Armstrong <matt <at> rfc20.org>
Cc: 56877 <at> debbugs.gnu.org
Subject: Re: bug#56877: 29.0.50; Built-in packages are not always
 package-installed-p
Date: Tue, 02 Aug 2022 12:14:29 +0200

Matt Armstrong <matt <at> rfc20.org> writes:

> Second try.  Replaced "Foo" with a real docstring!

Thanks; pushed to Emacs 29 now.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.