GNU bug report logs - #76264
[GNU ELPA] Use lisp-data-mode in generated *-pkg.el files

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 76264 in the body.
You can then email your comments to 76264 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Cc: Philip Kaludercic <philipk <at> posteo.net>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: [GNU ELPA] Use lisp-data-mode in generated *-pkg.el files
Date: Thu, 13 Feb 2025 04:24:21 -0800

Severity: wishlist

The *-pkg.el files are not source files that should be executed, but
Lisp data files.  This has been the source of much confusion.

For example, it is currently completely confused in the lispref manual,
which talks about `define-package' as a function, despite the fact that
it has never been working like that and is now obsolete.  (I'm working
on a patch.)

I think we should install the following to start clearing up some of the
confusion.

Thoughts?

diff --git a/elpa-admin.el b/elpa-admin.el
index c127f50dcc..2ccfe35f81 100644
--- a/elpa-admin.el
+++ b/elpa-admin.el
@@ -1516,7 +1516,7 @@ Rename DIR/ to PKG-VERS/, and return the descriptor."
 	(print-length nil))
     (elpaa--temp-file pkg-file)
     (write-region
-     (concat (format ";; Generated package description from %s.el
-*- no-byte-compile: t -*-\n"
+     (concat (format ";; Generated package description from %s.el
-*- mode: lisp-data; no-byte-compile: t -*-\n"
 		     name)
 	     (prin1-to-string
               (pcase-let ((`(,version ,desc ,requires ,extras)

Message #10 received at 76264-done <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 76264-done <at> debbugs.gnu.org
Subject: Re: [GNU ELPA] Use lisp-data-mode in generated *-pkg.el files
Date: Thu, 13 Feb 2025 07:44:11 -0800

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>> I think we should install the following to start clearing up some of the
>> confusion.
>
> LGTM,

Thanks, pushed, closing.

Message #13 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philip Kaludercic <philipk <at> posteo.net>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: bug-gnu-emacs <at> gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: [GNU ELPA] Use lisp-data-mode in generated *-pkg.el files
Date: Fri, 14 Feb 2025 15:45:54 +0000

Stefan Kangas <stefankangas <at> gmail.com> writes:

> Severity: wishlist
>
> The *-pkg.el files are not source files that should be executed, but
> Lisp data files.  This has been the source of much confusion.
>
> For example, it is currently completely confused in the lispref manual,
> which talks about `define-package' as a function, despite the fact that
> it has never been working like that and is now obsolete.  (I'm working
> on a patch.)

I do think that it used to be a function, but that was changed a number
of years ago.

> I think we should install the following to start clearing up some of the
> confusion.
>
> Thoughts?

The only negative to this patch is that users on Emacs 27 or before will
get a warning about an invalid mode, but otherwise I think that this is
a good idea.

> diff --git a/elpa-admin.el b/elpa-admin.el
> index c127f50dcc..2ccfe35f81 100644
> --- a/elpa-admin.el
> +++ b/elpa-admin.el
> @@ -1516,7 +1516,7 @@ Rename DIR/ to PKG-VERS/, and return the descriptor."
>  	(print-length nil))
>      (elpaa--temp-file pkg-file)
>      (write-region
> -     (concat (format ";; Generated package description from %s.el
> -*- no-byte-compile: t -*-\n"
> +     (concat (format ";; Generated package description from %s.el
> -*- mode: lisp-data; no-byte-compile: t -*-\n"
>  		     name)
>  	     (prin1-to-string
>                (pcase-let ((`(,version ,desc ,requires ,extras)

Message #20 received at 76264 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <basil <at> contovou.net>
To: Philip Kaludercic <philipk <at> posteo.net>
Cc: Stefan Kangas <stefankangas <at> gmail.com>, 76264 <at> debbugs.gnu.org,
 monnier <at> iro.umontreal.ca
Subject: Re: bug#76264: [GNU ELPA] Use lisp-data-mode in generated *-pkg.el
 files
Date: Wed, 19 Mar 2025 13:10:21 +0100

[Message part 1 (text/plain, inline)]

Philip Kaludercic [2025-02-14 15:45 +0000] wrote:

> The only negative to this patch is that users on Emacs 27 or before will
> get a warning about an invalid mode, but otherwise I think that this is
> a good idea.

Here I see more than just a warning, in Emacs versions 24 through 27:

0. cd "$(mktemp -d)"
1. HOME="$PWD" XDG_CONFIG_HOME="$PWD" emacs-27.2
2. M-: (setq package-check-signature nil) RET
3. M-x toggle-debug-on-error RET
4. M-x package-install RET dash RET

I attach backtraces from Emacs 24.5 and 27.2.

What I'm not sure about is why only some packages result in an error.
I'm guessing elpa.gnu.org is caching -pkg.el files of packages that have
not been updated in the meantime.

I assume we want to continue supporting older Emacs versions, so does
the elpa.git patch need reverting (plus manual intervention to revert
any cached -pkg.el files), or am I missing something / is there a better
solution?

Thanks,
-- 
Basil

[emacs-24.5-bt.txt (text/plain, attachment)]

[emacs-27.2-bt.txt (text/plain, attachment)]

[Message part 4 (text/plain, inline)]

In GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.18.2, Xaw3d scroll bars)
 of 2024-11-29 built on tais
Windowing system distributor 'The X.Org Foundation', version 11.0.12101016
System Description: Debian GNU/Linux trixie/sid

Recent messages:
Saving file /tmp/tmp.ScBnNSmPQL/.emacs...
Wrote /tmp/tmp.ScBnNSmPQL/.emacs
Saving file /tmp/tmp.ScBnNSmPQL/.emacs...
Wrote /tmp/tmp.ScBnNSmPQL/.emacs
Contacting host: elpa.gnu.org:443
Parsing tar file...done
Extracting...done
  INFO     Scraping files for dash-autoloads.el... 
Generating autoloads for dash-functional.el...done
Entering debugger...

Configured using:
 'configure --prefix=/home/blc/.local --program-suffix=-27.2-lucid
 --with-file-notification --with-cairo --with-x-toolkit=lucid --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB
NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT
LIBOTF ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS
LIBSYSTEMD JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: en_IE.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Debugger

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug sendmail cl-extra help-fns cl-print
debug backtrace help-mode find-func bug-reference autoload radix-tree
lisp-mnt tar-mode arc-mode archive-mode cus-edit wid-edit mm-archive
message dired dired-loaddefs format-spec rfc822 mml mml-sec epa derived
epg epg-config gnus-util rmail rmail-loaddefs text-property-search
time-date mailabbrev gmm-utils mailheader mm-decode mm-bodies mm-encode
mail-utils gnutls network-stream url-http mail-parse rfc2231 rfc2047
rfc2045 mm-util ietf-drums mail-prsvr url-gw nsm rmc puny url-cache
url-auth url url-proxy url-privacy url-expand url-methods url-history
url-cookie url-domsuf url-util mailcap finder-inf package easymenu
browse-url url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache json subr-x map url-vars seq
byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib cus-start
cus-load tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 132541 9164)
 (symbols 48 10938 1)
 (strings 32 46606 2064)
 (string-bytes 1 1918170)
 (vectors 16 17792)
 (vector-slots 8 215475 17462)
 (floats 8 50 75)
 (intervals 56 713 0)
 (buffers 1000 19))

Message #23 received at 76264 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "Basil L. Contovounesios" <basil <at> contovou.net>
Cc: Philip Kaludercic <philipk <at> posteo.net>,
 Stefan Kangas <stefankangas <at> gmail.com>, 76264 <at> debbugs.gnu.org
Subject: Re: bug#76264: [GNU ELPA] Use lisp-data-mode in generated *-pkg.el
 files
Date: Wed, 19 Mar 2025 11:00:38 -0400

> Here I see more than just a warning, in Emacs versions 24 through 27:
>
> 0. cd "$(mktemp -d)"
> 1. HOME="$PWD" XDG_CONFIG_HOME="$PWD" emacs-27.2
> 2. M-: (setq package-check-signature nil) RET
> 3. M-x toggle-debug-on-error RET
> 4. M-x package-install RET dash RET
>
> I attach backtraces from Emacs 24.5 and 27.2.

Does it signal an error if you don't enable `toggle-debug-on-error`, or
do we get a backtrace because of a `with-demoted-error` or some such?

> What I'm not sure about is why only some packages result in an error.

Because we don't constantly rebuild tarballs.

> I'm guessing elpa.gnu.org is caching -pkg.el files of packages that have
> not been updated in the meantime.

I wouldn't call it caching, but that's a way to look at it.


        Stefan

Message #26 received at 76264 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <basil <at> contovou.net>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Philip Kaludercic <philipk <at> posteo.net>,
 Stefan Kangas <stefankangas <at> gmail.com>, 76264 <at> debbugs.gnu.org
Subject: Re: bug#76264: [GNU ELPA] Use lisp-data-mode in generated *-pkg.el
 files
Date: Wed, 19 Mar 2025 22:32:41 +0100

Stefan Monnier [2025-03-19 11:00 -0400] wrote:

>> Here I see more than just a warning, in Emacs versions 24 through 27:
>>
>> 0. cd "$(mktemp -d)"
>> 1. HOME="$PWD" XDG_CONFIG_HOME="$PWD" emacs-27.2
>> 2. M-: (setq package-check-signature nil) RET
>> 3. M-x toggle-debug-on-error RET
>> 4. M-x package-install RET dash RET
>>
>> I attach backtraces from Emacs 24.5 and 27.2.
>
> Does it signal an error if you don't enable `toggle-debug-on-error`, or
> do we get a backtrace because of a `with-demoted-error` or some such?

AFAICT it's a real error, in the case of Emacs 27.2 (re)signalled from a
condition-case handler at the end of autoload-generate-file-autoloads:

    (error
     ;; Probably unbalanced parens in forward-sexp. In that case, the
     ;; condition is scan-error, and the signal data includes point
     ;; where the error was found; we'd like to convert that to
     ;; line:col, but line-number-at-pos gets the wrong line in batch
     ;; mode for some reason.
     ;;
     ;; At least this gets the file name in the error message; the
     ;; developer can use goto-char to get to the error position.
     (error "%s:0:0: error: %s: %s" file (car err) (cdr err)))

The example in the OP (installing dash from GNU ELPA) isn't so bad:
the user sees a scary error message, but after restarting Emacs dash
can be loaded without issue.

It can be worse, though:

0. cd "$(mktemp -d)"
1. HOME="$PWD" XDG_CONFIG_HOME="$PWD" emacs-27.2
2. M-: (setq package-check-signature nil) RET
3. M-: (setq package-archives '(("gnu" . "https://elpa.gnu.org/devel/"))) RET
4. M-x package-install RET hydra RET

This barfs while handling hydra's lv dependency, so hydra itself is
never installed.

-- 
Basil

Message #29 received at 76264 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: "Basil L. Contovounesios" <basil <at> contovou.net>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Philip Kaludercic <philipk <at> posteo.net>, 76264 <at> debbugs.gnu.org
Subject: Re: bug#76264: [GNU ELPA] Use lisp-data-mode in generated *-pkg.el
 files
Date: Wed, 19 Mar 2025 14:52:44 -0700

"Basil L. Contovounesios" <basil <at> contovou.net> writes:

> Stefan Monnier [2025-03-19 11:00 -0400] wrote:
>
>>> Here I see more than just a warning, in Emacs versions 24 through 27:
>>>
>>> 0. cd "$(mktemp -d)"
>>> 1. HOME="$PWD" XDG_CONFIG_HOME="$PWD" emacs-27.2
>>> 2. M-: (setq package-check-signature nil) RET
>>> 3. M-x toggle-debug-on-error RET
>>> 4. M-x package-install RET dash RET
>>>
>>> I attach backtraces from Emacs 24.5 and 27.2.
>>
>> Does it signal an error if you don't enable `toggle-debug-on-error`, or
>> do we get a backtrace because of a `with-demoted-error` or some such?
>
> AFAICT it's a real error, in the case of Emacs 27.2 (re)signalled from a
> condition-case handler at the end of autoload-generate-file-autoloads:

Maybe the patch should just be reverted, then?

Just to add some details:

See this issue in use-package, its recent comment, and linked issues for
more people that are affected:
https://github.com/jwiegley/use-package/issues/1083

What I don't understand is why the OP saw a similar error already in
September 2024.  Is there something else going on here as well?
Unfortunately, there is no backtrace, so it's hard to tell, but maybe
someone else will be able to eyeball it.

Message #32 received at 76264 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "Basil L. Contovounesios" <basil <at> contovou.net>
Cc: Philip Kaludercic <philipk <at> posteo.net>,
 Stefan Kangas <stefankangas <at> gmail.com>, 76264 <at> debbugs.gnu.org
Subject: Re: bug#76264: [GNU ELPA] Use lisp-data-mode in generated *-pkg.el
 files
Date: Wed, 19 Mar 2025 18:03:41 -0400

> AFAICT it's a real error, in the case of Emacs 27.2 (re)signalled from a
> condition-case handler at the end of autoload-generate-file-autoloads:

Then indeed, we need to refrain from using `lisp-data-mode` at least for
those packages that are meant to support Emacs<28.  I pushed a patch
to `elpa-admin.el` which does that.


        Stefan

Message #35 received at 76264 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <basil <at> contovou.net>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: Philip Kaludercic <philipk <at> posteo.net>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 76264 <at> debbugs.gnu.org
Subject: Re: bug#76264: [GNU ELPA] Use lisp-data-mode in generated *-pkg.el
 files
Date: Thu, 20 Mar 2025 02:08:45 +0100

Stefan Kangas [2025-03-19 14:52 -0700] wrote:

> See this issue in use-package, its recent comment, and linked issues for
> more people that are affected:
> https://github.com/jwiegley/use-package/issues/1083
>
> What I don't understand is why the OP saw a similar error already in
> September 2024.  Is there something else going on here as well?

Only MELPA beating us by a few months:
https://github.com/melpa/melpa/issues/9193

-- 
Basil

Message #38 received at 76264 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <basil <at> contovou.net>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Philip Kaludercic <philipk <at> posteo.net>,
 Stefan Kangas <stefankangas <at> gmail.com>, 76264 <at> debbugs.gnu.org
Subject: Re: bug#76264: [GNU ELPA] Use lisp-data-mode in generated *-pkg.el
 files
Date: Sat, 22 Mar 2025 17:34:53 +0100

Stefan Monnier [2025-03-19 18:03 -0400] wrote:

> Then indeed, we need to refrain from using `lisp-data-mode` at least for
> those packages that are meant to support Emacs<28.  I pushed a patch
> to `elpa-admin.el` which does that.

Thanks.  Does someone then need to trigger a tarball rebuild somehow?

-- 
Basil

Message #41 received at 76264 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "Basil L. Contovounesios" <basil <at> contovou.net>
Cc: Philip Kaludercic <philipk <at> posteo.net>,
 Stefan Kangas <stefankangas <at> gmail.com>, 76264 <at> debbugs.gnu.org
Subject: Re: bug#76264: [GNU ELPA] Use lisp-data-mode in generated *-pkg.el
 files
Date: Sat, 22 Mar 2025 17:00:38 -0400

Basil L. Contovounesios [2025-03-22 17:34:53] wrote:
> Stefan Monnier [2025-03-19 18:03 -0400] wrote:
>> Then indeed, we need to refrain from using `lisp-data-mode` at least for
>> those packages that are meant to support Emacs<28.  I pushed a patch
>> to `elpa-admin.el` which does that.
> Thanks.  Does someone then need to trigger a tarball rebuild somehow?

That would be ideal.  We usually try and refrain from modifying
existing files.  We could force them to be re-created, if needed, but
it's messy to do (e.g. the way things are setup, this will inevitably
involve a time window during which things are broken, and of course it
would require figuring out which ones need to be re-created).


        Stefan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.