GNU bug report logs - #24913
25.1.50; Emacs accepts undocumented and confusing combinations of &optional and &rest in argument lists

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 24913 in the body.
You can then email your comments to 24913 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 25.1.50;
 Emacs accepts undocumented and confusing combinations of &optional
 and &rest in argument lists
Date: Wed, 09 Nov 2016 22:17:54 +0100

For example:

(funcall (lambda (&optional &rest &rest &optional x) (list x)) 'a)
=> ((a))

Obviously here the &rest keyword "wins", but I think that's overly
confusing.  Such an argument list is most likely a programmer mistake,
and should signal an error to make the programmer aware of the mistake.



In GNU Emacs 25.1.50.13 (x86_64-unknown-linux-gnu, GTK+ Version 3.10.8)
 of 2016-11-09 built on localhost
Repository revision: eb364fddec1431f459166cebb36f09f6b371dd71
Windowing system distributor 'The X.Org Foundation', version 11.0.11501000
System Description:	Ubuntu 14.04 LTS

Configured using:
 'configure --with-modules --enable-checking
 --enable-check-lisp-object-type 'CFLAGS=-ggdb3 -O0''

Configured features:
XPM JPEG TIFF GIF PNG SOUND GSETTINGS NOTIFY GNUTLS FREETYPE XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired format-spec rfc822 mml
mml-sec password-cache epg epg-config gnus-util mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util help-fns help-mode easymenu
cl-loaddefs pcase cl-lib mail-prsvr mail-utils time-date mule-util
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote inotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 88939 12159)
 (symbols 48 19888 0)
 (miscs 40 343 148)
 (strings 32 14757 5052)
 (string-bytes 1 440451)
 (vectors 16 12779)
 (vector-slots 8 446082 5011)
 (floats 8 166 60)
 (intervals 56 209 0)
 (buffers 976 24)
 (heap 1024 23753 956))

-- 
Google Germany GmbH
Erika-Mann-Straße 33
80636 München

Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle

Diese E-Mail ist vertraulich.  Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den Absender und löschen
Sie die E-Mail und alle Anhänge.  Vielen Dank.

This e-mail is confidential.  If you are not the right addressee please do not
forward it, please inform the sender, and please erase this e-mail including
any attachments.  Thanks.

Message #8 received at 24913 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: 24913 <at> debbugs.gnu.org
Subject: Re: bug#24913: 25.1.50; Emacs accepts undocumented and confusing
 combinations of &optional and &rest in argument lists
Date: Thu, 10 Nov 2016 12:58:39 +0000

[Message part 1 (text/plain, inline)]

Philipp Stephani <p.stephani2 <at> gmail.com> schrieb am Mi., 9. Nov. 2016 um
22:19 Uhr:

>
> For example:
>
> (funcall (lambda (&optional &rest &rest &optional x) (list x)) 'a)
> => ((a))
>
> Obviously here the &rest keyword "wins", but I think that's overly
> confusing.  Such an argument list is most likely a programmer mistake,
> and should signal an error to make the programmer aware of the mistake.
>
>
Here's a patch that detects such argument lists.

[Message part 2 (text/html, inline)]

[0001-Prevent-dubious-argument-lists.txt (text/plain, attachment)]

Message #11 received at 24913 <at> debbugs.gnu.org (full text, mbox):

From: Gemini Lasswell <gazally <at> runbox.com>
To: Philipp Stephani <p.stephani2 <at> gmail.com>, 24913 <at> debbugs.gnu.org
Subject: Re: bug#24913: 25.1.50;
 Emacs accepts undocumented and confusing combinations of &optional
 and &rest in argument lists
Date: Thu, 10 Nov 2016 08:56:36 -0800

> Philipp Stephani <p.stephani2 <at> gmail.com> schrieb am Mi., 9. Nov. 2016 um 22:19 Uhr:
>
> Here's a patch that detects such argument lists. 
>
A more general solution would be to have the byte compiler try to match
Edebug specs, and issue a warning or an error if it fails. That would
help find errors in the invocations of all macros, not just ones with
argument lists.

Message #14 received at 24913 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 24913 <at> debbugs.gnu.org
Subject: Re: bug#24913: 25.1.50;
 Emacs accepts undocumented and confusing combinations of &optional
 and &rest in argument lists
Date: Fri, 18 Nov 2016 11:06:33 +0200

> From: Philipp Stephani <p.stephani2 <at> gmail.com>
> Date: Thu, 10 Nov 2016 12:58:39 +0000
> 
> Philipp Stephani <p.stephani2 <at> gmail.com> schrieb am Mi., 9. Nov. 2016 um 22:19 Uhr:
> 
>  For example:
> 
>  (funcall (lambda (&optional &rest &rest &optional x) (list x)) 'a)
>  => ((a))
> 
>  Obviously here the &rest keyword "wins", but I think that's overly
>  confusing. Such an argument list is most likely a programmer mistake,
>  and should signal an error to make the programmer aware of the mistake.
> 
> Here's a patch that detects such argument lists. 

Thanks, please push to master.

Message #19 received at 24913-done <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 24913-done <at> debbugs.gnu.org
Subject: Re: bug#24913: 25.1.50; Emacs accepts undocumented and confusing
 combinations of &optional and &rest in argument lists
Date: Fri, 18 Nov 2016 17:37:26 +0000

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> schrieb am Fr., 18. Nov. 2016 um 10:06 Uhr:

> > From: Philipp Stephani <p.stephani2 <at> gmail.com>
> > Date: Thu, 10 Nov 2016 12:58:39 +0000
> >
> > Philipp Stephani <p.stephani2 <at> gmail.com> schrieb am Mi., 9. Nov. 2016
> um 22:19 Uhr:
> >
> >  For example:
> >
> >  (funcall (lambda (&optional &rest &rest &optional x) (list x)) 'a)
> >  => ((a))
> >
> >  Obviously here the &rest keyword "wins", but I think that's overly
> >  confusing. Such an argument list is most likely a programmer mistake,
> >  and should signal an error to make the programmer aware of the mistake.
> >
> > Here's a patch that detects such argument lists.
>
> Thanks, please push to master.
>

Done.

[Message part 2 (text/html, inline)]

Message #22 received at 24913 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Gemini Lasswell <gazally <at> runbox.com>, 24913 <at> debbugs.gnu.org
Subject: Re: bug#24913: 25.1.50; Emacs accepts undocumented and confusing
 combinations of &optional and &rest in argument lists
Date: Sat, 19 Nov 2016 16:40:47 +0000

[Message part 1 (text/plain, inline)]

Gemini Lasswell <gazally <at> runbox.com> schrieb am Do., 10. Nov. 2016 um
17:56 Uhr:

>
> > Philipp Stephani <p.stephani2 <at> gmail.com> schrieb am Mi., 9. Nov. 2016
> um 22:19 Uhr:
> >
> > Here's a patch that detects such argument lists.
> >
> A more general solution would be to have the byte compiler try to match
> Edebug specs, and issue a warning or an error if it fails. That would
> help find errors in the invocations of all macros, not just ones with
> argument lists.
>

I don't understand how this is related. This is only about function
definitions in the evaluator and the byte compiler. I don't see how Edebug
could help here.

[Message part 2 (text/html, inline)]

Message #25 received at 24913 <at> debbugs.gnu.org (full text, mbox):

From: Gemini Lasswell <gazally <at> runbox.com>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 24913 <at> debbugs.gnu.org
Subject: Re: bug#24913: 25.1.50;
 Emacs accepts undocumented and confusing combinations of &optional
 and &rest in argument lists
Date: Sat, 19 Nov 2016 22:31:21 -0800

Philipp Stephani <p.stephani2 <at> gmail.com> writes:

>  A more general solution would be to have the byte compiler try to match
>  Edebug specs, and issue a warning or an error if it fails. That would
>  help find errors in the invocations of all macros, not just ones with
>  argument lists.
>
> I don't understand how this is related. This is only about function
> definitions in the evaluator and the byte compiler. I don't see how
> Edebug could help here.

Edebug specs describe the expected syntax for the arguments of a macro,
including the macros which define functions, such as lambda, defun,
defmacro etc. If Edebug can't match the actual arguments in a macro
invocation to the Edebug spec for that macro, it will signal an error.
For an example of Edebug catching a misplaced &optional in an argument
list, see bug#24762.

So part of my suggestion is that since there exists in Emacs a powerful
mechanism for checking macro argument lists, it would be better to use
it if we want to let programmers know that their macro invocations are
incorrect, instead of adding error checking to individual macros on a
case by case basis.

Another thought going into this suggestion is my observation that it's
not difficult to find bugs in Edebug specs in the Emacs sources right
now. One cause of that could be the Edebug spec documentation, which
could be improved. But I think the primary reason is that a macro with a
broken Edebug spec won't cause an error until someone tries to use
Edebug or Testcover on an invocation of that macro, which maybe isn't
common practice when reviewing changes. But if the byte compiler checked
Edebug specs and signaled errors, then at least those errors in Edebug
specs which can be found statically would be noticed immediately.

Message #28 received at 24913 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Gemini Lasswell <gazally <at> runbox.com>
Cc: 24913 <at> debbugs.gnu.org
Subject: Re: bug#24913: 25.1.50; Emacs accepts undocumented and confusing
 combinations of &optional and &rest in argument lists
Date: Sun, 20 Nov 2016 12:41:02 +0000

[Message part 1 (text/plain, inline)]

Gemini Lasswell <gazally <at> runbox.com> schrieb am So., 20. Nov. 2016 um
07:31 Uhr:

> Philipp Stephani <p.stephani2 <at> gmail.com> writes:
>
> >  A more general solution would be to have the byte compiler try to match
> >  Edebug specs, and issue a warning or an error if it fails. That would
> >  help find errors in the invocations of all macros, not just ones with
> >  argument lists.
> >
> > I don't understand how this is related. This is only about function
> > definitions in the evaluator and the byte compiler. I don't see how
> > Edebug could help here.
>
> Edebug specs describe the expected syntax for the arguments of a macro,
> including the macros which define functions, such as lambda, defun,
> defmacro etc. If Edebug can't match the actual arguments in a macro
> invocation to the Edebug spec for that macro, it will signal an error.
> For an example of Edebug catching a misplaced &optional in an argument
> list, see bug#24762.
>
> So part of my suggestion is that since there exists in Emacs a powerful
> mechanism for checking macro argument lists, it would be better to use
> it if we want to let programmers know that their macro invocations are
> incorrect, instead of adding error checking to individual macros on a
> case by case basis.
>
> Another thought going into this suggestion is my observation that it's
> not difficult to find bugs in Edebug specs in the Emacs sources right
> now. One cause of that could be the Edebug spec documentation, which
> could be improved. But I think the primary reason is that a macro with a
> broken Edebug spec won't cause an error until someone tries to use
> Edebug or Testcover on an invocation of that macro, which maybe isn't
> common practice when reviewing changes. But if the byte compiler checked
> Edebug specs and signaled errors, then at least those errors in Edebug
> specs which can be found statically would be noticed immediately.
>

Integrating Edebug checks, byte-compiler checks, and evaluator checks
sounds like the right thing to do, but also like a huge amount of work that
is out of scope for this bug. Please create a new bug if you want that.
Note that a generic checker would still need to catch cases such as
(funcall (function (lambda (&rest))) ())
that don't involve any macros.

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.