GNU bug report logs - #76447
29.4; ert-run-tests-batch now aborts if a test signals an undefined error

Previous Next

Full log

View this message in rfc822 format

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 76447 <at> debbugs.gnu.org, p.stephani2 <at> gmail.com
Subject: bug#76447: 29.4; ert-run-tests-batch now aborts if a test signals an undefined error
Date: Sat, 15 Mar 2025 13:11:14 +0200

Ping!

> Cc: 76447 <at> debbugs.gnu.org
> Date: Sat, 01 Mar 2025 14:24:42 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> 
> > From: Philipp <p.stephani2 <at> gmail.com>
> > Date: Thu, 20 Feb 2025 21:26:41 +0100
> > 
> > 
> > With Emacs 29.4:
> > 
> > $ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' -eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
> > Running 2 tests (2025-02-20 21:13:09+0100, selector ‘t’)
> > Test test-1 backtrace:
> >   signal(foo nil)
> >   (closure (t) nil (signal 'foo nil))()
> >   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
> >   ert-run-test(#s(ert-test :name test-1 :documentation nil :body (clos
> >   ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test 
> >   ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
> >   ert-run-tests-batch(nil)
> >   ert-run-tests-batch-and-exit()
> >   command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo) 
> >   command-line()
> >   normal-top-level()
> > Test test-1 condition:
> >     (foo)
> >    FAILED  1/2  test-1 (0.000060 sec) 
> >    passed  2/2  test-2 (0.000030 sec)
> > 
> > Ran 2 tests, 1 results as expected, 1 unexpected (2025-02-20 21:13:09+0100, 0.019570 sec)
> > 
> > 1 unexpected results:
> >    FAILED  test-1
> > 
> > 
> > With Emacs 30.1 release candidate 1:
> > 
> > $ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' -eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
> > Running 2 tests (2025-02-20 21:13:03+0100, selector ‘t’)
> > Test test-1 aborted with non-local exit
> >   ABORTED  1/2  test-1 (0.000073 sec) 
> > 
> > Aborted: Ran 2 tests, 0 results as expected, 0 unexpected (2025-02-20 21:13:03+0100, 0.000140 sec)
> > 
> > Error running tests
> >   backtrace()
> >   #f(compiled-function () #<bytecode -0x174c28bd26a373a>)()
> >   ert-run-tests-batch-and-exit()
> >   command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo) nil))" "-eval" "(ert-deftest test-2 ())" "-f" "ert-run-tests-batch-and-exit"))
> >   command-line()
> >   normal-top-level()
> > 
> > 
> > test-2 isn't executed.  I think the Emacs 29.4 behavior is more
> > "user-friendly", though this (signaling an error symbol that doesn't
> > derive from 'error') is arguably a corner case.
> > 
> > Probably the behavior change is fallout from switching to handler-bind.
> > To restore the Emacs 29.4 behavior, probably a construct such as
> > 
> > (condition-case err
> >     (handler-bind ((t ...)) ...)
> >   (t ...))
> > 
> > could be used.  The pseudo-condition t should be able to catch all
> > signals.
> 
> Stefan, any comments?
> 
> 
> 
> 

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.