GNU bug report logs - #76447
29.4; ert-run-tests-batch now aborts if a test signals an undefined error

Previous Next

Full log

Message #8 received at 76447 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philipp <p.stephani2 <at> gmail.com>, Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 76447 <at> debbugs.gnu.org
Subject: Re: bug#76447: 29.4;
 ert-run-tests-batch now aborts if a test signals an undefined error
Date: Sat, 01 Mar 2025 14:24:42 +0200

> From: Philipp <p.stephani2 <at> gmail.com>
> Date: Thu, 20 Feb 2025 21:26:41 +0100
> 
> 
> With Emacs 29.4:
> 
> $ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' -eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
> Running 2 tests (2025-02-20 21:13:09+0100, selector ‘t’)
> Test test-1 backtrace:
>   signal(foo nil)
>   (closure (t) nil (signal 'foo nil))()
>   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
>   ert-run-test(#s(ert-test :name test-1 :documentation nil :body (clos
>   ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test 
>   ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
>   ert-run-tests-batch(nil)
>   ert-run-tests-batch-and-exit()
>   command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo) 
>   command-line()
>   normal-top-level()
> Test test-1 condition:
>     (foo)
>    FAILED  1/2  test-1 (0.000060 sec) 
>    passed  2/2  test-2 (0.000030 sec)
> 
> Ran 2 tests, 1 results as expected, 1 unexpected (2025-02-20 21:13:09+0100, 0.019570 sec)
> 
> 1 unexpected results:
>    FAILED  test-1
> 
> 
> With Emacs 30.1 release candidate 1:
> 
> $ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' -eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
> Running 2 tests (2025-02-20 21:13:03+0100, selector ‘t’)
> Test test-1 aborted with non-local exit
>   ABORTED  1/2  test-1 (0.000073 sec) 
> 
> Aborted: Ran 2 tests, 0 results as expected, 0 unexpected (2025-02-20 21:13:03+0100, 0.000140 sec)
> 
> Error running tests
>   backtrace()
>   #f(compiled-function () #<bytecode -0x174c28bd26a373a>)()
>   ert-run-tests-batch-and-exit()
>   command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo) nil))" "-eval" "(ert-deftest test-2 ())" "-f" "ert-run-tests-batch-and-exit"))
>   command-line()
>   normal-top-level()
> 
> 
> test-2 isn't executed.  I think the Emacs 29.4 behavior is more
> "user-friendly", though this (signaling an error symbol that doesn't
> derive from 'error') is arguably a corner case.
> 
> Probably the behavior change is fallout from switching to handler-bind.
> To restore the Emacs 29.4 behavior, probably a construct such as
> 
> (condition-case err
>     (handler-bind ((t ...)) ...)
>   (t ...))
> 
> could be used.  The pseudo-condition t should be able to catch all
> signals.

Stefan, any comments?

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.