GNU bug report logs - #47320
Improve failure reporting in test/lisp/electrict-tests.el

Previous Next

Full log

Message #17 received at 47320 <at> debbugs.gnu.org (full text, mbox):

From: João Távora <joaotavora <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Alan Mackenzie <acm <at> muc.de>, 47320 <at> debbugs.gnu.org
Subject: Re: bug#47320: Improve failure reporting in
 test/lisp/electrict-tests.el
Date: Tue, 23 Mar 2021 15:59:03 +0000

[Message part 1 (text/plain, inline)]

As the author of electric-tests.el, I'm fine with either approach.
Michael's would seem more integrated, at a glance, not sure how hard it is
to switch to it. I'd have to see that patch. João

On Tue, Mar 23, 2021, 15:24 Michael Albinus <michael.albinus <at> gmx.de> wrote:

> Alan Mackenzie <acm <at> muc.de> writes:
>
> > Hello, Michael.
>
> Hi Alan,
>
> >> There is the explainer functionality in ert, see
> >
> >> (info "(ert)Defining Explanation Functions")
> >
> > That description is rather terse.  In fact it is incomplete - it does
> > not say when the explanation function gets called, nor does it say what
> > is done with any resulting explanation.
>
> 100% d'accord. The ert manual isn't as helpful as it should ...
>
> > It looks like a explanation function needs to duplicate the test code of
> > the test being explained - it seems the function has no access to the
> > internal state of the test.
>
> No, it has full access to the test via the ert machinery.
>
> (ert-get-test SYMBOL) returns the test object which has been created via
> (ert-deftest SYMBOL-NAME ...)
>
> (ert-running-test) returns the test object of the test just running.
>
> With that test object, you have different access functions, like
> ert-test-name, ert-test-documentation, ert-test-most-recent-result; see
> cl-defstruct ert-test. And there are also some other defstructs, which
> give you convenience functions like ert-test-failed-p.
>
> > I only wish to print the extra information when there is a test failure.
> > Otherwise the information in electric-tests.log would be swamped by
> > pointless voluminous doc strings.
>
> The ert explainer writes only something for failed functions.
>
> > Or am I missing something?
>
> <shameless advertisement>I've spent hours and hours to understand these
> ert functions. You might have a look on filenotify-tests.el or
> tramp-tests.el, for example.</>
>
> >> I recommend to use it, for better readability of the test code. Examples
> >> are in our test files.
> >
> > Thanks!
>
> If needed, I could help you in setting upt these explainers. But I must
> confess, I have no idea about electrict-tests.el.
>
> For the beginning, you might look on this:
>
> --8<---------------cut here---------------start------------->8---
> (defun test-function ()
>   "The result."
>   t)
>
> (defun test-function-explainer ()
>   "The explainer, a string."
>   (format "%s: %s" (ert-test-name (ert-running-test))
> (ert-test-documentation (ert-running-test))))
>
> (ert-deftest first-test ()
>   "Just the first test"
>   (should (test-function)))
>
> (ert-deftest second-test ()
>   "Just the second test"
>   (should-not (test-function)))
>
> (put 'test-function 'ert-explainer
>      'test-function-explainer)
> --8<---------------cut here---------------end--------------->8---
>
> If you run both tests, you get
>
> --8<---------------cut here---------------start------------->8---
> .F
>
> . first-test
>     Just the first test
>     passed
>
> F second-test
>     Just the second test
>     (ert-test-failed
>      ((should-not
>        (test-function))
>       :form
>       (test-function)
>       :value t :explanation "second-test: Just the second test"))
> --8<---------------cut here---------------end--------------->8---
>
> Best regards, Michael.
>

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.