GNU bug report logs - #22471
ert batch should print compile parsable error messages

Previous Next

Full log

View this message in rfc822 format

From: Stefan Kangas <stefan <at> marxist.se>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 22471 <at> debbugs.gnu.org, Phillip Lord <phillip.lord <at> russet.org.uk>
Subject: bug#22471: ert batch should print compile parsable error messages
Date: Fri, 22 Oct 2021 17:15:35 -0700

[Message part 1 (text/plain, inline)]

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> phillip.lord <at> russet.org.uk (Phillip Lord) writes:
>
>> Currently, when running in batch, ert prints out messages like so:
>>
>> Running 24 tests (2016-01-27 09:05:17+0000)
>>    passed   1/24  buffer-string=
>>    passed   2/24  buffer=
>> ...
>>    passed  22/24  sisyphus-test-with-find-file
>>    passed  23/24  to-string
>>    passed  24/24  with-temp-buffers
>>
>> Ran 24 tests, 23 results as expected, 1 unexpected (2016-01-27 09:05:18+0000)
>>
>> 1 unexpected results:
>>    FAILED  crash-out
>>
>> It would be nice to add some compilation-mode parsable data to this. So,
>> something like
>>
>>    passed   1/24  buffer-string=  (in ./test.sisyphus-test.el:22:)
>
> I think that would look rather cluttered for the "passed" lines, but it
> would indeed be helpful on the FAILED lines.

This would in my opinion be very useful and a great timesaver.  But yes,
for FAILED results only, much like how the compiler only gives us line
numbers if there is a problem.  (AFAIR, that's how the test frameworks
I've used do it.)

So I updated the patch from the feature/parsable-ert-output against
current master (patch attached), made sure that one of our tests failed,
and then tested it with "make check".

I got the following backtrace, but gave up after looking at it for a
while:

Debugger entered--Lisp error: (error "Don’t know where
‘fns-tests-string-bytes’ is defin...")
  signal(error ("Don’t know where ‘fns-tests-string-bytes’ is defin..."))
  error("Don't know where `%s' is defined" fns-tests-string-bytes)
  find-function-search-for-symbol(fns-tests-string-bytes ert-deftest nil)
  find-definition-noselect(fns-tests-string-bytes ert-deftest)
  ert-test-location([...])
[...]

[0001-Add-source-information-to-ERT-batch-mode.patch (text/x-diff, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.