GNU bug report logs - #72383
srfi-64: test-runner-reset clobbers the run list

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 72383 in the body.
You can then email your comments to 72383 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: bug-guile <at> gnu.org
Subject: srfi-64: test-runner-reset clobbers the run list
Date: Tue, 30 Jul 2024 21:51:59 +0200

Hello,

I think I found a bug in (srfi srfi-64) module shipped with GNU Guile.

Test runner is specified as:

> A test-runner is an object that runs a test-suite, and manages the state. The
> test group path, and the sets skip and expected-fail specifiers are part of
> the test-runner. A test-runner will also typically accumulate statistics about
> executed tests.

Based on this description, run list is *not* supposed to be part of the test
runner state, therefore test-runner-reset should not touch it.  However that
does not seem to be the case:

    (use-modules (srfi srfi-64))
    (test-apply (test-match-name "t-a")
                (λ ()
                  (test-begin "x")
                  (test-assert "t-a" (pk 't-a))
                  (test-assert "t-b" (pk 't-b))

                  (test-runner-reset (test-runner-current))

                  (test-begin "x")
                  (test-assert "t-a" (pk 't-a))
                  (test-assert "t-b" (pk 't-b))))

Prints:

    %%%% Starting test x  (Writing full log to "x.log")

    ;;; (t-a)
    %%%% Starting test x  (Writing full log to "x.log")

    ;;; (t-a)

    ;;; (t-b)

Notice that in the second part, t-b is executed despite not supposed to be on
the run list.

Have a nice day
Tomas Volf

Message #8 received at 72383 <at> debbugs.gnu.org (full text, mbox):

From: Taylan Kammer <taylan.kammer <at> gmail.com>
To: Tomas Volf <~@wolfsden.cz>, 72383 <at> debbugs.gnu.org
Subject: Re: bug#72383: srfi-64: test-runner-reset clobbers the run list
Date: Wed, 2 Oct 2024 01:35:03 +0200

On 30.07.2024 21:51, Tomas Volf wrote:
> Hello,
>
> I think I found a bug in (srfi srfi-64) module shipped with GNU Guile.
>
> Test runner is specified as:
>
>> A test-runner is an object that runs a test-suite, and manages the state. The
>> test group path, and the sets skip and expected-fail specifiers are part of
>> the test-runner. A test-runner will also typically accumulate statistics about
>> executed tests.
> Based on this description, run list is *not* supposed to be part of the test
> runner state, therefore test-runner-reset should not touch it.  However that
> does not seem to be the case:
>
>     (use-modules (srfi srfi-64))
>     (test-apply (test-match-name "t-a")
>                 (λ ()
>                   (test-begin "x")
>                   (test-assert "t-a" (pk 't-a))
>                   (test-assert "t-b" (pk 't-b))
>
>                   (test-runner-reset (test-runner-current))
>
>                   (test-begin "x")
>                   (test-assert "t-a" (pk 't-a))
>                   (test-assert "t-b" (pk 't-b))))
>
> Prints:
>
>     %%%% Starting test x  (Writing full log to "x.log")
>
>     ;;; (t-a)
>     %%%% Starting test x  (Writing full log to "x.log")
>
>     ;;; (t-a)
>
>     ;;; (t-b)
>
> Notice that in the second part, t-b is executed despite not supposed to be on
> the run list.
>
> Have a nice day
> Tomas Volf
>
>
I agree. The run-list is an implementation detail that only exists for test-apply, isn't used anywhere else, isn't exposed to the user, and is never mentioned in the spec. It shouldn't be possible for the user to ever affect it directly.

I've fixed this in my implementation of SRFI-64 as follows:

    https://codeberg.org/taylan/scheme-srfis/commit/ec07393f809edd249748a884641fb2e63292da79

- Taylan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.