Subject: srfi-64: test-runner-reset clobbers the run list

From: Tomas Volf <[email-masked]>
Date: Tue, 30 Jul 2024 21:51:59 +0200

Hello,

I think I found a bug in (srfi srfi-64) module shipped with GNU Guile.

Test runner is specified as:

> A test-runner is an object that runs a test-suite, and manages the state. The
> test group path, and the sets skip and expected-fail specifiers are part of
> the test-runner. A test-runner will also typically accumulate statistics about
> executed tests.

Based on this description, run list is *not* supposed to be part of the test
runner state, therefore test-runner-reset should not touch it.  However that
does not seem to be the case:

    (use-modules (srfi srfi-64))
    (test-apply (test-match-name "t-a")
                (λ ()
                  (test-begin "x")
                  (test-assert "t-a" (pk 't-a))
                  (test-assert "t-b" (pk 't-b))

                  (test-runner-reset (test-runner-current))

                  (test-begin "x")
                  (test-assert "t-a" (pk 't-a))
                  (test-assert "t-b" (pk 't-b))))

Prints:

    %%%% Starting test x  (Writing full log to "x.log")

    ;;; (t-a)
    %%%% Starting test x  (Writing full log to "x.log")

    ;;; (t-a)

    ;;; (t-b)

Notice that in the second part, t-b is executed despite not supposed to be on
the run list.

Have a nice day
Tomas Volf