GNU bug report logs - #57050
[PATCH 0/6] gnu: Update Racket to 8.6. Add Zuo.

Previous Next

Full log

Message #356 received at 57050 <at> debbugs.gnu.org (full text, mbox):

From: "Philip McGrath" <philip <at> philipmcgrath.com>
To: "Liliana Marie Prikler" <liliana.prikler <at> gmail.com>, 57050 <at> debbugs.gnu.org
Cc: "\(" <paren <at> disroot.org>, Maxime Devos <maximedevos <at> telenet.be>,
 Efraim Flashner <efraim <at> flashner.co.il>,
 Thiago Jung Bauermann <bauermann <at> kolabnow.com>
Subject: Re: [PATCH v4 08/14] gnu: racket: Update to 8.6.
Date: Sat, 27 Aug 2022 16:30:59 -0400

On Sat, Aug 27, 2022, at 3:21 PM, Liliana Marie Prikler wrote:
> Am Samstag, dem 27.08.2022 um 14:55 -0400 schrieb Philip McGrath:
>> +       ((#:tests? _ #t)
>> +        ;; FIXME: There have been some flaky test failures. Some
>> have been
>> +        ;; fixed upstream post-release but have proven non-trivial
>> to
>> +        ;; backport; at least one issue remains. Re-enable tests
>> once
>> +        ;; https://github.com/racket/racket/issues/4359 is fixed.
>> +        #f)
> Rather than discarding all tests, just discard the flaky ones.

I've tried to explain this before, but, as far as I've been able to determine, the problem doesn't seem to be that a particular test or group of tests is flaky: the mysterious problems all seem to be related to setting up the test environment. There have been problems, for example, with files not being found, or the complex pile of patches used to construct variants on the expected error output not working properly. At least the unix, windows, ieee, date, oop, primvars, and misc test groups have been affected. Since it's not been easy to reproduce the problem (I've thought it was fixed several times), I have no reason to believe it wouldn't affect other test groups if we skipped those. An unfortunate consequence of the way the tests are organized, skipping particular test groups means that expected error output is never checked and requires manually choosing the configurations (combinations of compiler/interpreter settings) you want to test.

I do not see a viable path to fixing the tests for this release, and I don't know a way of running a subset of tests that would be both reliable and meaningful.

Other distributions are also skipping these tests.

In fact, Guix is currently skipping an unknown (probably large) portion of these tests because of the issue with "/bin/sh", which has lead to things silently not running.

I hope we will get this fixed upstream for the 8.7 release. In fact, I've not yet encountered the problem while building from Git HEAD, but it's not clear what change might be relevant. (Indeed, the problem might still be there, and I might just not have hit it yet.)

-Philip

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.