Subject: Re: bug#24503: unstable test results (on hydra)

From: Glenn Morris <[email-masked]>
Date: Tue, 28 Nov 2017 12:57:37 -0500

Tests that "randomly" fail, like the ones mentioned in this report, are
a problem for downstream distributions that run "make check" and pay
attention to the results (I sometimes think they pay more attention than
Emacs does). They can waste their time investigating things that seem to
be platform-specific just because they happen to fail once on a
particular architecture.

Eg
https://bugs.debian.org/879020

So we do a disservice to the wider community with these tests that
sometimes fail. If they aren't good enough to be enabled on Emacs's own
CI system, they probably aren't good enough full-stop, and should be
disabled by default for everyone.