Subject: Re: bug#35521: Mariadb test suite failures on x86_64-linux

From: Giovanni Biscuolo <[email-masked]>
Date: Fri, 12 Jul 2019 10:24:04 +0200

Hi Marius,

Marius Bakke <[email-masked]> writes:

[...]

> Looking further into the changes between 10.1.37 and 10.1.38, I notice
> the 'tokudb.*' tests were enabled:
>
> https://github.com/MariaDB/server/commit/4c490d6df63695dc97b2c808e59954e6877d3a51

The very first thing I noticed lookng at that commit is it's subject:
"Updated list of unstable tests for 10.1.38 release"

The first comments of that file states:

--8<---------------cut here---------------start------------->8---
#  List the test cases which, unlike tests from disabled.def files,
#  can still be run on the current tree meaningfully, but are known
#  or suspected to fail sporadically on different reasons.
#
#  Most common reasons are either test failures observed in buildbot,
#  or recent modifications to the tests which make their stability
#  unknown.
#
#  Tests included due to recent modifications are later removed from the
#  list, if during a certain period they do not fail (and are not
#  modified again). Tests included due to intermittent failures are
#  removed when corresponding bug reports are closed.
#
#  Separate the test case name and the comment with ':'.
#
#    <suitename>.<testcasename> : MDEV-xxxxx - <comment>
#
#  '*' wildcard in testcase names is supported.
#
#  To use the list, run MTR with --skip-test-list=unstable-tests option.
--8<---------------cut here---------------end--------------->8---

So *all* those rests _are_ considered unstable upstream.

IMHO they should be *selectively* skipped when they causes build
problems in Guix, including non-deterministic frequent ones like in this
case.

> Watching the build on Berlin in real time, I also see that the test
> output grind nearly to a halt while running those.
> 'tokudb.hotindex-insert-2' took 2700439 milliseconds, or 45 minutes, if
> I'm reading the test output correctly.

The same is happening upstream:
https://jira.mariadb.org/browse/MDEV-15198
https://jira.mariadb.org/browse/MDEV-16040 (duplicate of the above)
https://jira.mariadb.org/browse/MDEV-15271

That bugs (and all others related to unstable tests) are currently
unresolved.

HTH! Gio'

-- 
Giovanni Biscuolo

Xelera IT Infrastructures