GNU bug report logs - #35521
Mariadb test suite failures on x86_64-linux

Previous Next

Full log

View this message in rfc822 format

From: Marius Bakke <mbakke <at> fastmail.com>
To: Mark H Weaver <mhw <at> netris.org>
Cc: Platoxia <platoxia <at> protonmail.com>, Chris Marusich <cmmarusich <at> gmail.com>, 35521 <at> debbugs.gnu.org
Subject: bug#35521: Mariadb test suite failures on x86_64-linux
Date: Fri, 12 Jul 2019 16:58:48 +0200

[Message part 1 (text/plain, inline)]

Marius Bakke <mbakke <at> fastmail.com> writes:

> Mark H Weaver <mhw <at> netris.org> writes:
>
>> Hi,
>>
>> Marius Bakke <mbakke <at> fastmail.com> writes:
>>
>>> Chris Marusich <cmmarusich <at> gmail.com> writes:
>>>
>>>> Hi,
>>>>
>>>> I've been encountering this failure off and on for a few weeks now, and
>>>> I'd like to help fix it.  In short, it seems like non-deterministic test
>>>> failures, to me.  I think we should gather data and report the issue
>>>> upstream, and maybe disable the offending tests in the meantime.
>>>
>>> I agree.  I notice many of these failing tests are for the TokuDB
>>> backend, which I doubt anyone is using in Guix anyway.
>>>
>>> Here is a patch that disables all tests mentioned in this report.  I
>>> would like to push it to core-updates.  Are there others?
>>
>> I'm concerned by how frequently and casually we simply disable failing
>> tests.  What is the utility of running test suites at all, if this is
>> how we respond?
>
> I had no idea this issue was so widespread until I noticed Berlins
> builders hit it more often than not.  I have not been able to reproduce
> these failures on my machines.  So it was kind of a panic reaction,
> being the person responsible for running these tests and all.
>
> Looking further into the changes between 10.1.37 and 10.1.38, I notice
> the 'tokudb.*' tests were enabled:
>
> https://github.com/MariaDB/server/commit/4c490d6df63695dc97b2c808e59954e6877d3a51
>
> Watching the build on Berlin in real time, I also see that the test
> output grind nearly to a halt while running those.
> 'tokudb.hotindex-insert-2' took 2700439 milliseconds, or 45 minutes, if
> I'm reading the test output correctly.
>
> The default test case timeout is 40 minutes (as specified in the Guix
> package), but I'm using 80 for this build (60 was insufficient).
>
> I suspect the problem is that the 'tokudb.*' tests put a lot of strain
> on the file system, which causes these other tests to fail.  It's
> interesting that disabling parallel build was insufficient though.

Update: Berlin built mariadb twice on core-updates with this patch:

[db.diff (text/x-patch, inline)]

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 6bfeaad9a2..64bc0938b6 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -753,7 +753,7 @@ Language.")
                  (with-directory-excursion "mysql-test"
                    (invoke "./mtr" "--verbose"
                            "--retry=3"
-                           "--testcase-timeout=40"
+                           "--testcase-timeout=80"
                            "--suite-timeout=600"
                            "--parallel" (number->string (parallel-job-count))
                            "--skip-test-list=unstable-tests"))

[Message part 3 (text/plain, inline)]

Mark, Chris: Can you try this change with MariaDB 10.1.40 and see if it
works for you?

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.