GNU bug report logs - #32221
[PATCH 0/5] MariaDB updates

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 32221 in the body.
You can then email your comments to 32221 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: guix-patches <at> gnu.org
Subject: [PATCH 0/5] MariaDB updates
Date: Fri, 20 Jul 2018 13:35:53 +0200

These patches gives us more control of the MariaDB package.  In
particular, we now invoke the test runner directly, so we can pass
custom arguments; there's a phase for disabling plugins in an attempt
to fix the armhf build; we now use system zlib, pcre, xz and snappy;
and the package is ~37 MiB smaller thanks to a new "static" output.

I've verified that 'qtbase' builds against this MariaDB.

Marius Bakke (5):
  gnu: mariadb: Disable plugin that fails on armhf.
  gnu: mariadb: Run the full test suite.
  gnu: mariadb: Update to 10.1.34.
  gnu: mariadb: Remove some bundled libraries.
  gnu: mariadb: Move static libraries to separate output.

 gnu/packages/databases.scm | 104 ++++++++++++++++++++++++++++++++++---
 1 file changed, 97 insertions(+), 7 deletions(-)

-- 
2.18.0

Message #8 received at 32221 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: 32221 <at> debbugs.gnu.org
Subject: [PATCH 1/5] gnu: mariadb: Disable plugin that fails on armhf.
Date: Fri, 20 Jul 2018 13:38:33 +0200

* gnu/packages/databases.scm (mariadb)[arguments]: Add 'disable-plugins' phase.
---
 gnu/packages/databases.scm | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index b9ae9ee15..c27381ac2 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -665,6 +665,19 @@ Language.")
          "-DINSTALL_SHAREDIR=share/mysql")
        #:phases
        (modify-phases %standard-phases
+         (add-before 'configure 'disable-plugins
+           (lambda _
+             (let ((disable-plugin (lambda (name)
+                                     (call-with-output-file
+                                         (string-append "plugin/" name
+                                                        "/CMakeLists.txt")
+                                       (lambda (port)
+                                         (format port "\n")))))
+                   (disabled-plugins '(;; FIXME: On armhf-linux, this plugin
+                                       ;; triggers a GCC ICE.  Disable for now.
+                                       "semisync")))
+               (for-each disable-plugin disabled-plugins)
+               #t)))
          (add-before
           'configure 'pre-configure
           (lambda _
-- 
2.18.0

Message #11 received at 32221 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: 32221 <at> debbugs.gnu.org
Subject: [PATCH 2/5] gnu: mariadb: Run the full test suite.
Date: Fri, 20 Jul 2018 13:38:34 +0200

* gnu/packages/databases.scm (mariadb)[arguments]: Override 'check' phase.
Add phase 'adjust-tests'.  Disable one more plugin.
[properties]: New field.
---
 gnu/packages/databases.scm | 35 ++++++++++++++++++++++++++++++++++-
 1 file changed, 34 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index c27381ac2..9e4adc340 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -665,6 +665,29 @@ Language.")
          "-DINSTALL_SHAREDIR=share/mysql")
        #:phases
        (modify-phases %standard-phases
+         (add-after 'unpack 'adjust-tests
+           (lambda _
+             (let ((disabled-tests
+                    '(;; These fail because root <at> hostname == root <at> localhost in
+                      ;; the build environment, causing a user count mismatch.
+                      ;; See <https://jira.mariadb.org/browse/MDEV-7761>.
+                      "main.join_cache"
+                      "main.explain_non_select"
+                      "roles.acl_statistics"))
+
+                   ;; This file contains a list of known-flaky tests for this
+                   ;; release.  Append our own list.
+                   (unstable-tests (open-file "mysql-test/unstable-tests" "a")))
+               (for-each (lambda (test)
+                           (format unstable-tests "~a : ~a\n"
+                                   test "Disabled in Guix"))
+                         disabled-tests)
+               (close-port unstable-tests)
+
+               (substitute* "mysql-test/mysql-test-run.pl"
+                 (("/bin/ls") (which "ls"))
+                 (("/bin/sh") (which "sh")))
+               #t)))
          (add-before 'configure 'disable-plugins
            (lambda _
              (let ((disable-plugin (lambda (name)
@@ -675,7 +698,9 @@ Language.")
                                          (format port "\n")))))
                    (disabled-plugins '(;; FIXME: On armhf-linux, this plugin
                                        ;; triggers a GCC ICE.  Disable for now.
-                                       "semisync")))
+                                       "semisync"
+                                       ;; XXX: Causes a test failure.
+                                       "disks")))
                (for-each disable-plugin disabled-plugins)
                #t)))
          (add-before
@@ -683,6 +708,11 @@ Language.")
           (lambda _
             (setenv "CONFIG_SHELL" (which "sh"))
             #t))
+         (replace 'check
+           (lambda _
+             (with-directory-excursion "mysql-test"
+               (invoke "./mtr" "--parallel" (number->string (parallel-job-count))
+                       "--verbose" "--skip-test-list=unstable-tests"))))
          (add-after
           'install 'post-install
           (lambda* (#:key outputs #:allow-other-keys)
@@ -708,6 +738,9 @@ Language.")
        ("openssl" ,openssl)
        ("pcre" ,pcre)
        ("zlib" ,zlib)))
+    ;; The test suite is very resource intensive and can take more than three
+    ;; hours on a x86_64 system.  Give slow and busy machines some leeway.
+    (properties '((timeout . 43200)))        ;12 hours
     (home-page "https://mariadb.org/")
     (synopsis "SQL database server")
     (description
-- 
2.18.0

Message #14 received at 32221 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: 32221 <at> debbugs.gnu.org
Subject: [PATCH 3/5] gnu: mariadb: Update to 10.1.34.
Date: Fri, 20 Jul 2018 13:38:35 +0200

* gnu/packages/databases.scm (mariadb): Update to 10.1.34.
---
 gnu/packages/databases.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 9e4adc340..3b767664a 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -627,7 +627,7 @@ Language.")
 (define-public mariadb
   (package
     (name "mariadb")
-    (version "10.1.33")
+    (version "10.1.34")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://downloads.mariadb.org/f/"
@@ -635,7 +635,7 @@ Language.")
                                   name "-" version ".tar.gz"))
               (sha256
                (base32
-                "0bax748j4srsyhw5cs5jvwigndh0zwmf4r2cjvhja31ckx8jqccl"))))
+                "0j2mdpyvj41vkq2rwrzky88b7170hzz6gy2vb2bc1447s2gp3q67"))))
     (build-system cmake-build-system)
     (arguments
      '(#:configure-flags
-- 
2.18.0

Message #17 received at 32221 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: 32221 <at> debbugs.gnu.org
Subject: [PATCH 4/5] gnu: mariadb: Remove some bundled libraries.
Date: Fri, 20 Jul 2018 13:38:36 +0200

* gnu/packages/databases.scm (mariadb)[source](snippet, modules): New fields.
[arguments]: Add explicit #:configure-flags for system libraries.  Add
'unbundle' phase.  Remove 'pre-configure' phase.
[inputs]: Add SNAPPY and XZ.
---
 gnu/packages/databases.scm | 43 ++++++++++++++++++++++++++++++++------
 1 file changed, 37 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 3b767664a..de1db2f93 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -635,7 +635,16 @@ Language.")
                                   name "-" version ".tar.gz"))
               (sha256
                (base32
-                "0j2mdpyvj41vkq2rwrzky88b7170hzz6gy2vb2bc1447s2gp3q67"))))
+                "0j2mdpyvj41vkq2rwrzky88b7170hzz6gy2vb2bc1447s2gp3q67"))
+              (modules '((guix build utils)))
+              (snippet
+               '(begin
+                  (delete-file-recursively "storage/tokudb/PerconaFT/third_party")
+                  (for-each (lambda (file)
+                              (unless (string-suffix? "CMakeLists.txt" file)
+                                (delete-file file)))
+                            (append (find-files "pcre") (find-files "zlib")))
+                  #t))))
     (build-system cmake-build-system)
     (arguments
      '(#:configure-flags
@@ -649,6 +658,11 @@ Language.")
          ;; For now, disable the features that that use libarchive (xtrabackup).
          "-DWITH_LIBARCHIVE=OFF"
 
+         ;; Ensure the system libraries are used.
+         "-DWITH_JEMALLOC=yes"
+         "-DWITH_PCRE=system"
+         "-DWITH_ZLIB=system"
+
          "-DDEFAULT_CHARSET=utf8"
          "-DDEFAULT_COLLATION=utf8_general_ci"
          "-DMYSQL_DATADIR=/var/lib/mysql"
@@ -665,6 +679,26 @@ Language.")
          "-DINSTALL_SHAREDIR=share/mysql")
        #:phases
        (modify-phases %standard-phases
+         (add-after 'unpack 'unbundle
+           (lambda _
+             ;; The bundled PCRE in MariaDB has a patch that was upstreamed
+             ;; in version 8.34.  Unfortunately the upstream patch behaves
+             ;; slightly differently and the build system fails to detect it.
+             ;; See <https://bugs.exim.org/show_bug.cgi?id=2173>.
+             ;; XXX: Consider patching PCRE instead.
+             (substitute* "cmake/pcre.cmake"
+               ((" OR NOT PCRE_STACK_SIZE_OK") ""))
+
+             (substitute* "storage/tokudb/PerconaFT/ft/CMakeLists.txt"
+               ;; Remove dependency on these CMake targets.
+               ((" build_lzma build_snappy") ""))
+
+             (substitute* "storage/tokudb/PerconaFT/CMakeLists.txt"
+               ;; This file checks that the bundled sources are present and
+               ;; declares build procedures for them.  We don't need that.
+               (("^include\\(TokuThirdParty\\)") ""))
+
+             #t))
          (add-after 'unpack 'adjust-tests
            (lambda _
              (let ((disabled-tests
@@ -703,11 +737,6 @@ Language.")
                                        "disks")))
                (for-each disable-plugin disabled-plugins)
                #t)))
-         (add-before
-          'configure 'pre-configure
-          (lambda _
-            (setenv "CONFIG_SHELL" (which "sh"))
-            #t))
          (replace 'check
            (lambda _
              (with-directory-excursion "mysql-test"
@@ -737,6 +766,8 @@ Language.")
        ("ncurses" ,ncurses)
        ("openssl" ,openssl)
        ("pcre" ,pcre)
+       ("snappy" ,snappy)
+       ("xz" ,xz)
        ("zlib" ,zlib)))
     ;; The test suite is very resource intensive and can take more than three
     ;; hours on a x86_64 system.  Give slow and busy machines some leeway.
-- 
2.18.0

Message #20 received at 32221 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: 32221 <at> debbugs.gnu.org
Subject: [PATCH 5/5] gnu: mariadb: Move static libraries to separate output.
Date: Fri, 20 Jul 2018 13:38:37 +0200

* gnu/packages/databases.scm (mariadb)[outputs]: New field.
[arguments]: Add 'move-static-libs' phase.
---
 gnu/packages/databases.scm | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index de1db2f93..cbc530bec 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -646,6 +646,7 @@ Language.")
                             (append (find-files "pcre") (find-files "zlib")))
                   #t))))
     (build-system cmake-build-system)
+    (outputs '("out" "static"))
     (arguments
      '(#:configure-flags
        '("-DBUILD_CONFIG=mysql_release"
@@ -742,6 +743,18 @@ Language.")
              (with-directory-excursion "mysql-test"
                (invoke "./mtr" "--parallel" (number->string (parallel-job-count))
                        "--verbose" "--skip-test-list=unstable-tests"))))
+         (add-after 'install 'move-static-libs
+           ;; Move ~37 MiB worth of static libraries to a separate output.
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let ((lib (string-append (assoc-ref outputs "out") "/lib"))
+                   (slib (string-append (assoc-ref outputs "static") "/lib")))
+               (mkdir-p slib)
+               (with-directory-excursion lib
+                 (for-each (lambda (ar)
+                             (link ar (string-append slib "/" (basename ar)))
+                             (delete-file ar))
+                           (find-files "." "\\.a$"))
+                 #t))))
          (add-after
           'install 'post-install
           (lambda* (#:key outputs #:allow-other-keys)
-- 
2.18.0

Message #25 received at 32221-done <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: 32221-done <at> debbugs.gnu.org
Subject: Re: [bug#32221] [PATCH 0/5] MariaDB updates
Date: Sat, 28 Jul 2018 17:39:06 +0200

[Message part 1 (text/plain, inline)]

Marius Bakke <mbakke <at> fastmail.com> writes:

> These patches gives us more control of the MariaDB package.  In
> particular, we now invoke the test runner directly, so we can pass
> custom arguments; there's a phase for disabling plugins in an attempt
> to fix the armhf build; we now use system zlib, pcre, xz and snappy;
> and the package is ~37 MiB smaller thanks to a new "static" output.
>
> I've verified that 'qtbase' builds against this MariaDB.
>
> Marius Bakke (5):
>   gnu: mariadb: Disable plugin that fails on armhf.
>   gnu: mariadb: Run the full test suite.
>   gnu: mariadb: Update to 10.1.34.
>   gnu: mariadb: Remove some bundled libraries.
>   gnu: mariadb: Move static libraries to separate output.

I've pushed this patchset with slight modifications:

* Also unbundled YaSSL
* Deleted static library instead separate output

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.