GNU bug report logs - #50617
[core-updates-frozen] CMake fails to build on i686-linux

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 50617 in the body.
You can then email your comments to 50617 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
To: bug-guix <at> gnu.org
Subject: [core-updates-frozen] CMake fails to build on i686-linux
Date: Thu, 16 Sep 2021 11:41:09 +0200

On ‘core-updates-frozen’, CMake has one test failure on i686-linux when
building on berlin (e.g., <https://ci.guix.gnu.org/build/790602/log/raw>):

--8<---------------cut here---------------start------------->8---
545/558 Test #518: RunCMake.CPack_TXZ ................................***Failed    3.79 sec
-- MINIMAL-MONOLITHIC-type - PASSED
-- TXZ/MINIMAL-MONOLITHIC-type - PASSED
-- MINIMAL-COMPONENT-type - PASSED
-- TXZ/MINIMAL-COMPONENT-type - PASSED
-- MINIMAL-package-target-MONOLITHIC-type - PASSED
-- TXZ/MINIMAL-package-target-MONOLITHIC-type - PASSED
-- MINIMAL-package-target-COMPONENT-type - PASSED
-- TXZ/MINIMAL-package-target-COMPONENT-type - PASSED
-- THREADED_ALL-package-target-MONOLITHIC-type - PASSED
CMake Error at RunCMake.cmake:201 (message):
  TXZ/THREADED_ALL-package-target-MONOLITHIC-type - FAILED:

  Result is [1], not [0].

  stderr does not match that expected.

  Command was:

   command> "/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/bin/cmake" "-DRunCMake_TEST=THREADED_ALL-package-target-MONOLITHIC-type" "-DRunCMake_TEST_FILE_PREFIX=THREADED_ALL" "-DRunCMake_SUBTEST_SUFFIX=" "-DGENERATOR_TYPE=TXZ" "-DPACKAGING_TYPE=MONOLITHIC" "-Dsrc_dir=/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack" "-Dbin_dir=/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/TXZ/CPack/THREADED_ALL-package-target-build" "-Dconfig_file=/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack/conf/TXZ_config.cmake" "-P" "/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack/VerifyResult.cmake"

  Actual stdout:

   actual-out>

  Expected stderr to match:

   expect-err> ^$

  Actual stderr:

   actual-err> CMake Error at /tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack/VerifyResult.cmake:88 (message):
   actual-err>   Found more than one file for file No.  '1'! Found files count '0'.  Files:
   actual-err>   '' Globbing expression: 'threaded_all-0.1.1-*.tar.xz'
   actual-err>
   actual-err>   CPack output: 'make[1]: Entering directory
   actual-err>   '/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/TXZ/CPack/THREADED_ALL-package-target-build'
   actual-err>
   actual-err>
   actual-err>   Run CPack packaging tool...
   actual-err>
   actual-err>   CPack: Create package using TXZ
   actual-err>
   actual-err>   CPack: Install projects
   actual-err>
   actual-err>   CPack: - Run preinstall target for:
   actual-err>   THREADED_ALL-package-target-MONOLITHIC-type
   actual-err>
   actual-err>   CPack: - Install project: THREADED_ALL-package-target-MONOLITHIC-type []
   actual-err>
   actual-err>   CPack: Create package
   actual-err>
   actual-err>   make[1]: Leaving directory
   actual-err>   '/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/TXZ/CPack/THREADED_ALL-package-target-build'
   actual-err>
   actual-err>
   actual-err>   '
   actual-err>
   actual-err>   CPack error: 'CPack Error: Problem to open archive
   actual-err>   </tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/TXZ/CPack/THREADED_ALL-package-target-build/_CPack_Packages/Linux/TXZ/threaded_all-0.1.1-Linux.tar.xz>,
   actual-err>   ERROR = archive_write_open: Internal error initializing compression
   actual-err>   library: Cannot allocate memory
   actual-err>
   actual-err>   CPack Error: Problem compressing the directory
   actual-err>
   actual-err>   CPack Error: Error when generating package: threaded_all
   actual-err>
   actual-err>   make[1]: *** [Makefile:146: package] Error 1
   actual-err>
   actual-err>   ';
   actual-err>
   actual-err>   CPack result: '';
   actual-err>
   actual-err>   config file: ''

Call Stack (most recent call first):
  RunCMake.cmake:215 (run_cmake)
  CPack/CPackTestHelpers.cmake:119 (run_cmake_command)
  CPack/CPackTestHelpers.cmake:143 (run_cpack_test_common_)
  CPack/RunCMakeTest.cmake:24 (run_cpack_test_package_target)
--8<---------------cut here---------------end--------------->8---

I cannot reproduce it on hardware with 32 cores.  I suspect it has to do
with the number of threads used for xz compression, which defaults to
the number of cores, and some of the build machines on berlin have way
more cores.

Ludo’.

Message #8 received at 50617 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Ludovic Courtès <ludovic.courtes <at> inria.fr>
Cc: 50617 <at> debbugs.gnu.org
Subject: Re: bug#50617: [core-updates-frozen] CMake fails to build on
 i686-linux
Date: Thu, 16 Sep 2021 13:13:55 +0000

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludovic.courtes <at> inria.fr> skribis:

> On ‘core-updates-frozen’, CMake has one test failure on i686-linux when
> building on berlin (e.g., <https://ci.guix.gnu.org/build/790602/log/raw>):
>
> --8<---------------cut here---------------start------------->8---
> 545/558 Test #518: RunCMake.CPack_TXZ ................................***Failed    3.79 sec
> [...]
> --8<---------------cut here---------------end--------------->8---
>
> I cannot reproduce it on hardware with 32 cores.  I suspect it has to do
> with the number of threads used for xz compression, which defaults to
> the number of cores, and some of the build machines on berlin have way
> more cores.
>
> Ludo’.

I tried a few times on a machine with 16 cores, and I can't reproduce
either. The build succeeded every time.

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 50617 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 50617 <at> debbugs.gnu.org
Subject: Re: bug#50617: [core-updates-frozen] CMake fails to build on
 i686-linux
Date: Thu, 16 Sep 2021 17:20:15 +0200

[Message part 1 (text/plain, inline)]

Hi,

Guillaume Le Vaillant <glv <at> posteo.net> skribis:

> Ludovic Courtès <ludovic.courtes <at> inria.fr> skribis:
>
>> On ‘core-updates-frozen’, CMake has one test failure on i686-linux when
>> building on berlin (e.g., <https://ci.guix.gnu.org/build/790602/log/raw>):
>>
>> --8<---------------cut here---------------start------------->8---
>> 545/558 Test #518: RunCMake.CPack_TXZ ................................***Failed    3.79 sec
>> [...]
>> --8<---------------cut here---------------end--------------->8---
>>
>> I cannot reproduce it on hardware with 32 cores.  I suspect it has to do
>> with the number of threads used for xz compression, which defaults to
>> the number of cores, and some of the build machines on berlin have way
>> more cores.
>>
>> Ludo’.
>
> I tried a few times on a machine with 16 cores, and I can't reproduce
> either. The build succeeded every time.

My latest attempt is to go as far as setting the number of threads used
by libarchive to 1 (patch attached).  However, that still fails
systematically on berlin’s 32-core build machines.

The Internet doesn’t seem to have much to say about this problem.

Ideas anyone?

Ludo’.

[Message part 2 (text/x-patch, inline)]

diff --git a/gnu/packages/cmake.scm b/gnu/packages/cmake.scm
index f76ee3ff3a..a235d922e0 100644
--- a/gnu/packages/cmake.scm
+++ b/gnu/packages/cmake.scm
@@ -92,7 +92,7 @@ using the CMake build system.")
 
 ;;; Build phases shared between 'cmake-bootstrap' and the later variants
 ;;; that use cmake-build-system.
-(define %common-build-phases
+(define (%common-build-phases)
   `((add-after 'unpack 'split-package
       ;; Remove files that have been packaged in other package recipes.
       (lambda _
@@ -100,6 +100,18 @@ using the CMake build system.")
         (substitute* "Auxiliary/CMakeLists.txt"
           ((".*cmake-mode.el.*") ""))
         #t))
+    ,@(let ((system (or (%current-target-system) (%current-system))))
+        (if (or (string-prefix? "i686-" system)
+                (string-prefix? "i586-" system))
+            '((add-after 'unpack 'clamp-xz-thread-number
+                (lambda _
+                  ;; By default, 'CPack' would use as many threads as the
+                  ;; number of available cores for xz compression.
+                  ;; However, this leads to test failures: <XXX>.
+                  (substitute* "Source/cmArchiveWrite.cxx"
+                    (("case CompressXZ:")
+                     "case CompressXZ: numThreads = 1;\n")))))
+            '()))
     (add-before 'configure 'patch-bin-sh
       (lambda _
         ;; Replace "/bin/sh" by the right path in... a lot of
@@ -188,7 +200,7 @@ using the CMake build system.")
            " --exclude-regex ^\\(" (string-join skipped-tests "\\|") "\\)$")))
        #:phases
        (modify-phases %standard-phases
-         ,@%common-build-phases
+         ,@(%common-build-phases)
          (add-before 'configure 'set-paths
            (lambda _
              ;; Help cmake's bootstrap process to find system libraries
@@ -295,7 +307,7 @@ and workspaces that can be used in the compiler environment of your choice.")
        #:build-type "Release"
        #:phases
        (modify-phases %standard-phases
-         ,@%common-build-phases
+         ,@(%common-build-phases)
          (add-after 'install 'delete-help-documentation
            (lambda* (#:key outputs #:allow-other-keys)
              (delete-file-recursively

Message #14 received at 50617 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Ludovic Courtès <ludovic.courtes <at> inria.fr>, 
 Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 50617 <at> debbugs.gnu.org
Subject: Re: bug#50617: [core-updates-frozen] CMake fails to build on
 i686-linux
Date: Thu, 16 Sep 2021 18:21:31 +0200

[Message part 1 (text/plain, inline)]

Ludovic Courtès schreef op do 16-09-2021 om 17:20 [+0200]:
> +    ,@(let ((system (or (%current-target-system) (%current-system))))
> +        (if (or (string-prefix? "i686-" system)
> +                (string-prefix? "i586-" system))

(Unrelated to the build failure) you could use target-x86-32? here
to make the code a little simpler:

,@(if (target-x86-32?)
      '((add-after ....))
      '()

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 50617 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: Guillaume Le Vaillant <glv <at> posteo.net>, 50617 <at> debbugs.gnu.org
Subject: Re: bug#50617: [core-updates-frozen] CMake fails to build on
 i686-linux
Date: Fri, 17 Sep 2021 09:05:04 +0200

Maxime Devos <maximedevos <at> telenet.be> skribis:

> Ludovic Courtès schreef op do 16-09-2021 om 17:20 [+0200]:
>> +    ,@(let ((system (or (%current-target-system) (%current-system))))
>> +        (if (or (string-prefix? "i686-" system)
>> +                (string-prefix? "i586-" system))
>
> (Unrelated to the build failure) you could use target-x86-32? here
> to make the code a little simpler:
>
> ,@(if (target-x86-32?)
>       '((add-after ....))
>       '()

Ah yes, definitely!  Thanks for the hint.

Ludo’.

Message #22 received at 50617-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 50617-done <at> debbugs.gnu.org
Subject: Re: bug#50617: [core-updates-frozen] CMake fails to build on
 i686-linux
Date: Tue, 21 Sep 2021 10:21:38 +0200

Hi,

Guillaume Le Vaillant <glv <at> posteo.net> skribis:

> Ludovic Courtès <ludovic.courtes <at> inria.fr> skribis:
>
>> On ‘core-updates-frozen’, CMake has one test failure on i686-linux when
>> building on berlin (e.g., <https://ci.guix.gnu.org/build/790602/log/raw>):
>>
>> --8<---------------cut here---------------start------------->8---
>> 545/558 Test #518: RunCMake.CPack_TXZ ................................***Failed    3.79 sec
>> [...]
>> --8<---------------cut here---------------end--------------->8---
>>
>> I cannot reproduce it on hardware with 32 cores.  I suspect it has to do
>> with the number of threads used for xz compression, which defaults to
>> the number of cores, and some of the build machines on berlin have way
>> more cores.
>>
>> Ludo’.
>
> I tried a few times on a machine with 16 cores, and I can't reproduce
> either. The build succeeded every time.

Since the bug only seems to manifest on specific machines for reasons
that escape me, I went ahead and arranged to just skip it in commit
f762d63ec072500cd327ec5342c2f5434c267222.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.