GNU bug report logs - #73818
[PATCH 0/2] Reduce size of openmpi@5 closure

Previous Next

Package: guix-patches;

Reported by: Ludovic Courtès <ludo <at> gnu.org>

Date: Tue, 15 Oct 2024 12:44:01 UTC

Severity: normal

Tags: patch

Done: Ludovic Courtès <ludovic.courtes <at> inria.fr>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 73818 in the body.
You can then email your comments to 73818 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to guix-patches <at> gnu.org:
bug#73818; Package guix-patches. (Tue, 15 Oct 2024 12:44:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Ludovic Courtès <ludo <at> gnu.org>:
New bug report received and forwarded. Copy sent to guix-patches <at> gnu.org. (Tue, 15 Oct 2024 12:44:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>, romain.garbage <at> inria.fr
Subject: [PATCH 0/2] Reduce size of openmpi <at> 5 closure
Date: Tue, 15 Oct 2024 14:42:39 +0200
Hello!

This patch set reduces the closure size of openmpi <at> 5 from
605 MiB to 249 MiB.  It also removes potentially misleading
argument inheritance from openmpi <at> 4 to openmpi <at> 5.

Thoughts?

Ludo'.

Ludovic Courtès (2):
  gnu: openmpi <at> 5: Remove reference to GCC and other build tools.
  gnu: prrte: Remove reference to GCC and other build tools.

 gnu/packages/mpi.scm      | 59 ++++++++++++++++++++++-----------------
 gnu/packages/parallel.scm | 22 +++++++++++++--
 2 files changed, 53 insertions(+), 28 deletions(-)


base-commit: 6757bfdfc0b22a1e23a3d33566155550182244fc
-- 
2.46.0





Information forwarded to guix-patches <at> gnu.org:
bug#73818; Package guix-patches. (Tue, 15 Oct 2024 12:46:01 GMT) Full text and rfc822 format available.

Message #8 received at 73818 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 73818 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludovic.courtes <at> inria.fr>
Subject: [PATCH 1/2] gnu: openmpi <at> 5: Remove reference to GCC and other build
 tools.
Date: Tue, 15 Oct 2024 14:45:03 +0200
From: Ludovic Courtès <ludovic.courtes <at> inria.fr>

This removes GCC from the direct references of openmpi <at> 5.

* gnu/packages/mpi.scm (openmpi-5)[arguments]: Replace arguments
entirely instead of using ‘substitute-keyword-arguments’.  Reintroduce
‘remove-absolute’ phase.  Remove ‘find-opensm-headers’ and ‘remove-logs’
phases, which are no longer needed.

Change-Id: Ia3b25230488ca13310cd5bb2ae45f6990e906fa5
---
 gnu/packages/mpi.scm | 59 +++++++++++++++++++++++++-------------------
 1 file changed, 34 insertions(+), 25 deletions(-)

diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm
index 75cfd4af66..6e1f972eb8 100644
--- a/gnu/packages/mpi.scm
+++ b/gnu/packages/mpi.scm
@@ -339,37 +339,46 @@ (define-public openmpi-5
 
     (outputs '("out" "debug"))
     (arguments
-     (substitute-keyword-arguments (package-arguments openmpi)
-       ((#:configure-flags _)
-        #~(list "--enable-mpi-ext=affinity" ;cr doesn't work
-                "--with-sge"
+     (list #:configure-flags
+           #~(list "--enable-mpi-ext=affinity"         ;cr doesn't work
+                   "--with-sge"
 
-                #$@(if (package? (this-package-input "valgrind"))
-                       #~("--enable-memchecker"
-                          "--with-valgrind")
-                       #~("--without-valgrind"))
+                   #$@(if (package? (this-package-input "valgrind"))
+                          #~("--enable-memchecker"
+                             "--with-valgrind")
+                          #~("--without-valgrind"))
 
-                "--with-hwloc=external"
-                "--with-libevent"
+                   "--with-hwloc=external"
+                   "--with-libevent"
 
-                ;; This replaces --enable-mpirun-prefix-by-default wich is deprecated
-                ;; since 5.x.
-                "--enable-prte-prefix-by-default"
+                   ;; This replaces --enable-mpirun-prefix-by-default wich is deprecated
+                   ;; since 5.x.
+                   "--enable-prte-prefix-by-default"
 
-                ;; Enable support for the 'Process Management Interface for Exascale'
-                ;; (PMIx) used e.g. by Slurm for the management communication and
-                ;; coordination of MPI processes.
-                (string-append "--with-pmix=" #$(this-package-input "openpmix"))
-                (string-append "--with-prrte=" #$(this-package-input "prrte"))
+                   ;; Enable support for the 'Process Management Interface for Exascale'
+                   ;; (PMIx) used e.g. by Slurm for the management communication and
+                   ;; coordination of MPI processes.
+                   (string-append "--with-pmix=" #$(this-package-input "openpmix"))
+                   (string-append "--with-prrte=" #$(this-package-input "prrte"))
 
-                ;; Since 5.x, Infiniband support is provided by ucx.
-                ;; See https://docs.open-mpi.org/en/main/release-notes/networks.html#miscellaneous-network-notes
-                (string-append "--with-ucx=" #$(this-package-input "ucx"))))
+                   ;; Since 5.x, Infiniband support is provided by ucx.
+                   ;; See https://docs.open-mpi.org/en/main/release-notes/networks.html#miscellaneous-network-notes
+                   (string-append "--with-ucx=" #$(this-package-input "ucx")))
 
-       ((#:phases phases)
-        #~(modify-phases #$phases
-            (delete 'remove-absolute)
-            (delete 'scrub-timestamps)))))))
+           #:phases
+           #~(modify-phases %standard-phases
+               (add-before 'build 'remove-absolute
+                 (lambda _
+                   ;; Remove compiler absolute file names (OPAL_FC_ABSOLUTE
+                   ;; etc.) to reduce the closure size.  See
+                   ;; <https://lists.gnu.org/archive/html/guix-devel/2017-07/msg00388.html>
+                   ;; and
+                   ;; <https://www.mail-archive.com/users <at> lists.open-mpi.org//msg31397.html>.
+                   (substitute* '("oshmem/tools/oshmem_info/param.c"
+                                  "ompi/tools/ompi_info/param.c")
+                     (("_ABSOLUTE") "")))))
+
+           #:disallowed-references (list (canonical-package gcc))))))
 
 (define-public openmpi-c++
   (package/inherit openmpi
-- 
2.46.0





Information forwarded to guix-patches <at> gnu.org:
bug#73818; Package guix-patches. (Tue, 15 Oct 2024 12:46:02 GMT) Full text and rfc822 format available.

Message #11 received at 73818 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 73818 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludovic.courtes <at> inria.fr>
Subject: [PATCH 2/2] gnu: prrte: Remove reference to GCC and other build tools.
Date: Tue, 15 Oct 2024 14:45:04 +0200
From: Ludovic Courtès <ludovic.courtes <at> inria.fr>

This reduces the closure of ‘prrte’ from 305 MiB to 99 MiB.

* gnu/packages/parallel.scm (prrte)[arguments]: Add #:phases.

Change-Id: Idc26cf5899dd5fa8378535fc8530dbdb3e20608d
---
 gnu/packages/parallel.scm | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index fc96136065..186aebad45 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -711,9 +711,25 @@ (define-public prrte
               "0wiy0vk37v4db1jgxza8bci0cczcvj34dalzsrlz05dk45zb7dl3"))))
    (build-system gnu-build-system)
    (arguments
-    (list #:configure-flags #~(list (string-append "--with-hwloc="
-                                                   (assoc-ref %build-inputs "hwloc"))
-                                    (string-append "--with-pmix=" #$(this-package-input "openpmix")))))
+    (list #:configure-flags
+          #~(list (string-append "--with-hwloc="
+                                 (assoc-ref %build-inputs "hwloc"))
+                  (string-append "--with-pmix="
+                                 #$(this-package-input "openpmix")))
+
+          #:phases
+          #~(modify-phases %standard-phases
+              (add-after 'unpack 'remove-absolute-references
+                (lambda _
+                  ;; Remove references to GCC, the shell, etc. (shown by
+                  ;; 'prte_info') to reduce the closure size.
+                  (substitute* "src/tools/prte_info/param.c"
+                    (("_ABSOLUTE")
+                     "")
+                    (("PRTE_CONFIGURE_CLI")
+                     "\"[elided to reduce closure]\"")))))
+
+          #:disallowed-references (list (canonical-package gcc))))
    (inputs (list libevent
                  `(,hwloc "lib")
                  openpmix))
-- 
2.46.0





Reply sent to Ludovic Courtès <ludovic.courtes <at> inria.fr>:
You have taken responsibility. (Fri, 25 Oct 2024 10:39:01 GMT) Full text and rfc822 format available.

Notification sent to Ludovic Courtès <ludo <at> gnu.org>:
bug acknowledged by developer. (Fri, 25 Oct 2024 10:39:02 GMT) Full text and rfc822 format available.

Message #16 received at 73818-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
To: 73818-done <at> debbugs.gnu.org
Cc: romain.garbage <at> inria.fr
Subject: Re: [bug#73818] [PATCH 0/2] Reduce size of openmpi <at> 5 closure
Date: Fri, 25 Oct 2024 12:38:16 +0200
Ludovic Courtès <ludo <at> gnu.org> skribis:

> This patch set reduces the closure size of openmpi <at> 5 from
> 605 MiB to 249 MiB.  It also removes potentially misleading
> argument inheritance from openmpi <at> 4 to openmpi <at> 5.

Pushed:

  38af41c4af * gnu: prrte: Remove reference to GCC and other build tools.
  30e469cc9c * gnu: openmpi <at> 5: Remove reference to GCC and other build tools.

Ludo'.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Fri, 22 Nov 2024 12:24:06 GMT) Full text and rfc822 format available.

This bug report was last modified 264 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.