GNU bug report logs - #31434
[PATCH 0/2] OpenMPI 3.0

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 31434 in the body.
You can then email your comments to 31434 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: bavier <at> cray.com, Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 0/2] OpenMPI 3.0
Date: Sat, 12 May 2018 18:01:21 +0200

Hello,

The attached patch updates OpenMPI to 3.0.1.  The ‘superlu-dist’ upgrade
was necessary to get there.

I was able to rebuild everything that depends on it on x86_64, except
‘dealii-openmpi’, where GCC would eat all the memory of the machine
before completing (I suppose it might work with --cores=1 but I haven’t
tried.)

Ludo’.

Ludovic Courtès (2):
  gnu: superlu-dist: Update to 5.3.0.
  gnu: openmpi: Update to 3.0.1.

 gnu/packages/maths.scm | 87 ++++++++++++++++++++++++++++++++++--------
 gnu/packages/mpi.scm   |  8 ++--
 2 files changed, 76 insertions(+), 19 deletions(-)

-- 
2.17.0

Message #8 received at 31434 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 31434 <at> debbugs.gnu.org
Cc: bavier <at> cray.com,
 Ludovic Courtès <ludovic.courtes <at> inria.fr>
Subject: [PATCH 1/2] gnu: superlu-dist: Update to 5.3.0.
Date: Sat, 12 May 2018 18:02:31 +0200

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>

* gnu/packages/maths.scm (superlu-dist): Update to 5.3.0.
[source](snippet): Adapt file name and function names to new version.
Remove 'delete-file' call.
---
 gnu/packages/maths.scm | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 2e6f79ad5..d2edecc54 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -2109,35 +2109,34 @@ also provides threshold-based ILU factorization preconditioners.")
 (define-public superlu-dist
   (package
     (name "superlu-dist")
-    (version "3.3")
+    (version "5.3.0")
     (source
      (origin
        (method url-fetch)
        (uri (string-append "http://crd-legacy.lbl.gov/~xiaoye/SuperLU/"
                            "superlu_dist_" version ".tar.gz"))
        (sha256
-        (base32 "1hnak09yxxp026blq8zhrl7685yip16svwngh1wysqxf8z48vzfj"))
-              (modules '((guix build utils)))
+        (base32 "0ja5ihqivkda1wd58y4lmzvmwssm9g91f70c5q0fzwhng6580h6y"))
+       (modules '((guix build utils)))
        (snippet
         ;; Replace the non-free implementation of MC64 with a stub
         '(begin
            (use-modules (ice-9 regex)
                         (ice-9 rdelim))
-           (call-with-output-file "SRC/mc64ad.c"
+           (call-with-output-file "SRC/mc64ad_dist.c"
              (lambda (port)
                (display "
 #include <stdio.h>
 #include <stdlib.h>
-void mc64id_(int *a) {
+void mc64id_dist(int *a) {
   fprintf (stderr, \"SuperLU_DIST: non-free MC64 not available.  Aborting.\\n\");
   abort ();
 }
-void mc64ad_ (int *a, int *b, int *c, int *d, int *e, double *f, int *g,
+void mc64ad_dist (int *a, int *b, int *c, int *d, int *e, double *f, int *g,
               int *h, int *i, int *j, int *k, double *l, int *m, int *n) {
   fprintf (stderr, \"SuperLU_DIST: non-free MC64 not available.  Aborting.\\n\");
   abort ();
 }\n" port)))
-           (delete-file "SRC/mc64ad.f.bak")
            (substitute* "SRC/util.c"    ;adjust default algorithm
              (("RowPerm[[:blank:]]*=[[:blank:]]*LargeDiag")
               "RowPerm = NOROWPERM"))))
-- 
2.17.0

Message #11 received at 31434 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 31434 <at> debbugs.gnu.org
Cc: bavier <at> cray.com,
 Ludovic Courtès <ludovic.courtes <at> inria.fr>
Subject: [PATCH 2/2] gnu: openmpi: Update to 3.0.1.
Date: Sat, 12 May 2018 18:02:32 +0200

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>

* gnu/packages/mpi.scm (openmpi): Update to 3.0.1.
[arguments]: In 'remove-absolute' phase, adjust romio file name.
* gnu/packages/maths.scm (scalapack)[arguments]: Add
'set-test-environment' phase.
(slepc-openmpi)[arguments]: Likewise.
(mumps-openmpi)[arguments]: Likewise.
(superlu-dist)[arguments]: In 'check' phase, set
'OMPI_MCA_plm_rsh_agent'.
(scotch)[arguments]: In 'build-esmumps', likewise.
(p4est)[arguments]: Add 'set-test-environment' phase.
(hypre-openmpi)[arguments]: Likewise.
---
 gnu/packages/maths.scm | 74 +++++++++++++++++++++++++++++++++++++-----
 gnu/packages/mpi.scm   |  8 ++---
 2 files changed, 70 insertions(+), 12 deletions(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index d2edecc54..bc2e1be37 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -554,7 +554,16 @@ problems in numerical linear algebra.")
        ("fortran" ,gfortran)
        ("lapack" ,lapack)))             ;for testing only
     (arguments
-     `(#:configure-flags `("-DBUILD_SHARED_LIBS:BOOL=YES")))
+     `(#:configure-flags `("-DBUILD_SHARED_LIBS:BOOL=YES")
+       #:phases (modify-phases %standard-phases
+                  (add-before 'check 'set-test-environment
+                    (lambda _
+                      ;; By default, running the test suite would fail because
+                      ;; 'ssh' could not be found in $PATH.  Define this
+                      ;; variable to placate Open MPI without adding a
+                      ;; dependency on OpenSSH (the agent isn't used anyway.)
+                      (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+                      #t)))))
     (home-page "http://www.netlib.org/scalapack/")
     (synopsis "Library for scalable numerical linear algebra")
     (description
@@ -1786,6 +1795,18 @@ arising after the discretization of partial differential equations.")
 (define-public slepc-openmpi
   (package (inherit slepc)
     (name "slepc-openmpi")
+    (arguments
+     (substitute-keyword-arguments (package-arguments slepc)
+       ((#:phases phases '%standard-phases)
+        `(modify-phases ,phases
+           (add-before 'check 'set-test-environment
+             (lambda _
+               ;; By default, running the test suite would fail because 'ssh'
+               ;; could not be found in $PATH.  Define this variable to
+               ;; placate Open MPI without adding a dependency on OpenSSH (the
+               ;; agent isn't used anyway.)
+               (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+               #t))))))
     (inputs
      `(("mpi" ,openmpi)
        ("arpack" ,arpack-ng-openmpi)
@@ -1955,11 +1976,18 @@ sparse system of linear equations A x = b using Guassian elimination.")
      (substitute-keyword-arguments (package-arguments mumps)
        ((#:phases phases)
         `(modify-phases ,phases
-           (replace
-            'check
-            (lambda _
-              ((assoc-ref ,phases 'check)
-               #:exec-prefix '("mpirun" "-n" "2"))))))))
+           (add-before 'check 'set-test-environment
+             (lambda _
+               ;; By default, running the test suite would fail because 'ssh'
+               ;; could not be found in $PATH.  Define this variable to
+               ;; placate Open MPI without adding a dependency on OpenSSH (the
+               ;; agent isn't used anyway.)
+               (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+               #t))
+           (replace 'check
+             (lambda _
+               ((assoc-ref ,phases 'check)
+                #:exec-prefix '("mpirun" "-n" "2"))))))))
     (synopsis "Multifrontal sparse direct solver (with MPI)")))
 
 (define-public mumps-metis-openmpi
@@ -2201,6 +2229,11 @@ CDEFS       = -DAdd_"
              #t))
          (replace 'check
            (lambda _
+             ;; By default, running the test suite would fail because 'ssh'
+             ;; could not be found in $PATH.  Define this variable to placate
+             ;; Open MPI without adding a dependency on OpenSSH (the agent
+             ;; isn't used anyway.)
+             (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
              (with-directory-excursion "EXAMPLE"
                (and
                 (zero? (system* "mpirun" "-n" "2"
@@ -2295,6 +2328,12 @@ YACC = bison -pscotchyy -y -b y
          (add-after
           'build 'build-esmumps
           (lambda _
+            ;; By default, running the test suite would fail because 'ssh'
+            ;; could not be found in $PATH.  Define this variable to placate
+            ;; Open MPI without adding a dependency on OpenSSH (the agent
+            ;; isn't used anyway.)
+            (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+
             (zero? (system* "make"
                             (format #f "-j~a" (parallel-job-count))
                             "esmumps"))))
@@ -2474,7 +2513,16 @@ schemes.")
                                            " -lopenblas")
                            ,(string-append "LAPACK_LIBS=-L"
                                            (assoc-ref %build-inputs "lapack")
-                                           " -llapack"))))
+                                           " -llapack"))
+       #:phases (modify-phases %standard-phases
+                  (add-before 'check 'set-test-environment
+                    (lambda _
+                      ;; By default, running the test suite would fail because
+                      ;; 'ssh' could not be found in $PATH.  Define this
+                      ;; variable to placate Open MPI without adding a
+                      ;; dependency on OpenSSH (the agent isn't used anyway.)
+                      (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+                      #t)))))
     (home-page "http://www.p4est.org")
     (synopsis "Adaptive mesh refinement on forests of octrees")
     (description
@@ -3531,7 +3579,17 @@ problems.")
      (substitute-keyword-arguments (package-arguments hypre)
        ((#:configure-flags flags)
         ``("--with-MPI"
-           ,@(delete "--without-MPI" ,flags)))))
+           ,@(delete "--without-MPI" ,flags)))
+       ((#:phases phases)
+        `(modify-phases ,phases
+           (add-before 'check 'set-test-environment
+             (lambda _
+               ;; By default, running the test suite would fail because 'ssh'
+               ;; could not be found in $PATH.  Define this variable to
+               ;; placate Open MPI without adding a dependency on OpenSSH (the
+               ;; agent isn't used anyway.)
+               (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+               #t))))))
     (synopsis "Parallel solvers and preconditioners for linear equations")
     (description
      "HYPRE is a software library of high performance preconditioners and
diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm
index 5dfb26ae7..5496ea9c9 100644
--- a/gnu/packages/mpi.scm
+++ b/gnu/packages/mpi.scm
@@ -153,7 +153,7 @@ bind processes, and much more.")
 (define-public openmpi
   (package
     (name "openmpi")
-    (version "1.10.7")
+    (version "3.0.1")
     (source
      (origin
       (method url-fetch)
@@ -162,7 +162,7 @@ bind processes, and much more.")
                           "/downloads/openmpi-" version ".tar.bz2"))
       (sha256
        (base32
-        "142s1vny9gllkq336yafxayjgcirj2jv0ddabj879jgya7hyr2d0"))))
+        "0pbqrm5faf57nasy1s81wqivl7zvxmv8lzjh8hvb0f3qxv8m0d36"))))
     (build-system gnu-build-system)
     (inputs
      `(("hwloc" ,hwloc "lib")
@@ -205,8 +205,8 @@ bind processes, and much more.")
                         (("_ABSOLUTE") ""))
                       ;; Avoid valgrind (which pulls in gdb etc.).
                       (substitute*
-                          '("./ompi/mca/io/romio/src/io_romio_component.c")
-                        (("MCA_io_romio_COMPLETE_CONFIGURE_FLAGS")
+                          '("./ompi/mca/io/romio314/src/io_romio314_component.c")
+                        (("MCA_io_romio314_COMPLETE_CONFIGURE_FLAGS")
                          "\"[elided to reduce closure]\""))
                       #t))
                   (add-before 'build 'scrub-timestamps ;reproducibility
-- 
2.17.0

Message #14 received at 31434 <at> debbugs.gnu.org (full text, mbox):

From: ludovic.courtes <at> inria.fr (Ludovic Courtès)
To: 31434 <at> debbugs.gnu.org, bavier <at> cray.com
Subject: Re: [bug#31434] [PATCH 0/2] OpenMPI 3.0
Date: Wed, 23 May 2018 10:09:35 +0200

Hello Eric,

Ludovic Courtès <ludo <at> gnu.org> skribis:

> The attached patch updates OpenMPI to 3.0.1.  The ‘superlu-dist’ upgrade
> was necessary to get there.

Did you have a chance to look into it?  You mentioned on IRC that there
might be a better way than setting OMPI_MCA_plm_rsh_agent.

Thanks in advance!

Ludo’.

Message #17 received at 31434 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: 31434 <at> debbugs.gnu.org
Cc: bavier <at> cray.com
Subject: Re: [bug#31434] [PATCH 0/2] OpenMPI 3.0
Date: Wed, 23 May 2018 14:00:23 +0200

A somewhat related issue that a colleague pointed out is whether we
should build with ‘--enable-mpirun-prefix-by-default’, which README
documents like this:

--8<---------------cut here---------------start------------->8---
--enable-mpirun-prefix-by-default
  This option forces the "mpirun" command to always behave as if
  "--prefix $prefix" was present on the command line (where $prefix is
  the value given to the --prefix option to configure).  This prevents
  most rsh/ssh-based users from needing to modify their shell startup
  files to set the PATH and/or LD_LIBRARY_PATH for Open MPI on remote
  nodes.  Note, however, that such users may still desire to set PATH
  -- perhaps even in their shell startup files -- so that executables
  such as mpicc and mpirun can be found without needing to type long
  path names.  --enable-orterun-prefix-by-default is a synonym for
  this option.
--8<---------------cut here---------------end--------------->8---

It looks like something we may want.  Thoughts?

Ludo’.

Message #22 received at 31434-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: 31434-done <at> debbugs.gnu.org
Cc: bavier <at> cray.com
Subject: Re: [bug#31434] [PATCH 0/2] OpenMPI 3.0
Date: Fri, 25 May 2018 13:44:36 +0200

Hello,

ludovic.courtes <at> inria.fr (Ludovic Courtès) skribis:

> Ludovic Courtès <ludo <at> gnu.org> skribis:
>
>> The attached patch updates OpenMPI to 3.0.1.  The ‘superlu-dist’ upgrade
>> was necessary to get there.
>
> Did you have a chance to look into it?  You mentioned on IRC that there
> might be a better way than setting OMPI_MCA_plm_rsh_agent.

I went ahead and apply the patches as they were so we can have a recent
Open MPI.  We can always revisit the OMPI_MCA_plm_rsh_agent issue later.

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.