From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 01 08:54:54 2017 Received: (at submit) by debbugs.gnu.org; 1 Aug 2017 12:54:54 +0000 Received: from localhost ([127.0.0.1]:36240 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dcWhC-0004j7-Pb for submit@debbugs.gnu.org; Tue, 01 Aug 2017 08:54:54 -0400 Received: from eggs.gnu.org ([208.118.235.92]:49597) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dcWh8-0004ir-9S for submit@debbugs.gnu.org; Tue, 01 Aug 2017 08:54:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dcWh0-0007oa-CP for submit@debbugs.gnu.org; Tue, 01 Aug 2017 08:54:36 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:46553) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dcWh0-0007oN-82 for submit@debbugs.gnu.org; Tue, 01 Aug 2017 08:54:34 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40561) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dcWgx-0000tp-Bh for guix-patches@gnu.org; Tue, 01 Aug 2017 08:54:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dcWgu-0007ib-1s for guix-patches@gnu.org; Tue, 01 Aug 2017 08:54:31 -0400 Received: from tranquility.mcc.ac.uk ([130.88.200.145]:57880) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dcWgt-0007gZ-Ex for guix-patches@gnu.org; Tue, 01 Aug 2017 08:54:27 -0400 Received: from asmtp2.its.manchester.ac.uk ([130.88.13.150]) by tranquility.mcc.ac.uk with esmtps (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1dcWgr-00081e-86 for guix-patches@gnu.org; Tue, 01 Aug 2017 13:54:25 +0100 Received: from albion.it.manchester.ac.uk ([130.88.138.79]:51876 helo=i-ulialbion.it.manchester.ac.uk) by asmtp2.its.manchester.ac.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1dcWgq-0001k3-Uq for guix-patches@gnu.org; Tue, 01 Aug 2017 13:54:24 +0100 From: Dave Love To: guix-patches@gnu.org Subject: changes for openmpi Date: Tue, 01 Aug 2017 13:54:24 +0100 Message-ID: <87zibja1of.fsf@i-ulialbion.it.manchester.ac.uk> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="LABLINK-Al-Qaeda-NOCS-RCMP-Steve-Case" X-Authenticated-Sender: David Love from albion.it.manchester.ac.uk (i-ulialbion.it.manchester.ac.uk) [130.88.138.79]:51876 X-Authenticated-From: Dave.Love@manchester.ac.uk X-SA-Exim-Connect-IP: 130.88.13.150 X-SA-Exim-Mail-From: fx@gnu.org X-SA-Exim-Scanned: No (on tranquility.mcc.ac.uk); SAEximRunCond expanded to false X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) --LABLINK-Al-Qaeda-NOCS-RCMP-Steve-Case Content-Type: text/plain Here's a series with suggestions for openmpi. I hope the log messages are sufficiently explanatory, otherwise I can comment. The hwloc and valgrind changes are in line with what I'm used to using with RHEL and Debian packaging. (I know you won't particularly want to follow them, but they make sense from the point of view of a user.) I think the last one will need to be used for gfortran-specific variants, as suggested on -devel. That will take the closure back up somewhat, but what I get now is: store item total self /gnu/store/la6mj9kh7fwws233955wyp80x39ag88w-openmpi-1.10.7 134.1 9.7 7.2% /gnu/store/b8ni7680lh6j8z26dam7ki9z6f9y6pnz-hwloc-1.11.7-nogui 89.9 2.9 2.1% /gnu/store/h7mx27bl0wynlz8vjszzykqqldccfwm5-ncurses-6.0 74.3 5.7 4.2% /gnu/store/w1mrskd2ddgvkr727r9241g8dlkf0rlf-gfortran-5.4.0-lib 73.0 34.5 25.7% /gnu/store/lsidb1rk8z24c516pqw99anm57cpm8r1-numactl-2.0.11 68.9 0.3 0.2% /gnu/store/4vdik5cc02yh2hypwnwi6n6799j6srgn-libpciaccess-0.13.5 68.7 0.1 0.1% /gnu/store/dhc2iy059hi91fk55dcv79z09kp6500y-gcc-5.4.0-lib 68.6 30.1 22.4% /gnu/store/k7029k5va68lkapbzcycdzj7m5bjb4b8-bash-4.4.12 50.9 5.4 4.1% /gnu/store/hvyk1qyph1hihfmym1w271ygp84adb0v-readline-7.0 45.5 1.3 1.0% /gnu/store/q1x4v3x8v2g59d244hl7k0i1n4h83c9a-ncurses-6.0 44.2 5.7 4.2% /gnu/store/rmjlycdgiq8pfy5hfi42qhw3k7p6kdav-glibc-2.25 38.5 37.1 27.7% /gnu/store/02426nwiy32cscm4h83729vn5ws1gs2i-bash-static-4.4.12 1.4 1.4 1.1% total: 134.1 MiB --LABLINK-Al-Qaeda-NOCS-RCMP-Steve-Case Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0002-gnu-Add-openmpi-thread-multiple-and-modify-openmpi-a.patch Content-Transfer-Encoding: quoted-printable >From 9f6330bcc336eae6aedc78f56c1ebbbf30ae0b2d Mon Sep 17 00:00:00 2001 From: Dave Love Date: Mon, 31 Jul 2017 14:54:29 +0100 Subject: [PATCH 2/8] gnu: Add openmpi-thread-multiple and modify openmpi accordingly. thread-multiple support hurts performance even if it's not used. * gnu/packages/mpi.scm (openmpi)[arguments]: Don't enable thread-multiple. --- gnu/packages/mpi.scm | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm index 968eb8870..aa0b78237 100644 --- a/gnu/packages/mpi.scm +++ b/gnu/packages/mpi.scm @@ -133,7 +133,6 @@ bind processes, and much more.") (arguments `(#:configure-flags `("--enable-static" =20 - "--enable-mpi-thread-multiple" "--enable-builtin-atomics" =20 "--enable-mpi-ext=3Dall" @@ -183,2 +182,18 @@ software vendors, application developers and computer = science researchers.") ;; See file://LICENSE (license bsd-2))) + +(define-public openmpi-thread-multiple + (package + (inherit openmpi) + (name "openmpi-thread-multiple") + (arguments + (substitute-keyword-arguments (package-arguments openmpi) + ((#:configure-flags flags) + `(cons "--enable-mpi-thread-multiple" ,flags)))) + (description (string-append (package-description openmpi) + "\ + +This version has an implementation of `MPI_Init_thread' that provides +=E2=80=98MPI_THREAD_MULTIPLE=E2=80=99. This won't work correctly with all= transports (e.g. +openib), and the performance is generally worse than the vanilla openmpi +package, which only provides =E2=80=98MPI_THREAD_FUNNELED=E2=80=99.")))) --=20 2.11.0 --LABLINK-Al-Qaeda-NOCS-RCMP-Steve-Case Content-Type: text/x-diff Content-Disposition: attachment; filename=0003-gnu-openmpi-Remove-static-output.patch >From aab1baa5b90088c7d72366c669e47e273bbd9084 Mon Sep 17 00:00:00 2001 From: Dave Love Date: Mon, 31 Jul 2017 14:57:21 +0100 Subject: [PATCH 3/8] gnu: openmpi: Remove "static" output. --enable-static removes dynamically-loaded MCA components. * gnu/packages/mpi.scm (openmpi)[outputs]: Remove field. [arguments]: Don't configure with --enable-static; remove 'move-static-libraries' phase. --- gnu/packages/mpi.scm | 19 +------------------ 1 file changed, 1 insertion(+), 18 deletions(-) diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm index aa0b78237..e166d23d8 100644 --- a/gnu/packages/mpi.scm +++ b/gnu/packages/mpi.scm @@ -122,7 +122,6 @@ bind processes, and much more.") (base32 "142s1vny9gllkq336yafxayjgcirj2jv0ddabj879jgya7hyr2d0")))) (build-system gnu-build-system) - (outputs '("out" "static")) (inputs `(("hwloc" ,hwloc "lib") ("gfortran" ,gfortran) @@ -131,9 +130,7 @@ bind processes, and much more.") `(("pkg-config" ,pkg-config) ("perl" ,perl))) (arguments - `(#:configure-flags `("--enable-static" - - "--enable-builtin-atomics" + `(#:configure-flags `("--enable-builtin-atomics" "--enable-mpi-ext=all" "--with-devel-headers" @@ -155,20 +152,6 @@ bind processes, and much more.") (lambda* (#:key outputs #:allow-other-keys) (let ((out (assoc-ref outputs "out"))) (for-each delete-file (find-files out "config.log")) - #t))) - (add-after 'install 'move-static-libraries - (lambda* (#:key outputs #:allow-other-keys) - ;; Move 19 MiB of static libraries to 'static'. - (let* ((out (assoc-ref outputs "out")) - (static (assoc-ref outputs "static")) - (lib (string-append out "/lib")) - (slib (string-append static "/lib"))) - (mkdir-p slib) - (for-each (lambda (file) - (rename-file - file - (string-append slib "/" (basename file)))) - (find-files lib "\\.a$")) #t)))))) (home-page "http://www.open-mpi.org") (synopsis "MPI-3 implementation") -- 2.11.0 --LABLINK-Al-Qaeda-NOCS-RCMP-Steve-Case Content-Type: text/x-diff Content-Disposition: attachment; filename=0004-gnu-hwloc-Replace-lib-output-with-nogui-containing-a.patch >From 67a59e734dd451d1e64d450dcebeb23d60996f3e Mon Sep 17 00:00:00 2001 From: Dave Love Date: Mon, 31 Jul 2017 14:58:39 +0100 Subject: [PATCH] gnu: hwloc: Replace "lib" output with "nogui", containing all but lstopo. A compute node typically wants the non-GUI programs available, which still have a small closure. * mpi.scm (hwloc)[outputs]: Replace lib with nogui. (hwloc)[arguments]: Change configure --prefix; use "nogui" output, not "lib"; populate "all" output. (openmpi)[inputs]: Use hwloc-nogui. --- gnu/packages/mpi.scm | 44 ++++++++++++++++++++++++++++++-------------- gnu/packages/parallel.scm | 2 +- 2 files changed, 31 insertions(+), 15 deletions(-) diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm index 385f9985a..66ffd7bdd 100644 --- a/gnu/packages/mpi.scm +++ b/gnu/packages/mpi.scm @@ -39,6 +39,7 @@ #:use-module (gnu packages pkg-config) #:use-module (gnu packages valgrind)) +;; Fixme: Replace hwloc-dump-hwdata.service with shepherd equivalent (define-public hwloc (package (name "hwloc") @@ -53,7 +54,7 @@ "0acph1mf7588hfx8ds26ncr6nw5fd9x92adm11fwin7f93i10sdb")))) (build-system gnu-build-system) (outputs '("out" ;'lstopo' & co., depends on Cairo, libx11, etc. - "lib" ;small closure + "nogui" ;small closure "debug")) (inputs `(("libx11" ,libx11) @@ -71,27 +72,42 @@ (native-inputs `(("pkg-config" ,pkg-config))) (arguments - `(#:configure-flags '("--localstatedir=/var") + `(#:configure-flags (list "--localstatedir=/var" + (string-append "--prefix=" + (assoc-ref %outputs "nogui"))) #:phases (modify-phases %standard-phases (add-after 'install 'refine-libnuma ;; Give -L arguments for libraries to avoid propagation (lambda* (#:key inputs outputs #:allow-other-keys) - (let ((out (assoc-ref outputs "lib")) - (numa (assoc-ref inputs "numactl"))) - (substitute* (map (lambda (f) (string-append out "/" f)) + (let ((out (assoc-ref outputs "out")) + (numa (assoc-ref inputs "numactl")) + (nogui (assoc-ref outputs "nogui"))) + (substitute* (map (lambda (f) (string-append nogui "/" f)) '("lib/pkgconfig/hwloc.pc" "lib/libhwloc.la")) (("-lnuma" lib) (string-append "-L" numa "/lib " lib)))))) - (add-after 'install 'avoid-circular-references + (add-after 'install 'move-lstopo (lambda* (#:key outputs #:allow-other-keys) - (let ((lib (assoc-ref outputs "lib"))) - ;; Suppress the 'prefix=' and 'exec_prefix=' lines so that the - ;; "lib" output doesn't refer to "out". - (substitute* (string-append lib "/lib/pkgconfig/hwloc.pc") - (("^.*prefix=.*$") - "")) - #t)))))) + (let* ((out (assoc-ref outputs "out")) + (nogui (assoc-ref outputs "nogui")) + (binout (string-append out "/bin")) + (appsout (string-append out "/share/applications")) + (man1out (string-append out "/share/man/man1"))) + (mkdir-p binout) + (mkdir-p appsout) + (mkdir-p man1out) + (and + (delete-file (string-append nogui "/share/man/man1/lstopo.1")) + (copy-file + (string-append nogui "/share/man/man1/lstopo-no-graphics.1") + (string-append man1out "/lstopo.1")) + (zero? + (system (format #f "mv ~a/bin/lstopo ~a" nogui binout))) + (zero? + (system + (format #f "mv ~a/share/applications/lstopo.desktop ~a" + nogui appsout)))))))))) (home-page "https://www.open-mpi.org/projects/hwloc/") (synopsis "Abstraction of hardware architectures") (description @@ -123,7 +139,7 @@ bind processes, and much more.") "142s1vny9gllkq336yafxayjgcirj2jv0ddabj879jgya7hyr2d0")))) (build-system gnu-build-system) (inputs - `(("hwloc" ,hwloc "lib") + `(("hwloc" ,hwloc "nogui") ("gfortran" ,gfortran) ("valgrind" ,valgrind))) (native-inputs diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm index 179fa7ed3..c320dfa62 100644 --- a/gnu/packages/parallel.scm +++ b/gnu/packages/parallel.scm @@ -113,7 +113,7 @@ and they are executed on lists of files, hosts, users or other items.") ;; in particular mysql and gtk+. (inputs `(("expect" ,expect) ("freeipmi" ,freeipmi) - ("hwloc" ,hwloc "lib") + ("hwloc" ,hwloc "nogui") ("json-c" ,json-c) ("linux-pam" , linux-pam) ("munge" ,munge) -- 2.11.0 --LABLINK-Al-Qaeda-NOCS-RCMP-Steve-Case Content-Type: text/x-diff; charset=iso-8859-1 Content-Disposition: attachment; filename=0005-gnu-valgrind-Add-doc-and-openmpi-outputs.patch Content-Transfer-Encoding: quoted-printable >From 1772aa47c3bc71521340d7f569d4d906ab7f53e9 Mon Sep 17 00:00:00 2001 From: Dave Love Date: Mon, 31 Jul 2017 15:01:59 +0100 Subject: [PATCH] gnu: valgrind: Add "doc" and "openmpi" outputs. Also don't configure openmpi with valgrind; rely on the wrapper library from the valgrind package, like Fedora and Debian. * gnu/packages/valgrind.scm (valgrind)[outputs]: New field. [arguments]: Add install-doc and install-openmpi phases. [description]: Mention openmpi output. * gnu/packages/mpi.scm (openmpi)[arguments]: Don't configure with valgrind. --- gnu/packages/mpi.scm | 9 ++------- gnu/packages/valgrind.scm | 25 +++++++++++++++++++++++-- 2 files changed, 25 insertions(+), 9 deletions(-) diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm index 64d1f497b..f3fc83b26 100644 --- a/gnu/packages/mpi.scm +++ b/gnu/packages/mpi.scm @@ -36,8 +36,7 @@ #:use-module (gnu packages xml) #:use-module (gnu packages perl) #:use-module (gnu packages ncurses) - #:use-module (gnu packages pkg-config) - #:use-module (gnu packages valgrind)) + #:use-module (gnu packages pkg-config)) =20 ;; Fixme: Replace hwloc-dump-hwdata.service with shepherd equivalent (define-public hwloc @@ -140,8 +139,7 @@ bind processes, and much more.") (build-system gnu-build-system) (inputs `(("hwloc" ,hwloc "nogui") - ("gfortran" ,gfortran) - ("valgrind" ,valgrind))) + ("gfortran" ,gfortran))) (native-inputs `(("pkg-config" ,pkg-config) ("perl" ,perl))) @@ -150,10 +148,7 @@ bind processes, and much more.") =20 "--enable-mpi-ext=3Dall" "--with-devel-headers" - "--enable-memchecker" "--with-sge" - ,(string-append "--with-valgrind=3D" - (assoc-ref %build-inputs "valgr= ind")) ,(string-append "--with-hwloc=3D" (assoc-ref %build-inputs "hwloc= "))) #:phases (modify-phases %standard-phases diff --git a/gnu/packages/valgrind.scm b/gnu/packages/valgrind.scm index 5f2bef16d..9bfba4761 100644 --- a/gnu/packages/valgrind.scm +++ b/gnu/packages/valgrind.scm @@ -3,6 +3,7 @@ ;;; Copyright =A9 2015 Andreas Enge ;;; Copyright =A9 2015 Mark H Weaver ;;; Copyright =A9 2016 Efraim Flashner +;;; Copyright =A9 2017 Dave Love ;;; ;;; This file is part of GNU Guix. ;;; @@ -26,6 +27,7 @@ #:use-module (guix licenses) #:use-module (gnu packages gdb) #:use-module (gnu packages perl) + #:use-module (gnu packages mpi) #:use-module (gnu packages)) =20 (define-public valgrind @@ -53,17 +55,36 @@ (("obj:/lib") "obj:*/lib") (("obj:/usr/X11R6/lib") "obj:*/lib") (("obj:/usr/lib") "obj:*/lib")) - #t)))))) + #t))) + (add-after 'install 'install-doc + (lambda* (#:key outputs #:allow-other-keys) + (let ((orig (format #f "~a/share/doc" (assoc-ref outputs "out= "))) + (dest (format #f "~a/share" (assoc-ref outputs "doc")))) + (mkdir-p dest) + (zero? (system* "mv" orig dest))))) + (add-after 'install 'install-openmpi + (lambda* (#:key outputs #:allow-other-keys) + (let ((dest (format #f "~a/lib/valgrind" + (assoc-ref outputs "openmpi")))) + (mkdir-p dest) + (zero? + (system (format #f "mv ~a/lib/valgrind/libmpiwrap* ~a" + (assoc-ref outputs "out") dest))))))))) (inputs `(;; GDB is needed to provide a sane default for `--db-command= '. ("gdb" ,gdb))) (native-inputs `(("perl" ,perl))) + (outputs '("doc" ;16 MB + "openmpi" "out")) (home-page "http://www.valgrind.org/") (synopsis "Debugging and profiling tool suite") (description "Valgrind is an instrumentation framework for building dynamic analys= is tools. There are Valgrind tools that can automatically detect many memory management and threading bugs, and profile your programs in detail. You c= an -also use Valgrind to build new tools.") +also use Valgrind to build new tools. + +The openmpi output contains a wrapper library for openmpi, as described +in the documenation.") (license gpl2+) =20 ;; Building VEX on mips64el-linux fails with "opcode not supported on = this --=20 2.11.0 --LABLINK-Al-Qaeda-NOCS-RCMP-Steve-Case Content-Type: text/x-diff Content-Disposition: attachment; filename=0006-gnu-openmpi-Modify-configuration-to-reduce-closure.patch >From a8bd87987f82e3f0f7f44a8c6d487a397ce6eda6 Mon Sep 17 00:00:00 2001 From: Dave Love Date: Mon, 31 Jul 2017 15:03:46 +0100 Subject: [PATCH 6/8] gnu: openmpi: Modify configuration to reduce closure. devel-headers are only for building MCA components, produce a large closure and aren't normally packaged. Only the affinity component of mpi-ext is useful. * mpi.scm (openmpi)[arguments]: Don't configure --with-devel-headers and --enable-mpi-ext=all. --- gnu/packages/mpi.scm | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm index f3fc83b26..bbc81e45a 100644 --- a/gnu/packages/mpi.scm +++ b/gnu/packages/mpi.scm @@ -144,10 +144,9 @@ bind processes, and much more.") `(("pkg-config" ,pkg-config) ("perl" ,perl))) (arguments - `(#:configure-flags `("--enable-builtin-atomics" + `(#:configure-flags `("--enable-builtin-atomics" ;why? - "--enable-mpi-ext=all" - "--with-devel-headers" + "--enable-mpi-ext=affinity" ;cr doesn't work "--with-sge" ,(string-append "--with-hwloc=" (assoc-ref %build-inputs "hwloc"))) -- 2.11.0 --LABLINK-Al-Qaeda-NOCS-RCMP-Steve-Case Content-Type: text/x-diff Content-Disposition: attachment; filename=0007-gnu-openmpi-Configure-without-vampirtrace.patch >From 8a0668951bc352aadc3cce5c12c9e4d9e2e7b6c6 Mon Sep 17 00:00:00 2001 From: Dave Love Date: Mon, 31 Jul 2017 15:04:44 +0100 Subject: [PATCH 7/8] gnu: openmpi: Configure without vampirtrace. VT is obsoleted by scorep (not currently packaged) and disabling it reduces the closure considerably. * mpi.scm (openmpi)[arguments]: Configure without vampirtrace. --- gnu/packages/mpi.scm | 1 + 1 file changed, 1 insertion(+) diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm index bbc81e45a..abda1cc43 100644 --- a/gnu/packages/mpi.scm +++ b/gnu/packages/mpi.scm @@ -148,6 +148,7 @@ bind processes, and much more.") "--enable-mpi-ext=affinity" ;cr doesn't work "--with-sge" + "--disable-vt" ,(string-append "--with-hwloc=" (assoc-ref %build-inputs "hwloc"))) #:phases (modify-phases %standard-phases -- 2.11.0 --LABLINK-Al-Qaeda-NOCS-RCMP-Steve-Case Content-Type: text/x-diff Content-Disposition: attachment; filename=0008-gnu-openmpi-Remove-references-to-compiler-pathnames-.patch >From ec65c9d847c30d51bf83b49b397bc1ca20b7ca11 Mon Sep 17 00:00:00 2001 From: Dave Love Date: Mon, 31 Jul 2017 17:15:19 +0100 Subject: [PATCH 8/8] gnu: openmpi: Remove references to compiler pathnames in "_info" programs. This reduces the closure greatly, but note that the Fortran .mod files are gfortran version-specific, so there should probably be development packages for each incompatible version. (The runtime is supposed to be more-or-less version-independent unless the libgfortran soname changes.) There may still be a case for a separate runtime output. * gnu/packages/mpi.scm (openmpi)[arguments]: Add "remove-absolute" phase. --- gnu/packages/mpi.scm | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm index abda1cc43..ffae04415 100644 --- a/gnu/packages/mpi.scm +++ b/gnu/packages/mpi.scm @@ -152,6 +152,14 @@ bind processes, and much more.") ,(string-append "--with-hwloc=" (assoc-ref %build-inputs "hwloc"))) #:phases (modify-phases %standard-phases + (add-before 'build 'remove-absolute + ;; Remove compilers (OPAL_FC_ABSOLUTE etc.) from closure. + (lambda _ + (substitute* '("orte/tools/orte-info/param.c" + "oshmem/tools/oshmem_info/param.c" + "ompi/tools/ompi_info/param.c") + (("_ABSOLUTE") "")) + #t)) (add-before 'build 'scrub-timestamps ;reproducibility (lambda _ (substitute* '("ompi/tools/ompi_info/param.c" -- 2.11.0 --LABLINK-Al-Qaeda-NOCS-RCMP-Steve-Case-- From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 21 11:12:14 2017 Received: (at 27905) by debbugs.gnu.org; 21 Aug 2017 15:12:14 +0000 Received: from localhost ([127.0.0.1]:47968 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1djoNC-0000o7-7H for submit@debbugs.gnu.org; Mon, 21 Aug 2017 11:12:14 -0400 Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104]:22948) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1djoNA-0000nt-Rh for 27905@debbugs.gnu.org; Mon, 21 Aug 2017 11:12:13 -0400 X-IronPort-AV: E=Sophos;i="5.41,409,1498514400"; d="scan'208";a="234944876" Received: from unknown (HELO ribbon) ([193.50.110.251]) by mail3-relais-sop.national.inria.fr with ESMTP/TLS/AES256-GCM-SHA384; 21 Aug 2017 17:12:06 +0200 From: ludovic.courtes@inria.fr (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Dave Love Subject: Re: [bug#27905] changes for openmpi References: <87zibja1of.fsf@i-ulialbion.it.manchester.ac.uk> Date: Mon, 21 Aug 2017 17:12:06 +0200 In-Reply-To: <87zibja1of.fsf@i-ulialbion.it.manchester.ac.uk> (Dave Love's message of "Tue, 01 Aug 2017 13:54:24 +0100") Message-ID: <87h8x1t0op.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 27905 Cc: 27905@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hi! I=E2=80=99ve applied most of the patches. I have a few remaining questions: Dave Love skribis: >>>From 67a59e734dd451d1e64d450dcebeb23d60996f3e Mon Sep 17 00:00:00 2001 > From: Dave Love > Date: Mon, 31 Jul 2017 14:58:39 +0100 > Subject: [PATCH] gnu: hwloc: Replace "lib" output with "nogui", containin= g=20 > all but lstopo. > > A compute node typically wants the non-GUI programs available, which still > have a small closure. > > * mpi.scm (hwloc)[outputs]: Replace lib with nogui. > (hwloc)[arguments]: Change configure --prefix; use "nogui" output, > not "lib"; populate "all" output. > (openmpi)[inputs]: Use hwloc-nogui. The downside of this is that the =E2=80=9Cnogui=E2=80=9D output is less dis= coverable (and it=E2=80=99s another user-visible breakage.) Also, it shouldn=E2=80=99t make any difference to the closure size of openm= pi anyway, no? >>>From 1772aa47c3bc71521340d7f569d4d906ab7f53e9 Mon Sep 17 00:00:00 2001 > From: Dave Love > Date: Mon, 31 Jul 2017 15:01:59 +0100 > Subject: [PATCH] gnu: valgrind: Add "doc" and "openmpi" outputs. > > Also don't configure openmpi with valgrind; rely on the wrapper > library from the valgrind package, like Fedora and Debian. > > * gnu/packages/valgrind.scm (valgrind)[outputs]: New field. > [arguments]: Add install-doc and install-openmpi phases. > [description]: Mention openmpi output. > * gnu/packages/mpi.scm (openmpi)[arguments]: Don't configure with > valgrind. I=E2=80=99ve installed the doc-output-for-valgrind part, as a separate patc= h. Regarding the rest: > + (add-after 'install 'install-openmpi > + (lambda* (#:key outputs #:allow-other-keys) > + (let ((dest (format #f "~a/lib/valgrind" > + (assoc-ref outputs "openmpi")))) > + (mkdir-p dest) > + (zero? > + (system (format #f "mv ~a/lib/valgrind/libmpiwrap* ~a" > + (assoc-ref outputs "out") dest))))))))) Why move it to a separate output? After all, we can keep it in =E2=80=9Cou= t=E2=80=9D since all it costs is the size of libmpiwrap.so, right? Also, I assume that this is functionally equivalent to Open MPI=E2=80=99s built-in Valgrind support, is it? >>>From ec65c9d847c30d51bf83b49b397bc1ca20b7ca11 Mon Sep 17 00:00:00 2001 > From: Dave Love > Date: Mon, 31 Jul 2017 17:15:19 +0100 > Subject: [PATCH 8/8] gnu: openmpi: Remove references to compiler pathname= s in > "_info" programs. > > This reduces the closure greatly, but note that the Fortran .mod files are > gfortran version-specific, so there should probably be development packag= es > for each incompatible version. (The runtime is supposed to be more-or-le= ss > version-independent unless the libgfortran soname changes.) There may st= ill > be a case for a separate runtime output. > > * gnu/packages/mpi.scm (openmpi)[arguments]: Add "remove-absolute" phase. Great, I added the URL of previous discussions on this topic. With the changes I pushed the closure size is already at 378.5 instead of 700.8 MiB, pretty cool! Thank you, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 23 09:01:00 2017 Received: (at 27905) by debbugs.gnu.org; 23 Aug 2017 13:01:00 +0000 Received: from localhost ([127.0.0.1]:50576 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dkVHH-00011E-Tu for submit@debbugs.gnu.org; Wed, 23 Aug 2017 09:01:00 -0400 Received: from tranquility.mcc.ac.uk ([130.88.200.145]:10889) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dkVHF-0000xq-Ff for 27905@debbugs.gnu.org; Wed, 23 Aug 2017 09:00:58 -0400 Received: from asmtp2.its.manchester.ac.uk ([130.88.13.150]) by tranquility.mcc.ac.uk with esmtps (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1dkVHC-0002mF-JN; Wed, 23 Aug 2017 14:00:54 +0100 Received: from albion.it.manchester.ac.uk ([130.88.138.79]:35922) by asmtp2.its.manchester.ac.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1dkVHC-0004Gv-9Q; Wed, 23 Aug 2017 14:00:54 +0100 From: Dave Love To: Ludovic =?iso-8859-1?Q?Court=E8s?= Subject: Re: [bug#27905] changes for openmpi References: <87zibja1of.fsf@i-ulialbion.it.manchester.ac.uk> <87h8x1t0op.fsf@gnu.org> X-Draft-From: ("Misc" 2611) Date: Wed, 23 Aug 2017 14:00:53 +0100 In-Reply-To: <87h8x1t0op.fsf@gnu.org> ("Ludovic \=\?iso-8859-1\?Q\?Court\=E8s\?\= \=\?iso-8859-1\?Q\?\=22's\?\= message of "Mon, 21 Aug 2017 17:12:06 +0200") Message-ID: <87d17mh20q.fsf@albion.it.manchester.ac.uk> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Authenticated-Sender: David Love from albion.it.manchester.ac.uk [130.88.138.79]:35922 X-Authenticated-From: Dave.Love@manchester.ac.uk X-SA-Exim-Connect-IP: 130.88.13.150 X-SA-Exim-Mail-From: fx@gnu.org X-SA-Exim-Scanned: No (on tranquility.mcc.ac.uk); SAEximRunCond expanded to false X-Spam-Score: -5.0(?) X-Spam-Flag: NO X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: 27905 Cc: 27905@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.3 (-) Ludovic Court=C3=A8s writes: >> * mpi.scm (hwloc)[outputs]: Replace lib with nogui. >> (hwloc)[arguments]: Change configure --prefix; use "nogui" output, >> not "lib"; populate "all" output. >> (openmpi)[inputs]: Use hwloc-nogui. > > The downside of this is that the =E2=80=9Cnogui=E2=80=9D output is less d= iscoverable > (and it=E2=80=99s another user-visible breakage.) I don't think that's a problem, as people who want to avoid the GUI stuff will look for an alternative. > Also, it shouldn=E2=80=99t make any difference to the closure size of ope= nmpi > anyway, no? No, but I think you should be able to run the hwloc programs on compute nodes without requiring X support, and you sometimes need to run openmpi programs specifically with openmpi (for memory affinity, for instance). > > + (add-after 'install 'install-openmpi > > + (lambda* (#:key outputs #:allow-other-keys) > > + (let ((dest (format #f "~a/lib/valgrind" > > + (assoc-ref outputs "openmpi")))) > > + (mkdir-p dest) > > + (zero? > > + (system (format #f "mv ~a/lib/valgrind/libmpiwrap* ~a" > > + (assoc-ref outputs "out") dest))))))))) >=20 > Why move it to a separate output? After all, we can keep it in =E2=80=9C= out=E2=80=9D > since all it costs is the size of libmpiwrap.so, right? That would still pull in valgrind, which drags in a lot else (gdb, perl, python...). The support isn't commonly used as far as I can tell, and isn't configured by default -- I forgot about the performance hit . > Also, I assume that this is functionally equivalent to Open MPI=E2=80=99s > built-in Valgrind support, is it? I think so, basically, but I can ask the question. (Actually it's occurred to me that the wrapper uses the profiling interface, so it won't work together with profiling tools without pnmpi multiplexing, but you're unlikely to want to stack one with memory debugging.) Anyhow, I vote against a performance hit generally. Also, versions of the wrapper library could be provided for other MPIs when they're packaged. If memchecker really needs to be built-in, I think it should be packaged separately openmpi, as for thread-multiple support. I'll report any reply I get about the built-in support. From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 31 03:58:52 2017 Received: (at 27905) by debbugs.gnu.org; 31 Aug 2017 07:58:52 +0000 Received: from localhost ([127.0.0.1]:36066 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dnKNH-00048T-SO for submit@debbugs.gnu.org; Thu, 31 Aug 2017 03:58:52 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:28639) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dnKNG-00048F-0s for 27905@debbugs.gnu.org; Thu, 31 Aug 2017 03:58:50 -0400 X-IronPort-AV: E=Sophos;i="5.41,451,1498514400"; d="scan'208";a="288975491" Received: from unknown (HELO ribbon) ([193.50.110.184]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/AES256-GCM-SHA384; 31 Aug 2017 09:58:43 +0200 From: ludovic.courtes@inria.fr (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Dave Love Subject: Re: [bug#27905] changes for openmpi Organization: Inria Bordeaux Sud-Ouest References: <87zibja1of.fsf@i-ulialbion.it.manchester.ac.uk> <87h8x1t0op.fsf@gnu.org> <87d17mh20q.fsf@albion.it.manchester.ac.uk> X-URL: http://people.bordeaux.inria.fr/lcourtes/ X-Revolutionary-Date: 14 Fructidor an 225 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Thu, 31 Aug 2017 09:58:43 +0200 In-Reply-To: <87d17mh20q.fsf@albion.it.manchester.ac.uk> (Dave Love's message of "Wed, 23 Aug 2017 14:00:53 +0100") Message-ID: <87fuc8w4lo.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 27905 Cc: 27905@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hi Dave, Dave Love skribis: > Ludovic Court=C3=A8s writes: > >>> * mpi.scm (hwloc)[outputs]: Replace lib with nogui. >>> (hwloc)[arguments]: Change configure --prefix; use "nogui" output, >>> not "lib"; populate "all" output. >>> (openmpi)[inputs]: Use hwloc-nogui. >> >> The downside of this is that the =E2=80=9Cnogui=E2=80=9D output is less = discoverable >> (and it=E2=80=99s another user-visible breakage.) > > I don't think that's a problem, as people who want to avoid the GUI > stuff will look for an alternative. > >> Also, it shouldn=E2=80=99t make any difference to the closure size of op= enmpi >> anyway, no? > > No, but I think you should be able to run the hwloc programs on compute > nodes without requiring X support, and you sometimes need to run openmpi > programs specifically with openmpi (for memory affinity, for instance). OK so the gain over the current status (with the =E2=80=9Clib=E2=80=9D outp= ut) is that people would be able to get, say, =E2=80=98hwloc-bind=E2=80=99, without get= ting the full =E2=80=98lstopo=E2=80=99 and its dependencies, right? I guess that makes sense, though at the same time =E2=80=98lstopo=E2=80=99 = is probably the most widely used program in hwloc. Perhaps we should keep the current =E2=80=9Clib=E2=80=9D separation, and instead provide an =E2=80=9Ch= wloc-minimal=E2=80=9D package that does not depend on X11/Cairo? >> > + (add-after 'install 'install-openmpi >> > + (lambda* (#:key outputs #:allow-other-keys) >> > + (let ((dest (format #f "~a/lib/valgrind" >> > + (assoc-ref outputs "openmpi")))) >> > + (mkdir-p dest) >> > + (zero? >> > + (system (format #f "mv ~a/lib/valgrind/libmpiwrap* ~a" >> > + (assoc-ref outputs "out") dest)))))))= )) >>=20 >> Why move it to a separate output? After all, we can keep it in =E2=80= =9Cout=E2=80=9D >> since all it costs is the size of libmpiwrap.so, right? > > That would still pull in valgrind, which drags in a lot else (gdb, perl, > python...). The support isn't commonly used as far as I can tell, and > isn't configured by default -- I forgot about the performance hit > . The hunk above is within Valgrind, so I don=E2=80=99t understand what you m= ean by =E2=80=9Cthat would still pull in valgrind.=E2=80=9D My suggestion was to: 1. Remove Valgrind from the inputs of Open MPI; 2. Not add the =E2=80=98install-openmpi=E2=80=99 phase above to Valgrind. Does that make sense? Thank you, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 01 07:06:18 2017 Received: (at 27905) by debbugs.gnu.org; 1 Sep 2017 11:06:18 +0000 Received: from localhost ([127.0.0.1]:39247 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dnjmE-0007oz-52 for submit@debbugs.gnu.org; Fri, 01 Sep 2017 07:06:18 -0400 Received: from serenity.mcc.ac.uk ([130.88.200.93]:33495) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dnjmC-0007or-6k for 27905@debbugs.gnu.org; Fri, 01 Sep 2017 07:06:16 -0400 Received: from asmtp1.its.manchester.ac.uk ([130.88.13.149]) by serenity.mcc.ac.uk with esmtps (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1dnjmB-000M7S-Kb; Fri, 01 Sep 2017 12:06:15 +0100 Received: from albion.it.manchester.ac.uk ([130.88.138.79]:55388) by asmtp1.its.manchester.ac.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1dnjmB-0007Vc-C1; Fri, 01 Sep 2017 12:06:15 +0100 From: Dave Love To: Ludovic =?iso-8859-1?Q?Court=E8s?= Subject: Re: [bug#27905] changes for openmpi References: <87zibja1of.fsf@i-ulialbion.it.manchester.ac.uk> <87h8x1t0op.fsf@gnu.org> X-Draft-From: ("Misc" 2611) Date: Fri, 01 Sep 2017 12:06:14 +0100 In-Reply-To: <87h8x1t0op.fsf@gnu.org> ("Ludovic \=\?iso-8859-1\?Q\?Court\=E8s\?\= \=\?iso-8859-1\?Q\?\=22's\?\= message of "Mon, 21 Aug 2017 17:12:06 +0200") Message-ID: <87d17aznix.fsf@albion.it.manchester.ac.uk> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Authenticated-Sender: David Love from albion.it.manchester.ac.uk [130.88.138.79]:55388 X-Authenticated-From: Dave.Love@manchester.ac.uk X-SA-Exim-Connect-IP: 130.88.13.149 X-SA-Exim-Mail-From: fx@gnu.org X-SA-Exim-Scanned: No (on serenity.mcc.ac.uk); SAEximRunCond expanded to false X-Spam-Score: -5.0(?) X-Spam-Flag: NO X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: 27905 Cc: 27905@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.3 (-) Ludovic Court=C3=A8s writes: >> * mpi.scm (hwloc)[outputs]: Replace lib with nogui. >> (hwloc)[arguments]: Change configure --prefix; use "nogui" output, >> not "lib"; populate "all" output. >> (openmpi)[inputs]: Use hwloc-nogui. > > The downside of this is that the =E2=80=9Cnogui=E2=80=9D output is less d= iscoverable > (and it=E2=80=99s another user-visible breakage.) I don't understand why it's worse than currently. "hwloc" will provide the same as before, won't it? I guess developer breakage could be fixed by retaining the lib output if it matters. Maybe it's helpful to try to document what sort of stability is expected currently? > Also, it shouldn=E2=80=99t make any difference to the closure size of ope= nmpi > anyway, no? Right. It wasn't for openmpi specifically. >> + (add-after 'install 'install-openmpi >> + (lambda* (#:key outputs #:allow-other-keys) >> + (let ((dest (format #f "~a/lib/valgrind" >> + (assoc-ref outputs "openmpi")))) >> + (mkdir-p dest) >> + (zero? >> + (system (format #f "mv ~a/lib/valgrind/libmpiwrap* ~a" >> + (assoc-ref outputs "out") dest))))))))) > > Why move it to a separate output? After all, we can keep it in =E2=80=9C= out=E2=80=9D > since all it costs is the size of libmpiwrap.so, right? > > Also, I assume that this is functionally equivalent to Open MPI=E2=80=99s > built-in Valgrind support, is it? This is probably moot. It isn't entirely equivalent but, more importantly, the builtin support apparently doesn't have the performance hit which was documented; I haven't checked experimentally. See this thread, though not all my questions were answered: . The wrapper library may still be relevant for mpich-y MPIs, if they get used -- I don't know. From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 01 07:24:35 2017 Received: (at 27905) by debbugs.gnu.org; 1 Sep 2017 11:24:35 +0000 Received: from localhost ([127.0.0.1]:39271 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dnk3u-0008Ek-RP for submit@debbugs.gnu.org; Fri, 01 Sep 2017 07:24:35 -0400 Received: from clarity.mcc.ac.uk ([130.88.200.144]:62497) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dnk3s-0008Ec-Hj for 27905@debbugs.gnu.org; Fri, 01 Sep 2017 07:24:32 -0400 Received: from asmtp2.its.manchester.ac.uk ([130.88.13.150]) by clarity.mcc.ac.uk with esmtps (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1dnk3q-00052Q-T5; Fri, 01 Sep 2017 12:24:30 +0100 Received: from albion.it.manchester.ac.uk ([130.88.138.79]:55410) by asmtp2.its.manchester.ac.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1dnk3q-0003do-JV; Fri, 01 Sep 2017 12:24:30 +0100 From: Dave Love To: Ludovic =?iso-8859-1?Q?Court=E8s?= Subject: Re: [bug#27905] changes for openmpi References: <87zibja1of.fsf@i-ulialbion.it.manchester.ac.uk> <87h8x1t0op.fsf@gnu.org> <87d17mh20q.fsf@albion.it.manchester.ac.uk> <87fuc8w4lo.fsf@inria.fr> X-Draft-From: ("Misc" 2747) Date: Fri, 01 Sep 2017 12:24:30 +0100 In-Reply-To: <87fuc8w4lo.fsf@inria.fr> ("Ludovic \=\?iso-8859-1\?Q\?Court\=E8s\?\= \=\?iso-8859-1\?Q\?\=22's\?\= message of "Thu, 31 Aug 2017 09:58:43 +0200") Message-ID: <878thyzmoh.fsf@albion.it.manchester.ac.uk> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Authenticated-Sender: David Love from albion.it.manchester.ac.uk [130.88.138.79]:55410 X-Authenticated-From: Dave.Love@manchester.ac.uk X-SA-Exim-Connect-IP: 130.88.13.150 X-SA-Exim-Mail-From: fx@gnu.org X-SA-Exim-Scanned: No (on clarity.mcc.ac.uk); SAEximRunCond expanded to false X-Spam-Score: -5.0(?) X-Spam-Flag: NO X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: 27905 Cc: 27905@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.3 (-) Ludovic Court=C3=A8s writes: > Hi Dave, > > Dave Love skribis: > >> Ludovic Court=C3=A8s writes: >> >>>> * mpi.scm (hwloc)[outputs]: Replace lib with nogui. >>>> (hwloc)[arguments]: Change configure --prefix; use "nogui" output, >>>> not "lib"; populate "all" output. >>>> (openmpi)[inputs]: Use hwloc-nogui. >>> >>> The downside of this is that the =E2=80=9Cnogui=E2=80=9D output is less= discoverable >>> (and it=E2=80=99s another user-visible breakage.) >> >> I don't think that's a problem, as people who want to avoid the GUI >> stuff will look for an alternative. >> >>> Also, it shouldn=E2=80=99t make any difference to the closure size of o= penmpi >>> anyway, no? >> >> No, but I think you should be able to run the hwloc programs on compute >> nodes without requiring X support, and you sometimes need to run openmpi >> programs specifically with openmpi (for memory affinity, for instance). > > OK so the gain over the current status (with the =E2=80=9Clib=E2=80=9D ou= tput) is that > people would be able to get, say, =E2=80=98hwloc-bind=E2=80=99, without g= etting the full > =E2=80=98lstopo=E2=80=99 and its dependencies, right? Right. > I guess that makes sense, though at the same time =E2=80=98lstopo=E2=80= =99 is probably > the most widely used program in hwloc. I'm surprised at that, since it's something you'd only normally run once on a node, and I'd normally only have the nogui variant on them anyhow. (You can dump the topology and display it separately if necessary, but the graphical output is often unwieldy on recent compute nodes -- or even not-so-recent ones.) > Perhaps we should keep the > current =E2=80=9Clib=E2=80=9D separation, and instead provide an =E2=80= =9Chwloc-minimal=E2=80=9D > package that does not depend on X11/Cairo? I've no strong feelings, but I'd still call it "nogui", or similar -- more descriptive. (Debian calls it -nox and Fedora has hwloc and hwloc-gui.) The no-GUI lstopo adds little to the closure. >>> > + (add-after 'install 'install-openmpi >>> > + (lambda* (#:key outputs #:allow-other-keys) >>> > + (let ((dest (format #f "~a/lib/valgrind" >>> > + (assoc-ref outputs "openmpi")))) >>> > + (mkdir-p dest) >>> > + (zero? >>> > + (system (format #f "mv ~a/lib/valgrind/libmpiwrap* ~= a" >>> > + (assoc-ref outputs "out") dest))))))= ))) >>>=20 >>> Why move it to a separate output? After all, we can keep it in =E2=80= =9Cout=E2=80=9D >>> since all it costs is the size of libmpiwrap.so, right? >> >> That would still pull in valgrind, which drags in a lot else (gdb, perl, >> python...). The support isn't commonly used as far as I can tell, and >> isn't configured by default -- I forgot about the performance hit >> . > > The hunk above is within Valgrind, so I don=E2=80=99t understand what you= mean > by =E2=80=9Cthat would still pull in valgrind.=E2=80=9D Sorry -- confused by the time lag. The rationale was that it either depends upon, or needs when it's used -- I can't remember -- the rest, which involves gdb's closure. I'm happy to drop libmpiwrap support anyhow, at least until another MPI might need it, if I was wrong that the openmpi built-in is a performance issue. Apologies for the misinformation and wasted time, though it was based on what's on the openmpi web site. > My suggestion was to: > > 1. Remove Valgrind from the inputs of Open MPI; > 2. Not add the =E2=80=98install-openmpi=E2=80=99 phase above to Valgrin= d. > > Does that make sense? > > Thank you, > Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 04 11:10:33 2017 Received: (at 27905) by debbugs.gnu.org; 4 Sep 2017 15:10:33 +0000 Received: from localhost ([127.0.0.1]:49029 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dot1E-0005FE-Tz for submit@debbugs.gnu.org; Mon, 04 Sep 2017 11:10:33 -0400 Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104]:9279) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dot1D-0005Ew-82 for 27905@debbugs.gnu.org; Mon, 04 Sep 2017 11:10:31 -0400 X-IronPort-AV: E=Sophos;i="5.41,475,1498514400"; d="scan'208";a="236234235" Received: from unknown (HELO ribbon) ([193.50.110.184]) by mail3-relais-sop.national.inria.fr with ESMTP/TLS/AES256-GCM-SHA384; 04 Sep 2017 17:10:24 +0200 From: ludovic.courtes@inria.fr (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Dave Love Subject: Re: [bug#27905] changes for openmpi Organization: Inria References: <87zibja1of.fsf@i-ulialbion.it.manchester.ac.uk> <87h8x1t0op.fsf@gnu.org> <87d17aznix.fsf@albion.it.manchester.ac.uk> X-URL: http://people.bordeaux.inria.fr/lcourtes/ X-Revolutionary-Date: 18 Fructidor an 225 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Mon, 04 Sep 2017 17:10:24 +0200 In-Reply-To: <87d17aznix.fsf@albion.it.manchester.ac.uk> (Dave Love's message of "Fri, 01 Sep 2017 12:06:14 +0100") Message-ID: <873782xzxb.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 27905 Cc: 27905@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Dave Love skribis: > Ludovic Court=C3=A8s writes: > >>> * mpi.scm (hwloc)[outputs]: Replace lib with nogui. >>> (hwloc)[arguments]: Change configure --prefix; use "nogui" output, >>> not "lib"; populate "all" output. >>> (openmpi)[inputs]: Use hwloc-nogui. >> >> The downside of this is that the =E2=80=9Cnogui=E2=80=9D output is less = discoverable >> (and it=E2=80=99s another user-visible breakage.) > > I don't understand why it's worse than currently. "hwloc" will provide > the same as before, won't it? I guess developer breakage could be fixed > by retaining the lib output if it matters. > > Maybe it's helpful to try to document what sort of stability is expected > currently? Concretely, I have a bunch of packages for linear algebra software developed at work. When we add/remove an output to hwloc, those packages may fail to build (for instance, currently they expect the =E2=80=9Clib=E2=80=9D output of hwloc.) Likewise, =E2=80=9Cguix package -u=E2=80=9D doesn=E2=80=99t deal with outpu= t changes (we do have a mechanism to deal with package renames, but not with output changes.) >> Also, it shouldn=E2=80=99t make any difference to the closure size of op= enmpi >> anyway, no? > > Right. It wasn't for openmpi specifically. > >>> + (add-after 'install 'install-openmpi >>> + (lambda* (#:key outputs #:allow-other-keys) >>> + (let ((dest (format #f "~a/lib/valgrind" >>> + (assoc-ref outputs "openmpi")))) >>> + (mkdir-p dest) >>> + (zero? >>> + (system (format #f "mv ~a/lib/valgrind/libmpiwrap* ~a" >>> + (assoc-ref outputs "out") dest))))))))) >> >> Why move it to a separate output? After all, we can keep it in =E2=80= =9Cout=E2=80=9D >> since all it costs is the size of libmpiwrap.so, right? >> >> Also, I assume that this is functionally equivalent to Open MPI=E2=80=99s >> built-in Valgrind support, is it? > > This is probably moot. It isn't entirely equivalent but, more > importantly, the builtin support apparently doesn't have the performance > hit which was documented; I haven't checked experimentally. See this > thread, though not all my questions were answered: > . > > The wrapper library may still be relevant for mpich-y MPIs, if they get > used -- I don't know. OK. So to me that means we can apply the patch below and be done with it. Fine with you? Thanks, Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm index 93157e269..ded9d4fda 100644 --- a/gnu/packages/mpi.scm +++ b/gnu/packages/mpi.scm @@ -36,8 +36,7 @@ #:use-module (gnu packages xml) #:use-module (gnu packages perl) #:use-module (gnu packages ncurses) - #:use-module (gnu packages pkg-config) - #:use-module (gnu packages valgrind)) + #:use-module (gnu packages pkg-config)) (define-public hwloc (package @@ -126,8 +125,7 @@ bind processes, and much more.") `(("hwloc" ,hwloc "lib") ("gfortran" ,gfortran) ("libfabric" ,libfabric) - ("rdma-core" ,rdma-core) - ("valgrind" ,valgrind))) + ("rdma-core" ,rdma-core))) (native-inputs `(("pkg-config" ,pkg-config) ("perl" ,perl))) @@ -142,8 +140,6 @@ bind processes, and much more.") ;; it reduces the closure size considerably. "--disable-vt" - ,(string-append "--with-valgrind=" - (assoc-ref %build-inputs "valgrind")) ,(string-append "--with-hwloc=" (assoc-ref %build-inputs "hwloc"))) #:phases (modify-phases %standard-phases --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 07 12:14:48 2017 Received: (at 27905) by debbugs.gnu.org; 7 Sep 2017 16:14:48 +0000 Received: from localhost ([127.0.0.1]:54526 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dpzS3-0003Hb-73 for submit@debbugs.gnu.org; Thu, 07 Sep 2017 12:14:47 -0400 Received: from clarity.mcc.ac.uk ([130.88.200.144]:25188) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dpzS1-0003HQ-2I for 27905@debbugs.gnu.org; Thu, 07 Sep 2017 12:14:45 -0400 Received: from asmtp1.its.manchester.ac.uk ([130.88.13.149]) by clarity.mcc.ac.uk with esmtps (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1dpzRz-0008th-Sq; Thu, 07 Sep 2017 17:14:43 +0100 Received: from albion.it.manchester.ac.uk ([130.88.138.79]:41246) by asmtp1.its.manchester.ac.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1dpzRz-0003Nr-KD; Thu, 07 Sep 2017 17:14:43 +0100 From: Dave Love To: Ludovic =?iso-8859-1?Q?Court=E8s?= Subject: Re: [bug#27905] changes for openmpi References: <87zibja1of.fsf@i-ulialbion.it.manchester.ac.uk> <87h8x1t0op.fsf@gnu.org> <87d17aznix.fsf@albion.it.manchester.ac.uk> <873782xzxb.fsf@inria.fr> X-Draft-From: ("Misc" 2815) Date: Thu, 07 Sep 2017 17:14:43 +0100 In-Reply-To: <873782xzxb.fsf@inria.fr> ("Ludovic \=\?iso-8859-1\?Q\?Court\=E8s\?\= \=\?iso-8859-1\?Q\?\=22's\?\= message of "Mon, 4 Sep 2017 17:10:24 +0200") Message-ID: <877exaqydo.fsf@albion.it.manchester.ac.uk> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Authenticated-Sender: David Love from albion.it.manchester.ac.uk [130.88.138.79]:41246 X-Authenticated-From: Dave.Love@manchester.ac.uk X-SA-Exim-Connect-IP: 130.88.13.149 X-SA-Exim-Mail-From: fx@gnu.org X-SA-Exim-Scanned: No (on clarity.mcc.ac.uk); SAEximRunCond expanded to false X-Spam-Score: -5.0(?) X-Spam-Flag: NO X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: 27905 Cc: 27905@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.3 (-) Ludovic Court=C3=A8s writes: >>> Also, I assume that this is functionally equivalent to Open MPI=E2=80= =99s >>> built-in Valgrind support, is it? >> >> This is probably moot. It isn't entirely equivalent but, more >> importantly, the builtin support apparently doesn't have the performance >> hit which was documented; I haven't checked experimentally. See this >> thread, though not all my questions were answered: >> . >> >> The wrapper library may still be relevant for mpich-y MPIs, if they get >> used -- I don't know. > > OK. > > So to me that means we can apply the patch below and be done with it. > Fine with you? No, I now think it shouldn't be changed, since the valgrind integration is supposed not to impose a significant speed penalty, and I can remove valgrind from the closure simply. I'll send a new patch later. > Thanks, > Ludo=E2=80=99. > > > diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm > index 93157e269..ded9d4fda 100644 > --- a/gnu/packages/mpi.scm > +++ b/gnu/packages/mpi.scm > @@ -36,8 +36,7 @@ > #:use-module (gnu packages xml) > #:use-module (gnu packages perl) > #:use-module (gnu packages ncurses) > - #:use-module (gnu packages pkg-config) > - #:use-module (gnu packages valgrind)) > + #:use-module (gnu packages pkg-config)) >=20=20 > (define-public hwloc > (package > @@ -126,8 +125,7 @@ bind processes, and much more.") > `(("hwloc" ,hwloc "lib") > ("gfortran" ,gfortran) > ("libfabric" ,libfabric) > - ("rdma-core" ,rdma-core) > - ("valgrind" ,valgrind))) > + ("rdma-core" ,rdma-core))) > (native-inputs > `(("pkg-config" ,pkg-config) > ("perl" ,perl))) > @@ -142,8 +140,6 @@ bind processes, and much more.") > ;; it reduces the closure size considerably. > "--disable-vt" >=20=20 > - ,(string-append "--with-valgrind=3D" > - (assoc-ref %build-inputs "val= grind")) > ,(string-append "--with-hwloc=3D" > (assoc-ref %build-inputs "hwl= oc"))) > #:phases (modify-phases %standard-phases From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 11 16:24:10 2017 Received: (at 27905) by debbugs.gnu.org; 11 Sep 2017 20:24:10 +0000 Received: from localhost ([127.0.0.1]:34131 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1drVFa-000519-Eh for submit@debbugs.gnu.org; Mon, 11 Sep 2017 16:24:10 -0400 Received: from clarity.mcc.ac.uk ([130.88.200.144]:51271) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1drVFY-000511-Io for 27905@debbugs.gnu.org; Mon, 11 Sep 2017 16:24:08 -0400 Received: from asmtp1.its.manchester.ac.uk ([130.88.13.149]) by clarity.mcc.ac.uk with esmtps (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1drVFW-000Fsb-9J; Mon, 11 Sep 2017 21:24:06 +0100 Received: from albion.it.manchester.ac.uk ([130.88.138.79]:58158) by asmtp1.its.manchester.ac.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1drVFW-0001Gs-0s; Mon, 11 Sep 2017 21:24:06 +0100 From: Dave Love To: Ludovic =?iso-8859-1?Q?Court=E8s?= Subject: Re: [bug#27905] changes for openmpi References: <87zibja1of.fsf@i-ulialbion.it.manchester.ac.uk> <87h8x1t0op.fsf@gnu.org> <87d17aznix.fsf@albion.it.manchester.ac.uk> <873782xzxb.fsf@inria.fr> <877exaqydo.fsf@albion.it.manchester.ac.uk> X-Draft-From: ("nnfolder+archive:misc-mail-201709" 10) Date: Mon, 11 Sep 2017 21:24:05 +0100 In-Reply-To: <877exaqydo.fsf@albion.it.manchester.ac.uk> (Dave Love's message of "Thu, 07 Sep 2017 17:14:43 +0100") Message-ID: <87ingpj862.fsf@albion.it.manchester.ac.uk> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="ARC-IRS-Mace-Avian-Talent" X-Authenticated-Sender: David Love from albion.it.manchester.ac.uk [130.88.138.79]:58158 X-Authenticated-From: Dave.Love@manchester.ac.uk X-SA-Exim-Connect-IP: 130.88.13.149 X-SA-Exim-Mail-From: fx@gnu.org X-SA-Exim-Scanned: No (on clarity.mcc.ac.uk); SAEximRunCond expanded to false X-Spam-Score: -5.0(?) X-Spam-Flag: NO X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: 27905 Cc: 27905@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.3 (-) --ARC-IRS-Mace-Avian-Talent Content-Type: text/plain I wrote: >> So to me that means we can apply the patch below and be done with it. >> Fine with you? > > No, I now think it shouldn't be changed, since the valgrind integration > is supposed not to impose a significant speed penalty, and I can remove > valgrind from the closure simply. I'll send a new patch later. Here it is, eventually, which gets rid of a lot from the closure. --ARC-IRS-Mace-Avian-Talent Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-gnu-openmpi-Remove-valgrind-from-closure.patch >From 6b47b2ce671bfbdab3c5f4f2546f02bcfee66d68 Mon Sep 17 00:00:00 2001 From: Dave Love Date: Mon, 4 Sep 2017 18:04:21 +0100 Subject: [PATCH 2/2] gnu openmpi: Remove valgrind from closure. * mpi.scm (openmpi)[arguments]: Elide romio config info to avoid valgrind path. --- gnu/packages/mpi.scm | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm index 76154a055..04f9d7ebe 100644 --- a/gnu/packages/mpi.scm +++ b/gnu/packages/mpi.scm @@ -146,16 +146,21 @@ bind processes, and much more.") (assoc-ref %build-inputs "hwloc"))) #:phases (modify-phases %standard-phases (add-before 'build 'remove-absolute - ;; Remove compiler absolute file names (OPAL_FC_ABSOLUTE - ;; etc.) to reduce the closure size. See - ;; - ;; and - ;; . (lambda _ + ;; Remove compiler absolute file names (OPAL_FC_ABSOLUTE + ;; etc.) to reduce the closure size. See + ;; + ;; and + ;; . (substitute* '("orte/tools/orte-info/param.c" "oshmem/tools/oshmem_info/param.c" "ompi/tools/ompi_info/param.c") (("_ABSOLUTE") "")) + ;; Avoid valgrind (which pulls in gdb etc.). + (substitute* + '("./ompi/mca/io/romio/src/io_romio_component.c") + (("MCA_io_romio_COMPLETE_CONFIGURE_FLAGS") + "\"[elided to reduce closure]\"")) #t)) (add-before 'build 'scrub-timestamps ;reproducibility (lambda _ -- 2.11.0 --ARC-IRS-Mace-Avian-Talent-- From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 12 03:00:32 2017 Received: (at 27905-done) by debbugs.gnu.org; 12 Sep 2017 07:00:32 +0000 Received: from localhost ([127.0.0.1]:34626 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1drfBQ-0005OT-46 for submit@debbugs.gnu.org; Tue, 12 Sep 2017 03:00:32 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:15367) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1drfBN-0005OF-SX for 27905-done@debbugs.gnu.org; Tue, 12 Sep 2017 03:00:30 -0400 X-IronPort-AV: E=Sophos;i="5.42,382,1500933600"; d="scan'208";a="290609804" Received: from unknown (HELO ribbon) ([193.50.110.232]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/AES256-GCM-SHA384; 12 Sep 2017 09:00:22 +0200 From: ludovic.courtes@inria.fr (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Dave Love Subject: Re: [bug#27905] changes for openmpi Organization: Inria References: <87zibja1of.fsf@i-ulialbion.it.manchester.ac.uk> <87h8x1t0op.fsf@gnu.org> <87d17aznix.fsf@albion.it.manchester.ac.uk> <873782xzxb.fsf@inria.fr> <877exaqydo.fsf@albion.it.manchester.ac.uk> <87ingpj862.fsf@albion.it.manchester.ac.uk> X-URL: http://people.bordeaux.inria.fr/lcourtes/ X-Revolutionary-Date: 26 Fructidor an 225 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Tue, 12 Sep 2017 09:00:22 +0200 In-Reply-To: <87ingpj862.fsf@albion.it.manchester.ac.uk> (Dave Love's message of "Mon, 11 Sep 2017 21:24:05 +0100") Message-ID: <87a820e709.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 27905-done Cc: 27905-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Dave Love skribis: > I wrote:=20 > >>> So to me that means we can apply the patch below and be done with it. >>> Fine with you? >> >> No, I now think it shouldn't be changed, since the valgrind integration >> is supposed not to impose a significant speed penalty, and I can remove >> valgrind from the closure simply. I'll send a new patch later. > > Here it is, eventually, which gets rid of a lot from the closure. > > From 6b47b2ce671bfbdab3c5f4f2546f02bcfee66d68 Mon Sep 17 00:00:00 2001 > From: Dave Love > Date: Mon, 4 Sep 2017 18:04:21 +0100 > Subject: [PATCH 2/2] gnu openmpi: Remove valgrind from closure. > > * mpi.scm (openmpi)[arguments]: Elide romio config info to avoid valgrind > path. Awesome! I tweaked the commit log and pushed. Now we=E2=80=99re down to 156=C2=A0MiB for the whole closure, which is much= better. There=E2=80=99s still room for optimization (Bash, xz, util-linux?), but we= =E2=80=99ll get there: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix size openmpi store item total s= elf /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25 38.5 = 37.1 23.7% /gnu/store/8j1h29zcgrg13dc2md7lalxliv1jrq2p-gfortran-5.4.0-lib 73.0 = 34.5 22.0% /gnu/store/3x53yv4v144c9xp02rs64z7j597kkqax-gcc-5.4.0-lib 68.6 = 30.1 19.2% /gnu/store/z77nhww8zh96w6lb5ak6h3jb4niain3b-eudev-3.2.2 103.2 = 14.1 9.0% /gnu/store/dy81cx0yshq8vban59vjsdl4rvxnwxab-util-linux-2.30 87.6 = 12.0 7.7% /gnu/store/jk8bcr9q79cj6j97xb6rdil1fw0g8hd6-openmpi-1.10.7 156.5 = 10.1 6.5% /gnu/store/09j7scnl3hahcmql986fsjpzj6gqsmzv-ncurses-6.0 74.3 = 5.7 3.6% /gnu/store/bhawz0mpfdjhwq423q6kk2jz34dpcsx5-libnl-3.3.0 72.3 = 3.6 2.3% /gnu/store/n2k1kmwj0rswq6qija8v8kz9ramj2a83-rdma-core-14 108.8 = 2.0 1.3% /gnu/store/808hmh1bp6khhbfrbljcsnly9497bxvy-libfabric-1.4.1 110.4 = 1.6 1.0% /gnu/store/zhrajv6qf2hzn9c3g2bb07559hyrz5xp-bash-static-4.4.12 1.4 = 1.4 0.9% /gnu/store/g3nari57wcfnm00kv9bnpyzdzfq4h8pk-xz-5.2.2 70.7 = 1.1 0.7% /gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12 39.5 = 1.0 0.6% /gnu/store/hf6k2i6aqqs50p181bs1aa7xw49kd6xn-hwloc-1.11.8-lib 72.8 = 0.6 0.4% /gnu/store/ljzqi3ajkc6l5r8hwdz7kr1zwbli3i7y-pciutils-3.5.5 71.8 = 0.5 0.3% /gnu/store/sfx1wh27i6gsrk21p87rdyikc64v7d51-zlib-1.2.11 69.0 = 0.4 0.2% /gnu/store/bdys6wm9hwd7akd5mc00xw0y4cz0j1fg-numactl-2.0.11 68.9 = 0.3 0.2% /gnu/store/insr5wrif9pn1mlqa5rl9k3sr5qf2q1y-kmod-24 71.3 = 0.3 0.2% /gnu/store/0p4gxh2xiz31v2zx8mg43nv2djjyfwmn-libpciaccess-0.13.5 71.9 = 0.1 0.1% total: 156.5 MiB --8<---------------cut here---------------end--------------->8--- Thanks! Ludo=E2=80=99. From unknown Tue Jun 24 15:39:36 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 10 Oct 2017 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator