GNU bug report logs - #54635
[PATCH 0/5] Add wfmash

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 54635 in the body.
You can then email your comments to 54635 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: guix-patches <at> gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>,
 Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH 0/5] Add wfmash
Date: Wed, 30 Mar 2022 14:49:13 +0530

This patchset adds wfmash, and ensures that, in addition to x86_64, it also
cross-compiles to riscv64.

Arun Isaac (5):
  gnu: gsl: Force bootstrap when cross-compiling to riscv64-linux.
  gnu: htslib: Add bzip2 and xz to inputs.
  gnu: atomic-queue: Run tests correctly.
  gnu: atomic-queue: Do not depend on boost when cross-compiling.
  gnu: Add wfmash.

 gnu/packages/bioinformatics.scm | 55 +++++++++++++++++++++++++++++++--
 gnu/packages/cpp.scm            | 26 +++++++++++++---
 gnu/packages/maths.scm          | 15 ++++++++-
 3 files changed, 88 insertions(+), 8 deletions(-)

-- 
2.34.0

Message #8 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 54635 <at> debbugs.gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>,
 Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH 2/5] gnu: htslib: Add bzip2 and xz to inputs.
Date: Wed, 30 Mar 2022 14:53:10 +0530

htslib links to libbz2 from bzip2 and liblzma from xz. Therefore, bzip2 and xz
should be listed in inputs even though they are already present implicitly in
native-inputs. Else, cross-compilation will fail.

* gnu/packages/bioinformatics.scm (htslib)[inputs]: Add bzip2 and xz.
---
 gnu/packages/bioinformatics.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 4e1028b3ce..2f6f3efb66 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -8,7 +8,7 @@
 ;;; Copyright © 2016, 2020, 2022 Marius Bakke <marius <at> gnu.org>
 ;;; Copyright © 2016, 2018 Raoul Bonnal <ilpuccio.febo <at> gmail.com>
 ;;; Copyright © 2017, 2018 Tobias Geerinckx-Rice <me <at> tobias.gr>
-;;; Copyright © 2017, 2021 Arun Isaac <arunisaac <at> systemreboot.net>
+;;; Copyright © 2017, 2021, 2022 Arun Isaac <arunisaac <at> systemreboot.net>
 ;;; Copyright © 2018 Joshua Sierles, Nextjournal <joshua <at> nextjournal.com>
 ;;; Copyright © 2018 Gábor Boskovits <boskovits <at> gmail.com>
 ;;; Copyright © 2018, 2019, 2020, 2021 Mădălin Ionel Patrașcu <madalinionel.patrascu <at> mdc-berlin.de>
@@ -4788,7 +4788,7 @@ (define-public htslib
                            "--enable-libcurl"
                            "--enable-s3")))
     (inputs
-     (list curl openssl))
+     (list bzip2 curl openssl xz))
     ;; This is referred to in the pkg-config file as a required library.
     (propagated-inputs
      (list zlib))
-- 
2.34.0

Message #11 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 54635 <at> debbugs.gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>,
 Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH 1/5] gnu: gsl: Force bootstrap when cross-compiling to
 riscv64-linux.
Date: Wed, 30 Mar 2022 14:53:09 +0530

* gnu/packages/maths.scm (gsl)[arguments]: Force autotools bootstrap when
cross-compiling to riscv64-linux.
[native-inputs]: Add autoconf, automake and libtool when cross-compiling to
riscv64-linux.
---
 gnu/packages/maths.scm | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index ecb85642ec..f01bf51580 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -19,7 +19,7 @@
 ;;; Copyright © 2017 Nikita <nikita <at> n0.is>
 ;;; Copyright © 2017 Ben Woodcroft <donttrustben <at> gmail.com>
 ;;; Copyright © 2017 Theodoros Foradis <theodoros <at> foradis.org>
-;;; Copyright © 2017, 2019 Arun Isaac <arunisaac <at> systemreboot.net>
+;;; Copyright © 2017, 2019, 2022 Arun Isaac <arunisaac <at> systemreboot.net>
 ;;; Copyright © 2017–2021 Tobias Geerinckx-Rice <me <at> tobias.gr>
 ;;; Copyright © 2017 Dave Love <me <at> fx <at> gnu.org>
 ;;; Copyright © 2018, 2019, 2020, 2021 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
@@ -533,6 +533,13 @@ (define-public gsl
          #:phases
          (modify-phases %standard-phases
            ,@(cond
+              ((target-riscv64?)
+               '((add-after 'unpack 'force-bootstrap
+                   (lambda _
+                     ;; gsl ships with an old configure script that does not
+                     ;; support riscv64. Regenerate it.
+                     (delete-file "configure")))))
+
               ((or (string-prefix? "aarch64" system)
                    (string-prefix? "powerpc" system))
                ;; Some sparse matrix tests are failing on AArch64 and PowerPC:
@@ -568,6 +575,12 @@ (define-public gsl
                         (string-append "exit (77);\n" all)))))))
 
               (else '()))))))
+    (native-inputs
+     (if (target-riscv64?)
+         `(("autoconf" ,autoconf)
+           ("automake" ,automake)
+           ("libtool" ,libtool))
+         '()))
     (home-page "https://www.gnu.org/software/gsl/")
     (synopsis "Numerical library for C and C++")
     (description
-- 
2.34.0

Message #14 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 54635 <at> debbugs.gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>,
 Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH 4/5] gnu: atomic-queue: Do not depend on boost when
 cross-compiling.
Date: Wed, 30 Mar 2022 14:53:12 +0530

* gnu/packages/cpp.scm (atomic-queue)[arguments]: When cross-compiling, add
do-not-check-for-boost phase and delete the build phase.
[native-inputs]: Do not include boost when cross-compiling.
---
 gnu/packages/cpp.scm | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index eb48902034..649a57b0d5 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -1294,6 +1294,21 @@ (define-public atomic-queue
      `(#:configure-flags '("-Dbenchmarks=false")
        #:phases
        (modify-phases %standard-phases
+         ,@(if (%current-target-system)
+               `(;; boost is a test dependency. We don't run tests when
+                 ;; cross-compiling. So, do not check for it.
+                 (add-after 'unpack 'do-not-check-for-boost
+                   (lambda _
+                     (substitute* "meson.build"
+                       (("unit_test_framework =" all)
+                        (string-append "# " all))
+                       ((", unit_test_framework") ""))))
+                 ;; atomic-queue is a header-only library. Excepting the
+                 ;; tests, no building is required. And since we don't run
+                 ;; tests when cross-compiling, delete the build phase
+                 ;; entirely.
+                 (delete 'build))
+               '())
          (replace 'check
            (lambda* (#:key tests? #:allow-other-keys)
              (when tests?
@@ -1303,9 +1318,11 @@ (define-public atomic-queue
              (copy-recursively "../source/include/atomic_queue"
                                (string-append (assoc-ref outputs "out")
                                               "/include/atomic_queue")))))))
-     (native-inputs
-      (list boost
-            pkg-config))
+    (native-inputs
+     (cons pkg-config
+           (if (%current-target-system)
+               '()
+               (list boost))))
     (home-page "https://github.com/max0x7ba/atomic_queue")
     (synopsis "C++ lockless queue")
     (description
-- 
2.34.0

Message #17 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 54635 <at> debbugs.gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>,
 Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH 3/5] gnu: atomic-queue: Run tests correctly.
Date: Wed, 30 Mar 2022 14:53:11 +0530

* gnu/packages/cpp.scm (atomic-queue)[arguments]: In the check phase, run
tests instead of returning a function to run them.
---
 gnu/packages/cpp.scm | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index bb7f5ab95a..eb48902034 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -1297,8 +1297,7 @@ (define-public atomic-queue
          (replace 'check
            (lambda* (#:key tests? #:allow-other-keys)
              (when tests?
-               (lambda _
-                 (invoke "make" "run_tests")))))
+               (invoke "./tests"))))
          (replace 'install
            (lambda* (#:key outputs #:allow-other-keys)
              (copy-recursively "../source/include/atomic_queue"
-- 
2.34.0

Message #20 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 54635 <at> debbugs.gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>,
 Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH 5/5] gnu: Add wfmash.
Date: Wed, 30 Mar 2022 14:53:13 +0530

* gnu/packages/bioinformatics.scm (wfmash): New variable.
---
 gnu/packages/bioinformatics.scm | 51 +++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 2f6f3efb66..500ba4ac60 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -16107,3 +16107,54 @@ (define-public ccwl
 @acronym{EDSL, Embedded Domain Specific Language} in the Scheme programming
 language.")
     (license license:gpl3+)))
+
+(define-public wfmash
+  (package
+    (name "wfmash")
+    (version "0.8.1")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://github.com/ekg/wfmash/releases/download/v"
+                           version "/wfmash-v" version ".tar.gz"))
+       (sha256
+        (base32
+         "031cm1arpfckvihb28vlk69mirpnmlag81zcscfba1bac58wvr7c"))
+       (modules '((guix build utils)))
+       (snippet
+        '(begin
+           ;; Unbundle atomic-queue.
+           (delete-file-recursively "src/common/atomic_queue")
+           (substitute* "src/align/include/computeAlignments.hpp"
+             (("\"common/atomic_queue/atomic_queue.h\"")
+              "<atomic_queue/atomic_queue.h>"))))))
+    (build-system cmake-build-system)
+    (arguments
+     `(#:tests? #f                      ; no tests
+       #:phases
+       (modify-phases %standard-phases
+         ;; Remove compiler flags and checks specific to x86 when not
+         ;; targeting it.
+         ,@(if (target-x86-64?)
+               '()
+               '((add-after 'unpack 'remove-x86-specific-compile-flags
+                   (lambda _
+                     (substitute* (find-files "." "CMakeLists\\.txt")
+                       (("-mcx16") "")
+                       (("-march=native") ""))
+                     (substitute* "src/common/dset64.hpp"
+                       (("!__x86_64__") "0")))))))))
+    (inputs
+     (list atomic-queue
+           gsl
+           htslib
+           jemalloc
+           zlib))
+    (synopsis "Base-accurate DNA sequence aligner")
+    (description "@code{wfmash} is a DNA sequence read mapper based on mash
+distances and the wavefront alignment algorithm.  It is a fork of MashMap that
+implements base-level alignment via the wflign tiled wavefront global
+alignment algorithm.  It completes MashMap with a high-performance alignment
+module capable of computing base-level alignments for very large sequences.")
+    (home-page "https://github.com/ekg/wfmash")
+    (license license:expat)))
-- 
2.34.0

Message #23 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Arun Isaac <arunisaac <at> systemreboot.net>, 54635 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: Re: [bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when
 cross-compiling to riscv64-linux.
Date: Wed, 30 Mar 2022 13:33:12 +0200

[Message part 1 (text/plain, inline)]

Arun Isaac schreef op wo 30-03-2022 om 14:53 [+0530]:
> +    (native-inputs
> +     (if (target-riscv64?)
> +         `(("autoconf" ,autoconf)
> +           ("automake" ,automake)
> +           ("libtool" ,libtool))
> +         '()))

Nowadays input labels are not required anymore here, you can do

 (native-inputs (if (targer-riscv64?) (list autoconf automake libtool)
'()))

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #26 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Arun Isaac <arunisaac <at> systemreboot.net>, 54635 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: Re: [bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when
 cross-compiling to riscv64-linux.
Date: Wed, 30 Mar 2022 13:36:34 +0200

[Message part 1 (text/plain, inline)]

Arun Isaac schreef op wo 30-03-2022 om 14:53 [+0530]:
>             ,@(cond
> +              ((target-riscv64?)
> +               '((add-after 'unpack 'force-bootstrap
> +                   (lambda _
> +                     ;; gsl ships with an old configure script that does not
> +                     ;; support riscv64. Regenerate it.
> +                     (delete-file "configure")))))
> +

WDYT of making this unconditional?  Two benefits:

  * if Guix is ported to another new architecture,
    then no changes are necessary to the package definition.

  * 'configure' and 'Makefile.in' are not source code,
    and more difficult to audit for things like malware than
    'configure.ac' and 'Makefile.am'.

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #29 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 54635 <at> debbugs.gnu.org, Arun Isaac <arunisaac <at> systemreboot.net>
Subject: Re: [bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when
 cross-compiling to riscv64-linux.
Date: Wed, 30 Mar 2022 14:39:54 +0300

[Message part 1 (text/plain, inline)]

On Wed, Mar 30, 2022 at 01:36:34PM +0200, Maxime Devos wrote:
> Arun Isaac schreef op wo 30-03-2022 om 14:53 [+0530]:
> >             ,@(cond
> > +              ((target-riscv64?)
> > +               '((add-after 'unpack 'force-bootstrap
> > +                   (lambda _
> > +                     ;; gsl ships with an old configure script that does not
> > +                     ;; support riscv64. Regenerate it.
> > +                     (delete-file "configure")))))
> > +
> 
> WDYT of making this unconditional?  Two benefits:
> 
>   * if Guix is ported to another new architecture,
>     then no changes are necessary to the package definition.
> 
>   * 'configure' and 'Makefile.in' are not source code,
>     and more difficult to audit for things like malware than
>     'configure.ac' and 'Makefile.am'.

This can be with a TODO for core-updates. gsl itself has about 2000
dependant packages.

That said, I'm not convinced about unilaterally removing configure
unless we make it a policy to remove it. Also, I haven't had trouble
with building gsl on riscv64-linux without this patch.


-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #32 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Arun Isaac <arunisaac <at> systemreboot.net>, 54635 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: Re: [bug#54635] [PATCH 4/5] gnu: atomic-queue: Do not depend on
 boost when cross-compiling.
Date: Wed, 30 Mar 2022 13:49:35 +0200

[Message part 1 (text/plain, inline)]

Arun Isaac schreef op wo 30-03-2022 om 14:53 [+0530]:
> * gnu/packages/cpp.scm (atomic-queue)[arguments]: When cross-compiling, add
> do-not-check-for-boost phase and delete the build phase.
> [native-inputs]: Do not include boost when cross-compiling.

I think it's simpler and less tedious to just always include test
dependencies in native-inputs, without special-casing cross-
compilation.  WYDT?

Also, being header-only does not per-se mean ‘no building’.  E.g., what
if some of the headers are generated?  I think it's a bit simpler and
less likely to cause trouble to just always run the build phase, even
if sometimes it is a no-op.

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #35 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Maxime Devos <maximedevos <at> telenet.be>, 54635 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: Re: [bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when
 cross-compiling to riscv64-linux.
Date: Thu, 31 Mar 2022 11:44:03 +0530

> Nowadays input labels are not required anymore here, you can do
>
>  (native-inputs (if (targer-riscv64?) (list autoconf automake libtool)
> '()))

Yes, indeed! Will remove the input labels in the next version of the
patchset.

Message #38 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Efraim Flashner <efraim <at> flashner.co.il>, Maxime Devos
 <maximedevos <at> telenet.be>
Cc: 54635 <at> debbugs.gnu.org
Subject: Re: [bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when
 cross-compiling to riscv64-linux.
Date: Thu, 31 Mar 2022 12:03:39 +0530

>> >             ,@(cond
>> > +              ((target-riscv64?)
>> > +               '((add-after 'unpack 'force-bootstrap
>> > +                   (lambda _
>> > +                     ;; gsl ships with an old configure script that does not
>> > +                     ;; support riscv64. Regenerate it.
>> > +                     (delete-file "configure")))))
>> > +
>> 
>> WDYT of making this unconditional?  Two benefits:
>> 
>>   * if Guix is ported to another new architecture,
>>     then no changes are necessary to the package definition.
>> 
>>   * 'configure' and 'Makefile.in' are not source code,
>>     and more difficult to audit for things like malware than
>>     'configure.ac' and 'Makefile.am'.
>
> This can be with a TODO for core-updates. gsl itself has about 2000
> dependant packages.

I agree. That was my reasoning as well. If we agree that making it
unconditional is the way forward, I can send another patch for
core-updates after this patchset is pushed to master.

> That said, I'm not convinced about unilaterally removing configure
> unless we make it a policy to remove it. Also, I haven't had trouble
> with building gsl on riscv64-linux without this patch.

Without the force-boostrap phase, the configure phase fails during
cross-compilation.

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build --target=riscv64-linux-gnu gsl
[...]
starting phase `configure'
source directory: "/tmp/guix-build-gsl-2.7.drv-0/gsl-2.7" (relative from build: ".")
build directory: "/tmp/guix-build-gsl-2.7.drv-0/gsl-2.7"
configure flags: ("CC_FOR_BUILD=gcc" "CONFIG_SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "--prefix=/gnu/store/x7ag3i38ykn2l3f6sfn06bn9356kdk0x-gsl-2.7" "--enable-fast-install" "--build=x86_64-unknown-linux-gnu" "--host=riscv64-linux-gnu" "--disable-static")
checking for a BSD-compatible install... /gnu/store/d251rfgc9nm2clzffzhgiipdvfvzkvwi-coreutils-8.32/bin/install -c
checking whether build environment is sane... yes
checking for riscv64-linux-gnu-strip... riscv64-linux-gnu-strip
checking for a race-free mkdir -p... /gnu/store/d251rfgc9nm2clzffzhgiipdvfvzkvwi-coreutils-8.32/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... no
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for a sed that does not truncate output... /gnu/store/wxgv6i8g0p24q5gcyzd0yr07s8kn9680-sed-4.8/bin/sed
checking whether make sets $(MAKE)... (cached) no
checking build system type... x86_64-unknown-linux-gnu
checking host system type... Invalid configuration `riscv64-linux-gnu': machine `riscv64' not recognized
configure: error: /gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash ./config.sub riscv64-linux-gnu failed
error: in phase 'configure': uncaught exception:
%exception #<&invoke-error program: "/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" arguments: ("./configure" "CC_FOR_BUILD=gcc" "CONFIG_SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "--prefix=/gnu/store/x7ag3i38ykn2l3f6sfn06bn9356kdk0x-gsl-2.7" "--enable-fast-install" "--build=x86_64-unknown-linux-gnu" "--host=riscv64-linux-gnu" "--disable-static") exit-status: 1 term-signal: #f stop-signal: #f> 
phase `configure' failed after 0.3 seconds
command "/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "./configure" "CC_FOR_BUILD=gcc" "CONFIG_SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "--prefix=/gnu/store/x7ag3i38ykn2l3f6sfn06bn9356kdk0x-gsl-2.7" "--enable-fast-install" "--build=x86_64-unknown-linux-gnu" "--host=riscv64-linux-gnu" "--disable-static" failed with status 1
builder for `/gnu/store/fwjvvklmswc9midrcdg6qir2knvmraif-gsl-2.7.drv' failed with exit code 1
build of /gnu/store/fwjvvklmswc9midrcdg6qir2knvmraif-gsl-2.7.drv failed
View build log at '/var/log/guix/drvs/fw/jvvklmswc9midrcdg6qir2knvmraif-gsl-2.7.drv.gz'.
guix build: error: build of `/gnu/store/fwjvvklmswc9midrcdg6qir2knvmraif-gsl-2.7.drv' failed
--8<---------------cut here---------------end--------------->8---

Message #41 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Maxime Devos <maximedevos <at> telenet.be>, 54635 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: Re: [bug#54635] [PATCH 4/5] gnu: atomic-queue: Do not depend on
 boost when cross-compiling.
Date: Thu, 31 Mar 2022 12:54:41 +0530

> I think it's simpler and less tedious to just always include test
> dependencies in native-inputs, without special-casing cross-
> compilation.  WYDT?

Sure, agreed!

> Also, being header-only does not per-se mean ‘no building’.  E.g., what
> if some of the headers are generated?  I think it's a bit simpler and
> less likely to cause trouble to just always run the build phase, even
> if sometimes it is a no-op.

Agreed too!

Will address both in the next version of my patchset.

Message #44 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Efraim Flashner <efraim <at> flashner.co.il>,
 Maxime Devos <maximedevos <at> telenet.be>
Cc: 54635 <at> debbugs.gnu.org, Arun Isaac <arunisaac <at> systemreboot.net>
Subject: [PATCH v2 1/5] gnu: gsl: Force bootstrap when cross-compiling to
 riscv64-linux.
Date: Thu, 31 Mar 2022 12:58:45 +0530

* gnu/packages/maths.scm (gsl)[arguments]: Force autotools bootstrap when
cross-compiling to riscv64-linux.
[native-inputs]: Add autoconf, automake and libtool when cross-compiling to
riscv64-linux.
---
 gnu/packages/maths.scm | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index ecb85642ec..a05f89cb4e 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -19,7 +19,7 @@
 ;;; Copyright © 2017 Nikita <nikita <at> n0.is>
 ;;; Copyright © 2017 Ben Woodcroft <donttrustben <at> gmail.com>
 ;;; Copyright © 2017 Theodoros Foradis <theodoros <at> foradis.org>
-;;; Copyright © 2017, 2019 Arun Isaac <arunisaac <at> systemreboot.net>
+;;; Copyright © 2017, 2019, 2022 Arun Isaac <arunisaac <at> systemreboot.net>
 ;;; Copyright © 2017–2021 Tobias Geerinckx-Rice <me <at> tobias.gr>
 ;;; Copyright © 2017 Dave Love <me <at> fx <at> gnu.org>
 ;;; Copyright © 2018, 2019, 2020, 2021 Jan (janneke) Nieuwenhuizen <janneke <at> gnu.org>
@@ -533,6 +533,13 @@ (define-public gsl
          #:phases
          (modify-phases %standard-phases
            ,@(cond
+              ((target-riscv64?)
+               '((add-after 'unpack 'force-bootstrap
+                   (lambda _
+                     ;; gsl ships with an old configure script that does not
+                     ;; support riscv64. Regenerate it.
+                     (delete-file "configure")))))
+
               ((or (string-prefix? "aarch64" system)
                    (string-prefix? "powerpc" system))
                ;; Some sparse matrix tests are failing on AArch64 and PowerPC:
@@ -568,6 +575,10 @@ (define-public gsl
                         (string-append "exit (77);\n" all)))))))
 
               (else '()))))))
+    (native-inputs
+     (if (target-riscv64?)
+         (list autoconf automake libtool)
+         '()))
     (home-page "https://www.gnu.org/software/gsl/")
     (synopsis "Numerical library for C and C++")
     (description
-- 
2.34.0

Message #47 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Efraim Flashner <efraim <at> flashner.co.il>,
 Maxime Devos <maximedevos <at> telenet.be>
Cc: 54635 <at> debbugs.gnu.org, Arun Isaac <arunisaac <at> systemreboot.net>
Subject: [PATCH v2 3/5] gnu: atomic-queue: Run tests correctly.
Date: Thu, 31 Mar 2022 12:58:47 +0530

* gnu/packages/cpp.scm (atomic-queue)[arguments]: In the check phase, run
tests instead of returning a function to run them.
---
 gnu/packages/cpp.scm | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index bb7f5ab95a..eb48902034 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -1297,8 +1297,7 @@ (define-public atomic-queue
          (replace 'check
            (lambda* (#:key tests? #:allow-other-keys)
              (when tests?
-               (lambda _
-                 (invoke "make" "run_tests")))))
+               (invoke "./tests"))))
          (replace 'install
            (lambda* (#:key outputs #:allow-other-keys)
              (copy-recursively "../source/include/atomic_queue"
-- 
2.34.0

Message #50 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Efraim Flashner <efraim <at> flashner.co.il>,
 Maxime Devos <maximedevos <at> telenet.be>
Cc: 54635 <at> debbugs.gnu.org, Arun Isaac <arunisaac <at> systemreboot.net>
Subject: [PATCH v2 5/5] gnu: Add wfmash.
Date: Thu, 31 Mar 2022 12:58:49 +0530

* gnu/packages/bioinformatics.scm (wfmash): New variable.
---
 gnu/packages/bioinformatics.scm | 51 +++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 2f6f3efb66..500ba4ac60 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -16107,3 +16107,54 @@ (define-public ccwl
 @acronym{EDSL, Embedded Domain Specific Language} in the Scheme programming
 language.")
     (license license:gpl3+)))
+
+(define-public wfmash
+  (package
+    (name "wfmash")
+    (version "0.8.1")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://github.com/ekg/wfmash/releases/download/v"
+                           version "/wfmash-v" version ".tar.gz"))
+       (sha256
+        (base32
+         "031cm1arpfckvihb28vlk69mirpnmlag81zcscfba1bac58wvr7c"))
+       (modules '((guix build utils)))
+       (snippet
+        '(begin
+           ;; Unbundle atomic-queue.
+           (delete-file-recursively "src/common/atomic_queue")
+           (substitute* "src/align/include/computeAlignments.hpp"
+             (("\"common/atomic_queue/atomic_queue.h\"")
+              "<atomic_queue/atomic_queue.h>"))))))
+    (build-system cmake-build-system)
+    (arguments
+     `(#:tests? #f                      ; no tests
+       #:phases
+       (modify-phases %standard-phases
+         ;; Remove compiler flags and checks specific to x86 when not
+         ;; targeting it.
+         ,@(if (target-x86-64?)
+               '()
+               '((add-after 'unpack 'remove-x86-specific-compile-flags
+                   (lambda _
+                     (substitute* (find-files "." "CMakeLists\\.txt")
+                       (("-mcx16") "")
+                       (("-march=native") ""))
+                     (substitute* "src/common/dset64.hpp"
+                       (("!__x86_64__") "0")))))))))
+    (inputs
+     (list atomic-queue
+           gsl
+           htslib
+           jemalloc
+           zlib))
+    (synopsis "Base-accurate DNA sequence aligner")
+    (description "@code{wfmash} is a DNA sequence read mapper based on mash
+distances and the wavefront alignment algorithm.  It is a fork of MashMap that
+implements base-level alignment via the wflign tiled wavefront global
+alignment algorithm.  It completes MashMap with a high-performance alignment
+module capable of computing base-level alignments for very large sequences.")
+    (home-page "https://github.com/ekg/wfmash")
+    (license license:expat)))
-- 
2.34.0

Message #53 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Efraim Flashner <efraim <at> flashner.co.il>,
 Maxime Devos <maximedevos <at> telenet.be>
Cc: 54635 <at> debbugs.gnu.org, Arun Isaac <arunisaac <at> systemreboot.net>
Subject: [PATCH v2 2/5] gnu: htslib: Add bzip2 and xz to inputs.
Date: Thu, 31 Mar 2022 12:58:46 +0530

htslib links to libbz2 from bzip2 and liblzma from xz. Therefore, bzip2 and xz
should be listed in inputs even though they are already present implicitly in
native-inputs. Else, cross-compilation will fail.

* gnu/packages/bioinformatics.scm (htslib)[inputs]: Add bzip2 and xz.
---
 gnu/packages/bioinformatics.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 4e1028b3ce..2f6f3efb66 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -8,7 +8,7 @@
 ;;; Copyright © 2016, 2020, 2022 Marius Bakke <marius <at> gnu.org>
 ;;; Copyright © 2016, 2018 Raoul Bonnal <ilpuccio.febo <at> gmail.com>
 ;;; Copyright © 2017, 2018 Tobias Geerinckx-Rice <me <at> tobias.gr>
-;;; Copyright © 2017, 2021 Arun Isaac <arunisaac <at> systemreboot.net>
+;;; Copyright © 2017, 2021, 2022 Arun Isaac <arunisaac <at> systemreboot.net>
 ;;; Copyright © 2018 Joshua Sierles, Nextjournal <joshua <at> nextjournal.com>
 ;;; Copyright © 2018 Gábor Boskovits <boskovits <at> gmail.com>
 ;;; Copyright © 2018, 2019, 2020, 2021 Mădălin Ionel Patrașcu <madalinionel.patrascu <at> mdc-berlin.de>
@@ -4788,7 +4788,7 @@ (define-public htslib
                            "--enable-libcurl"
                            "--enable-s3")))
     (inputs
-     (list curl openssl))
+     (list bzip2 curl openssl xz))
     ;; This is referred to in the pkg-config file as a required library.
     (propagated-inputs
      (list zlib))
-- 
2.34.0

Message #56 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Efraim Flashner <efraim <at> flashner.co.il>,
 Maxime Devos <maximedevos <at> telenet.be>
Cc: 54635 <at> debbugs.gnu.org, Arun Isaac <arunisaac <at> systemreboot.net>
Subject: [PATCH v2 4/5] gnu: atomic-queue: Do not look for boost when
 cross-compiling.
Date: Thu, 31 Mar 2022 12:58:48 +0530

* gnu/packages/cpp.scm (atomic-queue)[arguments]: When cross-compiling, add a
do-not-check-for-boost phase.
---
 gnu/packages/cpp.scm | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index eb48902034..b0d4fdeb3e 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -1294,6 +1294,15 @@ (define-public atomic-queue
      `(#:configure-flags '("-Dbenchmarks=false")
        #:phases
        (modify-phases %standard-phases
+         ,@(if (%current-target-system)
+               `(;; boost is a test dependency. We don't run tests when
+                 ;; cross-compiling. Disable all targets that depend on it.
+                 (add-after 'unpack 'do-not-check-for-boost
+                   (lambda _
+                     (substitute* "meson.build"
+                       (("unit_test_framework = [^\n]*" all)
+                        "unit_test_framework = disabler()")))))
+               '())
          (replace 'check
            (lambda* (#:key tests? #:allow-other-keys)
              (when tests?
-- 
2.34.0

Message #59 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Arun Isaac <arunisaac <at> systemreboot.net>, Efraim Flashner
 <efraim <at> flashner.co.il>
Cc: 54635 <at> debbugs.gnu.org
Subject: Re: [bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when
 cross-compiling to riscv64-linux.
Date: Thu, 31 Mar 2022 13:29:48 +0200

[Message part 1 (text/plain, inline)]

Arun Isaac schreef op do 31-03-2022 om 12:03 [+0530]:
> > > WDYT of making this unconditional?  Two benefits:
> > > 
> > >    * if Guix is ported to another new architecture,
> > >      then no changes are necessary to the package definition.
> > > 
> > >    * 'configure' and 'Makefile.in' are not source code,
> > >      and more difficult to audit for things like malware than
> > >      'configure.ac' and 'Makefile.am'.
> > 
> > This can be with a TODO for core-updates. gsl itself has about 2000
> > dependant packages.
> 
> I agree. That was my reasoning as well. If we agree that making it
> unconditional is the way forward, I can send another patch for
> core-updates after this patchset is pushed to master.

I started an e-mail thread about this at
<https://lists.gnu.org/archive/html/guix-devel/2022-03/msg00226.html>.

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #62 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Arun Isaac <arunisaac <at> systemreboot.net>, Efraim Flashner
 <efraim <at> flashner.co.il>
Cc: 54635 <at> debbugs.gnu.org
Subject: Re: [PATCH v2 5/5] gnu: Add wfmash.
Date: Thu, 31 Mar 2022 13:34:29 +0200

[Message part 1 (text/plain, inline)]

Arun Isaac schreef op do 31-03-2022 om 12:58 [+0530]:
> +                       (("-march=native") ""))

This is also wrong for x86 systems because it makes the build non-
reproducible.  Also, has upstream been informed about some of the
compiler flags being architecture-specific?

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #65 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 54635 <at> debbugs.gnu.org, Arun Isaac <arunisaac <at> systemreboot.net>
Subject: Re: [PATCH v2 5/5] gnu: Add wfmash.
Date: Thu, 31 Mar 2022 15:18:21 +0300

[Message part 1 (text/plain, inline)]

On Thu, Mar 31, 2022 at 01:34:29PM +0200, Maxime Devos wrote:
> Arun Isaac schreef op do 31-03-2022 om 12:58 [+0530]:
> > +                       (("-march=native") ""))
> 
> This is also wrong for x86 systems because it makes the build non-
> reproducible.  Also, has upstream been informed about some of the
> compiler flags being architecture-specific?

I'm pretty sure upstream is aware of it, and the -mcx16 flag. That whole
phase doesn't need to be non-x86_64 only, upstream prefers it that way
to get fater results but IMO it would be fine to move it into a snippet.


-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #68 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: 54635 <at> debbugs.gnu.org, Maxime Devos <maximedevos <at> telenet.be>
Subject: Re: [bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when
 cross-compiling to riscv64-linux.
Date: Thu, 31 Mar 2022 15:35:20 +0300

[Message part 1 (text/plain, inline)]

On Thu, Mar 31, 2022 at 12:03:39PM +0530, Arun Isaac wrote:
> 
> >> >             ,@(cond
> >> > +              ((target-riscv64?)
> >> > +               '((add-after 'unpack 'force-bootstrap
> >> > +                   (lambda _
> >> > +                     ;; gsl ships with an old configure script that does not
> >> > +                     ;; support riscv64. Regenerate it.
> >> > +                     (delete-file "configure")))))
> >> > +
> >> 
> >> WDYT of making this unconditional?  Two benefits:
> >> 
> >>   * if Guix is ported to another new architecture,
> >>     then no changes are necessary to the package definition.
> >> 
> >>   * 'configure' and 'Makefile.in' are not source code,
> >>     and more difficult to audit for things like malware than
> >>     'configure.ac' and 'Makefile.am'.
> >
> > This can be with a TODO for core-updates. gsl itself has about 2000
> > dependant packages.
> 
> I agree. That was my reasoning as well. If we agree that making it
> unconditional is the way forward, I can send another patch for
> core-updates after this patchset is pushed to master.
> 
> > That said, I'm not convinced about unilaterally removing configure
> > unless we make it a policy to remove it. Also, I haven't had trouble
> > with building gsl on riscv64-linux without this patch.
> 
> Without the force-boostrap phase, the configure phase fails during
> cross-compilation.
> 
> --8<---------------cut here---------------start------------->8---
..snip..
> --8<---------------cut here---------------end--------------->8---

I somehow missed that when I was testing it before. It builds fine
natively on riscv64-linux and I haven't tested cross-building from
riscv64-linux to another architecture. I think for now we can tag it as
(target-riscv64?) and (%current-target-system) so it only takes effect
when needed.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #71 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Efraim Flashner <efraim <at> flashner.co.il>
Cc: 54635 <at> debbugs.gnu.org, Arun Isaac <arunisaac <at> systemreboot.net>
Subject: Re: [PATCH v2 5/5] gnu: Add wfmash.
Date: Thu, 31 Mar 2022 15:07:49 +0200

[Message part 1 (text/plain, inline)]

Efraim Flashner schreef op do 31-03-2022 om 15:18 [+0300]:
> > Arun Isaac schreef op do 31-03-2022 om 12:58 [+0530]:
> > > +                       (("-march=native") ""))
> > 
> > This is also wrong for x86 systems because it makes the build non-
> > reproducible.  Also, has upstream been informed about some of the
> > compiler flags being architecture-specific?
> 
> I'm pretty sure upstream is aware of it, and the -mcx16 flag. That
> whole phase doesn't need to be non-x86_64 only, upstream prefers it
> that way to get fater results

wfmash could be written to detect CPU features at runtime and there is
also --tune.  Also, upstream preferring march=native does not make the
build reproducible.

> but IMO it would be fine to move it into a snippet.

It does not have to be in a snippet, it just needs to be reproducible
(so no march=native, whether on x86 or not).

Upstream seems to be aware of the non-x86
(https://github.com/ekg/wfmash/issues/125) but they do not seem to be
aware of the problems with march=native.

Greetings,
Maxime

[signature.asc (application/pgp-signature, inline)]

Message #74 received at 54635 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 54635 <at> debbugs.gnu.org, Arun Isaac <arunisaac <at> systemreboot.net>
Subject: Re: [PATCH v2 5/5] gnu: Add wfmash.
Date: Thu, 31 Mar 2022 16:09:07 +0300

[Message part 1 (text/plain, inline)]

On Thu, Mar 31, 2022 at 03:07:49PM +0200, Maxime Devos wrote:
> Efraim Flashner schreef op do 31-03-2022 om 15:18 [+0300]:
> > > Arun Isaac schreef op do 31-03-2022 om 12:58 [+0530]:
> > > > +                       (("-march=native") ""))
> > > 
> > > This is also wrong for x86 systems because it makes the build non-
> > > reproducible.  Also, has upstream been informed about some of the
> > > compiler flags being architecture-specific?
> > 
> > I'm pretty sure upstream is aware of it, and the -mcx16 flag. That
> > whole phase doesn't need to be non-x86_64 only, upstream prefers it
> > that way to get fater results
> 
> wfmash could be written to detect CPU features at runtime and there is
> also --tune.  Also, upstream preferring march=native does not make the
> build reproducible.
> 
> > but IMO it would be fine to move it into a snippet.
> 
> It does not have to be in a snippet, it just needs to be reproducible
> (so no march=native, whether on x86 or not).
> 
> Upstream seems to be aware of the non-x86
> (https://github.com/ekg/wfmash/issues/125) but they do not seem to be
> aware of the problems with march=native.

I will let them know. I also added a patch which runs a test suite based
on the github action.


-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #79 received at 54635-done <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 54635-done <at> debbugs.gnu.org, Arun Isaac <arunisaac <at> systemreboot.net>
Subject: Re: [PATCH v2 5/5] gnu: Add wfmash.
Date: Thu, 31 Mar 2022 16:31:28 +0300

[Message part 1 (text/plain, inline)]

On Thu, Mar 31, 2022 at 03:07:49PM +0200, Maxime Devos wrote:
> Efraim Flashner schreef op do 31-03-2022 om 15:18 [+0300]:
> > > Arun Isaac schreef op do 31-03-2022 om 12:58 [+0530]:
> > > > +                       (("-march=native") ""))
> > > 
> > > This is also wrong for x86 systems because it makes the build non-
> > > reproducible.  Also, has upstream been informed about some of the
> > > compiler flags being architecture-specific?
> > 
> > I'm pretty sure upstream is aware of it, and the -mcx16 flag. That
> > whole phase doesn't need to be non-x86_64 only, upstream prefers it
> > that way to get fater results
> 
> wfmash could be written to detect CPU features at runtime and there is
> also --tune.  Also, upstream preferring march=native does not make the
> build reproducible.
> 
> > but IMO it would be fine to move it into a snippet.
> 
> It does not have to be in a snippet, it just needs to be reproducible
> (so no march=native, whether on x86 or not).

I suppose not, but the -mcx16 should be in the snippet, since it adds
compiler flags which only work on some architectures. And while we're at
it we can do the -march=native one too.

> Upstream seems to be aware of the non-x86
> (https://github.com/ekg/wfmash/issues/125) but they do not seem to be
> aware of the problems with march=native.
> 

I've pushed these patches with a few tweaks and an additional patch to
run a test suite based on the github workflow in the repository. The
entire check phase takes ~2 minutes on my pinebook pro, so aarch64
doesn't seem to need to skip some tests like riscv64 does.


-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #82 received at 54635-done <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Efraim Flashner <efraim <at> flashner.co.il>, Maxime Devos
 <maximedevos <at> telenet.be>
Cc: 54635-done <at> debbugs.gnu.org
Subject: Re: [PATCH v2 5/5] gnu: Add wfmash.
Date: Fri, 01 Apr 2022 13:10:11 +0530

Thanks for reviewing and merging this patchset!

>> wfmash could be written to detect CPU features at runtime and there is
>> also --tune.

wfmash's speed is one of the critical features it brings to the
table. We might have rendered the package more or less unusable by
disabling CPU optimizations.

I could try talking to upstream about detecting and using CPU features
at runtime. But, I don't know too much about the topic. Any quick links
I can share with them?

> I've pushed these patches with a few tweaks and an additional patch to
> run a test suite based on the github workflow in the repository. The
> entire check phase takes ~2 minutes on my pinebook pro, so aarch64
> doesn't seem to need to skip some tests like riscv64 does.

I think it is better if upstream provides us a `make check' target to
run these tests. I have asked them:
https://github.com/ekg/wfmash/issues/130 Hard-coding their CI tests
makes the package fragile. In the future, upstream could change the
tests they run.

Message #85 received at 54635-done <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Arun Isaac <arunisaac <at> systemreboot.net>, Efraim Flashner
 <efraim <at> flashner.co.il>
Cc: 54635-done <at> debbugs.gnu.org
Subject: Re: [PATCH v2 5/5] gnu: Add wfmash.
Date: Fri, 01 Apr 2022 11:39:16 +0200

[Message part 1 (text/plain, inline)]

Arun Isaac schreef op vr 01-04-2022 om 13:10 [+0530]:
> wfmash's speed is one of the critical features it brings to the
> table. We might have rendered the package more or less unusable by
> disabling CPU optimizations.
> 
> I could try talking to upstream about detecting and using CPU features
> at runtime. But, I don't know too much about the topic. Any quick links
> I can share with them?

For the concept, maybe
<https://hpc.guix.info/blog/2022/01/tuning-packages-for-a-cpu-micro-architecture/>
, which is about combining high performance with reproducibility using
the (Guix-specific) --tune package transformation.  To implement run-
time detection, the following could be useful:

* Guix article:
<https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/>
* LWN article: <https://lwn.net/Articles/691932/>.

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #88 received at 54635-done <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Maxime Devos <maximedevos <at> telenet.be>, Efraim Flashner
 <efraim <at> flashner.co.il>
Cc: 54635-done <at> debbugs.gnu.org
Subject: Re: [PATCH v2 5/5] gnu: Add wfmash.
Date: Mon, 04 Apr 2022 23:31:27 +0530

Hi Maxime,

> For the concept, maybe
> <https://hpc.guix.info/blog/2022/01/tuning-packages-for-a-cpu-micro-architecture/>
> , which is about combining high performance with reproducibility using
> the (Guix-specific) --tune package transformation.  To implement run-
> time detection, the following could be useful:
>
> * Guix article:
> <https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/>
> * LWN article: <https://lwn.net/Articles/691932/>.

This is helpful. I will read and pass it on to wfmash upstream.

Thanks,
Arun

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.