Subject: bug#53296: closed (Re: bug#53296: Intermittent segfaults when parsing (?) custom package from repo.)

From: [email-masked] (GNU bug Tracking System)
Date: Tue, 14 Feb 2023 12:26:02 +0000

Your bug report

#53296: Intermittent segfaults when parsing (?) custom package from repo.

which was filed against the guix package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
53296: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=53296
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#53296: Intermittent segfaults when parsing (?) custom package from repo.
From: Simon Tournier <[email-masked]>
To: [email-masked]
Cc: [email-masked], Ludovic Courtès <[email-masked]>
Date: Tue, 14 Feb 2023 13:04:52 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi,

>> If there’s no reliable way to reproduce it, I’ll close the bug soon.
>
> This was almost certainly a local hardware issue. The machine this happened on
> ended up showing progressively more mysterious behaviour that could not be
> reproduced on other machines.
>
> Feel free to close!

So closing!

Thanks,
simon

Subject: Intermittent segfaults when parsing (?) custom package from repo.
From: [email-masked]
To: [email-masked]
Date: Sun, 16 Jan 2022 15:12:08 +0900
List-Id: <debbugs-submit.debbugs.gnu.org>

Hey Guix,

Recently working on a package, I have been encounting intermittent segfault
during a build.

The segfault seems to only occur when I have some error in my code that causes
a crash, and the segfaults tend to cluster, appearing unexpectedly for a few
build attempts, and then disappearing right as I think they are reproducible
and try to grab an strace or something. Unfortunately, that's about the extent
of information I have been able to gleen.

The latest segfault happened with the attached package definition. Note, the
offending code is at line 77, where I forgot to remove a docstring from a
variable that used to be a procedure.

Given that non-segfault runs seem to error out so early, is this better thought
to be an issue with Guile? FWIW, I see nothing interesting under
/var/log/guix-daemon.log. However, lines like following show up in the kernel
messages ring:

[318026.268095] guix[7419]: segfault at 18 ip 00007f56ef6a01a3 sp 00007fff15588980 error 4 in libgc.so.1.4.3[7f56ef693000+1b000]
[318026.268116] Code: 8d 2d 71 93 01 00 90 4a 8d 04 e5 00 00 00 00 48 89 04 24 49 8b 45 00 4e 8b 3c e0 4d 85 ff 74 2a 31 ed 0f 1f 44 00 00 4d 89 fe <4d> 8b 7f 08 49 8b 7e 10 48 f7 d7 e8 6d 35 ff ff 85 c0 0f 84 3d 01
[318029.715621] guix[7761]: segfault at 10 ip 00007f9e80b919b9 sp 00007fffd1b2ad20 error 4 in libgc.so.1.4.3[7f9e80b7b000+1b000]
[318029.715638] Code: f7 d2 48 21 d0 48 8b 13 4c 8d 3c c5 00 00 00 00 48 8b 04 c2 48 85 c0 74 78 48 89 ea 48 f7 d2 eb 09 48 8b 40 08 48 85 c0 74 67 <48> 39 10 75 f2 44 8b 05 03 1c 04 00 49 f7 d4 4c 89 60 10 41 bc 01
[318041.537171] guix[8660]: segfault at 10 ip 00007f0d2603c9b9 sp 00007ffc72e998d0 error 4 in libgc.so.1.4.3[7f0d26026000+1b000]
[318041.537185] Code: f7 d2 48 21 d0 48 8b 13 4c 8d 3c c5 00 00 00 00 48 8b 04 c2 48 85 c0 74 78 48 89 ea 48 f7 d2 eb 09 48 8b 40 08 48 85 c0 74 67 <48> 39 10 75 f2 44 8b 05 03 1c 04 00 49 f7 d4 4c 89 60 10 41 bc 01

In the off chance it's helpful, below are some random machine details. Please
let me know if there is anything more pointed or specific I provide.

$ guix system describe

Generation 4	Jan 12 2022 18:59:48	(current)
  file name: /var/guix/profiles/system-4-link
  canonical file name: /gnu/store/sb01mnd31a9x2a0bznzlb2lsy91qwgk6-system
  label: GNU with Linux 5.15.13
  bootloader: grub-efi
  root device: label: "root"
  kernel: /gnu/store/bdf2yw10jr02mhyiwm05yp2qibywqz47-linux-5.15.13/bzImage
  channels:
    guix-bmw:
      repository URL: git://[email-masked]/guix-bmw.git
      branch: master
      commit: 9fb59483371bb5d59fbd27e47baac88263410ac5
    nonguix:
      repository URL: https://gitlab.com/nonguix/nonguix
      branch: master
      commit: 023508df4804dbd9f39cb197525f166bc259f995
    guix:
      repository URL: https://git.savannah.gnu.org/git/guix.git
      branch: master
      commit: 9a2cf2c9232e229f7bb1ab065df2cf0740f65996
  configuration file: /gnu/store/pkf4vzlck0g32hkyvijmlcnp15vh8njv-configuration.scm

$ lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   43 bits physical, 48 bits virtual
Byte Order:                      Little Endian
CPU(s):                          8
On-line CPU(s) list:             0-7
Vendor ID:                       AuthenticAMD
Model name:                      AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx
CPU family:                      23
Model:                           24
Thread(s) per core:              2
Core(s) per socket:              4
Socket(s):                       1
Stepping:                        1
Frequency boost:                 enabled
CPU max MHz:                     2100.0000
CPU min MHz:                     1400.0000
BogoMIPS:                        4191.75
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate ssbd ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca sme sev sev_es
Virtualization:                  AMD-V
L1d cache:                       128 KiB (4 instances)
L1i cache:                       256 KiB (4 instances)
L2 cache:                        2 MiB (4 instances)
L3 cache:                        4 MiB (1 instance)
NUMA node(s):                    1
NUMA node0 CPU(s):               0-7
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full AMD retpoline, IBPB conditional, STIBP disabled, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected

$ lsmem
RANGE                                 SIZE  STATE REMOVABLE  BLOCK
0x0000000000000000-0x00000000bfffffff   3G online       yes   0-23
0x0000000100000000-0x00000005bfffffff  19G online       yes 32-183

Memory block size:       128M
Total online memory:      22G
Total offline memory:      0B

$ lsirq
IRQ     TOTAL NAME
RES 535178549 Rescheduling interrupts
LOC 262669447 Local timer interrupts
 82 125390312 PCI-MSI 2621440-edge amdgpu
CAL  66644945 Function call interrupts
TLB  28850164 TLB shootdowns
 12   7316777 IO-APIC 12-edge i8042
 77   2140601 PCI-MSI 2097156-edge iwlwifi:queue_4
  9   1653508 IO-APIC 9-fasteoi acpi
 73   1583984 PCI-MSI 2097152-edge iwlwifi:default_queue
IWI   1102407 IRQ work interrupts
 65    759824 PCI-MSI 3145728-edge ahci[0000:06:00.0]
  1    532374 IO-APIC 1-edge i8042
 76    253951 PCI-MSI 2097155-edge iwlwifi:queue_3
 75    186427 PCI-MSI 2097154-edge iwlwifi:queue_2
 74    168179 PCI-MSI 2097153-edge iwlwifi:queue_1
  7    100000 IO-APIC 7-fasteoi pinctrl_amd
 52     41159 PCI-MSI 524289-edge nvme0q1
 58     38523 PCI-MSI 524295-edge nvme0q7
 56     38206 PCI-MSI 524293-edge nvme0q5
 53     36960 PCI-MSI 524290-edge nvme0q2
 55     30286 PCI-MSI 524292-edge nvme0q4
 54     28877 PCI-MSI 524291-edge nvme0q3
 57     27421 PCI-MSI 524294-edge nvme0q6
 59     27142 PCI-MSI 524296-edge nvme0q8
MCP      8512 Machine check polls
 35      2317 PCI-MSI 2627584-edge xhci_hcd
 81      1290 PCI-MSI 2633728-edge snd_hda_intel:card1
 44      1215 PCI-MSI 2629632-edge xhci_hcd
 67       338 PCI-MSI 1572864-edge rtsx_pci
 80       316 PCI-MSI 2623488-edge snd_hda_intel:card0
  0        34 IO-APIC 2-edge timer
 78        33 PCI-MSI 2097157-edge iwlwifi:exception
 34        28 PCI-MSI 524288-edge nvme0q0
  8         1 IO-APIC 8-edge rtc0
 25         0 PCI-MSI 18432-edge PCIe PME, aerdrv
 26         0 PCI-MSI 20480-edge PCIe PME, aerdrv
 27         0 PCI-MSI 22528-edge PCIe PME, aerdrv
 28         0 PCI-MSI 28672-edge PCIe PME, aerdrv, pciehp
 29         0 PCI-MSI 133120-edge PCIe PME
 30         0 PCI-MSI 135168-edge PCIe PME
 36         0 PCI-MSI 2627585-edge xhci_hcd
 37         0 PCI-MSI 2627586-edge xhci_hcd
 38         0 PCI-MSI 2627587-edge xhci_hcd
 39         0 PCI-MSI 2627588-edge xhci_hcd
 40         0 PCI-MSI 2627589-edge xhci_hcd
 41         0 PCI-MSI 2627590-edge xhci_hcd
 42         0 PCI-MSI 2627591-edge xhci_hcd
 45         0 PCI-MSI 2629633-edge xhci_hcd
 46         0 PCI-MSI 2629634-edge xhci_hcd
 47         0 PCI-MSI 2629635-edge xhci_hcd
 48         0 PCI-MSI 2629636-edge xhci_hcd
 49         0 PCI-MSI 2629637-edge xhci_hcd
 50         0 PCI-MSI 2629638-edge xhci_hcd
 51         0 PCI-MSI 2629639-edge xhci_hcd
 60         0 PCI-MSI 524297-edge nvme0q9
 61         0 PCI-MSI 524298-edge nvme0q10
 62         0 PCI-MSI 524299-edge nvme0q11
 63         0 PCI-MSI 524300-edge nvme0q12
 71         0 PCI-MSI 2625537-edge ccp-1
 79         0 PCI-MSI 1048576-edge enp2s0
NMI         0 Non-maskable interrupts
SPU         0 Spurious interrupts
PMI         0 Performance monitoring interrupts
RTR         0 APIC ICR read retries
TRM         0 Thermal event interrupts
THR         0 Threshold APIC interrupts
DFR         0 Deferred Error APIC interrupts
MCE         0 Machine check exceptions
ERR         0 
MIS         0 
PIN         0 Posted-interrupt notification event
NPI         0 Nested posted-interrupt event
PIW         0 Posted-interrupt wakeup event

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2022 B. Wilson <[email-masked]>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (gnu packages jsoftware)
  #:use-module (guix build utils)
  #:use-module (guix build-system gnu)
  #:use-module (guix build-system trivial)
  #:use-module (guix git-download)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (guix packages)
  #:use-module (guix utils)
  #:use-module (gnu packages)
  #:use-module (gnu packages libedit)
  #:use-module (gnu packages llvm)
  #:use-module (gnu packages maths)
  #:use-module (guix gexp)
  #:use-module (ice-9 ftw)
  #:use-module (ice-9 match)
  #:use-module (ice-9 regex)
  #:use-module (ice-9 rdelim)
  #:use-module (srfi srfi-1)
  #:use-module (srfi srfi-26)
  #:use-module (srfi srfi-71))

;;; TODO: Make importer and packages for J addons:
;;; http://www.jsoftware.com/jal/

;;; TODO: Package up j80x series

(define (jname prefix release-type)
  "Return a package name for J, including RELEASE-TYPE only if not 'release."
  (match release-type
    ('release prefix)
    (_        (string-append prefix "-" (symbol->string release-type)))))

;; We want a version string where packages specifications like pkg@MAJOR work.
;; This requires that the first version part separator be dot.  Subsequent
;; separators are hyphen, mirror `git-version' etc.
(define* (jversion->string major #:optional minor revision commit)
  "Return a version string formatted like MAJOR.MINOR-REVISION-COMMIT.  Only
  MAJOR is required, and MINOR defaults to ``0'' if not supplied."
  (let* ((commit (and commit (string-take commit 7)))
         (minor (or minor "0"))
         (sub-parts (filter (cut (compose not eq?) #f <>)
                            (list minor revision commit))))
    (string-append major "." (string-join sub-parts "-"))))

(define* (jrelease-string release-type #:optional version-minor)
  "Construct J release identifier string."
  (let ((release-type (symbol->string release-type)))
    (if version-minor
     (string-append release-type "-" version-minor)
     release-type)))

(define* (jinfo->git-tag version-major release-type #:optional version-minor)
  "Given version parameters, construct a git tag for upstream releases."
  (string-append "j" version-major (jrelease-string release-type version-minor)))

(define ijconsole
  "G-exp script that detects AVX/AVX2 support at runtime and executes jconsole
  with the appropriate libj.so and profile.ijs."
  (with-imported-modules '((guix cpu)
                           (guix memoization)
                           (guix profiling)
                           (guix sets)
                           (srfi srfi-26))
    (program-file "ijconsole"
      #~(begin
          (use-modules ((guix cpu)     #:select (cpu-flags current-cpu))
                       ((guix sets)    #:select (set-contains?))
                       ((srfi srfi-26) #:select (cute)))

          ;; Assume that this script will be installed under bin/.
          (define %basedir (dirname (dirname (current-filename))))

          (let* ((jconsole (string-append %basedir "/libexec/j/jconsole"))
                 (cpu-has-flag?
                   (cute set-contains? (cpu-flags (current-cpu)) <>))
                 (libj (format #f "~a/lib/j/libj~a.so" %basedir
                               (cond ((cpu-has-flag? "avx2") "-avx2")
                                     ((cpu-has-flag? "avx") "-avx")
                                     (else ""))))
                 (jprofile (string-append %basedir "/etc/j/profile.ijs")))
            (apply execl jconsole "ijconsole" "-lib" libj "-jprofile" jprofile
                   (cdr (command-line))))))))

(define* (make-j #:key
                 version
                 revision
                 hash
                 tag
                 commit
                 (release-type 'release)
                 (patches '())
                 (extra-inputs '())
                 (extra-envars '())
                 (builder "guix.gnu.org"))
 (let* ((version-major version-minor (if (pair? version)
                                       (car+cdr version)
                                       (values version #f))))
 (package
   (name (jname "jsoftware-j" release-type))
   (version (jversion->string version-major version-minor revision commit))
   (source
    (origin
      (method git-fetch)
      (uri (git-reference
            (url "https://github.com/jsoftware/jsource")
            (commit (or commit tag
                        (jinfo->git-tag version-major
                                        release-type
                                        version-minor)))))
      (sha256 (base32 hash))
      (file-name (git-file-name name version))
      (patches patches)))
   (build-system gnu-build-system)
   (native-inputs (list clang-toolchain))
   (inputs (cons* `("libedit" ,libedit)
                  `("libomp" ,libomp)
                  `("ijconsole" ,ijconsole)
                   ;; profilex.ijs overrides ~install and ~addons
                   ;; directories to reside under the user-writable ~user.
                   ;; This allows local-install of addons via pacman.  TODO:
                   ;; Guix-ify J addons as well.
                  `("profilex" ,(search-auxiliary-file
                                  "jsoftware/profilex.ijs"))
                  extra-inputs))
   (arguments
    `(#:modules (((ice-9 ftw) #:select (scandir))
                 ((ice-9 popen) #:select (open-pipe* close-pipe))
                 ((ice-9 regex) #:select (match:substring string-match))
                 ((ice-9 threads) #:select (parallel par-for-each))
                 ((srfi srfi-26) #:select (cut))
                 ((srfi srfi-1) #:select (fold))
                 ,@%gnu-build-system-modules)
      #:phases
      ;; Upstream's build system consists of ad-hoc scripts that build build up
      ;; (very complicated) environment variables to pass to make.  The basic
      ;; build process looks like this:
      ;;
      ;;   1) Copy jsrc/jversion-x.h to jsrc/jversion.h and edit values;
      ;;   2) Set jplatform and j64x environment variables;
      ;;   3) Run make2/build_jconsole.sh and make2/build_libj.sh;
      ;;
      ;; However, upstream expects users to run J directly from the source
      ;; directory; they do not supply a make `install' target.  Thus it takes
      ;; some massaging to install files in FHS-style directories.
      (modify-phases %standard-phases
        ;; In particular, we have to set up
        ;;
        ;;   1) jsrc/jversion.h as in a typical build;
        ;;   2) jlibrary/bin/profilex.ijs to point to writable directories;
        ;;   3) make2/build_*.sh to respect standard build conventions;
        ;;   4) jsrc/jconsole.c to fix libedit dlopen; and
        ;;   5) Hard coded references to addons directory.
        (replace 'configure
          (lambda* (#:key target inputs outputs #:allow-other-keys)
            (let* ((clang-toolchain (assoc-ref inputs "clang-toolchain"))
                   (clang (string-append clang-toolchain "/bin/clang"))
                   (libedit (assoc-ref inputs "libedit"))
                   (out (assoc-ref outputs "out")))
              ;; Set up build constants
              (copy-file "jsrc/jversion-x.h" "jsrc/jversion.h")
              (substitute* "jsrc/jversion.h"
                (("^#define jversion.*$")
                 (format #f "#define jversion ~s\n" ,version-major))
                (("^#define jtype.*$")
                 (format #f "#define jtype ~s\n"
                         ,(jrelease-string release-type version-minor)))
                (("^#define jbuilder.*$")
                 (format #f "#define jbuilder ~s\n" ,builder)))
              ;; Munge the build scripts into reason:
              ;; 1. Short-circuit the fragile compiler detection;
              ;; 2. Make sure to include our CFLAGS and LFLAGS; and
              ;; 3. Propagate script errors to top level.
              (for-each
               (lambda (file)
                 (with-directory-excursion "make2"
                   (substitute* file
                     ;; The `compiler' variable doesn't point to the actual
                     ;; compiler.  It is just a switch to tell the build
                     ;; scripts whether to use gcc- or clang-specific flags.
                     (("^compiler=.*$") "compiler=clang\n")
                     (("^LDFLAGS=\"" def) (string-append def "$LDFLAGS "))
                     (("^(common=\")(\\$USETHREAD.*)$" _ def rest)
                      (string-append def "$CFLAGS " rest))
                     (("^#!.*" shebang)
                      (string-append shebang "set -o errexit\n")))))
                 '("build_jconsole.sh" "build_libj.sh"))
              ;; The jconsole manually loads libedit with dlopen.  The path
              ;; must be absolute to correctly point to our input.
              (substitute* "jsrc/jconsole.c"
                (("libedit\\.so\\.[0-9]" so-file)
                 (format #f "~a/lib/~a" libedit so-file)))
              ;; The ~addons/dev directory supplies tentative J-script
              ;; definitions of new J engine functionality.  Since we point
              ;; ~addons under the ~user directory, we move it under ~system
              ;; instead, which sits as-is in the output.
              (with-directory-excursion "jsrc"
                (for-each
                  (lambda (file)
                    (substitute* file (("~addons/dev") "~system/dev")))
                  (scandir "."
                    (lambda (f) (eq? (stat:type (stat f)) 'regular)))))
              ;; Implementation of 9!:14 records build time which breaks build
              ;; reproducibility.  Note that upstream code depends on the exact
              ;; format of these strings, so we need to mimic the standard.
              (substitute* "jsrc/j.c"
                (("__DATE__") "\"Jan 01 1970\"")
                (("__TIME__") "\"00:00:00\""))
              ;; Upstream recommends using clang, with GCC support being
              ;; second-class, often resulting in build failures.
              (setenv "CC" clang))))

        ;; The build output depends primarily on the values of the `jplatform'
        ;; and `j64x' environment variables.  If the target is ARM, then
        ;; `jplatform' is "raspberry", otherwise it is `linux'.  In addition to
        ;; 32- and 64- bit versions, `j64x' controlls whether AVX or AVX2
        ;; variants of libj are built.
        ;;
        ;; However, build targets are not fine-grained enough to distinguish
        ;; between CPU features.  Thus we build and install all variants of
        ;; libj, expecting jconsole to be called with a wrapper script that
        ;; detects AVX features and loads the appropriate libj at runtime.
        (replace 'build
          (lambda _
            (setenv "USE_OPENMP" "1")
            (setenv "USE_THREAD" "1")
            (for-each (lambda (var-val) (apply setenv var-val))
                      (quote ,extra-envars))
            ;; The build scripts assume that PWD is make2.
            (with-directory-excursion "make2"
              (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
                     (target-bit ,(if (target-64bit?) "64" "32"))
                     (run (lambda* (script #:key (avx ""))
                            (invoke "env"
                                    (string-append "jplatform=" platform)
                                    (string-append "j64x=j" target-bit avx)
                                    script))))
                (parallel
                  ;; Since jconsole doesn't depend on AVX features, we just
                  ;; build it once.
                  (run "./build_jconsole.sh")
                  (run "./build_libj.sh")
                  (if ,(target-64bit?)
                    (parallel
                      (run "./build_libj.sh" #:avx "avx")
                      (run "./build_libj.sh" #:avx "avx2"))))))))
        ;; The test suite is expected to be run as follows for each variant of
        ;; libj that we build:
        ;;
        ;;     $ echo 'RUN ddall' | jconsole test/tsu.ijs
        ;;
        ;; This requires a working jconsole with accessible jlibrary files.  We
        ;; simply place these all under test/bin.
        (replace 'check
          (lambda* (#:key tests? #:allow-other-keys)
            (when tests?
              (let ((platform ,(if (target-arm?) "raspberry" "linux")))
                (mkdir-p "test/bin")
                (for-each
                  (lambda (dir)
                    (let ((source (string-append "jlibrary/" dir))
                          (dest (string-append "test/bin/" dir)))
                    (begin
                      (mkdir-p dest)
                      (copy-recursively source dest))))
                  '("system" "tools" "addons"))
                ;; The jlibrary/dev directory only sometimes exists, but needs
                ;; to be copied into the ~system directory when it does.
                (for-each
                  (lambda (dev-dir)
                    (if (file-exists? dev-dir)
                      (copy-recursively dev-dir "test/bin/system/dev")))
                  '("jlibrary/dev" "jlibrary/addons/dev"))
                (par-for-each
                  (lambda (dir)
                    (let* ((bin (string-append "bin/" platform))
                           (jbit ,(if (target-64bit?) "j64" "j32"))
                           (jconsole (string-append bin "/" jbit
                                                    "/jconsole"))
                           (source (string-append bin "/" dir))
                           (dest (string-append "test/bin/" dir)))
                      (begin
                        (mkdir-p dest)
                        (copy-recursively source dest)
                        (install-file "jlibrary/bin/profile.ijs" dest)
                        (install-file jconsole dest)
                        (let* ((jconsole (string-append dest "/jconsole"))
                               (tests "test/tsu.ijs")
                               (port (open-pipe* OPEN_WRITE jconsole tests)))
                          (display "RUN ddall\n" port)
                          (when (not (zero? (status:exit-val
                                              (close-pipe port))))
                            (error "Some J build tests failed."))))))
                  (scandir (string-append "bin/" platform)
                           (negate (cut member <> '("." "..")))))
                #t))))
        ;; Now that everything is built, installation is fairly
        ;; straightforward, following FHS conventions.  The only quirk is that
        ;; we install jconsole under /libexec to make room for the wrapper
        ;; replacement under /bin.
        (replace 'install
          (lambda* (#:key outputs inputs #:allow-other-keys)
            (let* ((platform ,(if (target-arm?) "raspberry" "linux"))
                   (jbit ,(if (target-64bit?) "j64" "j32"))
                   (out (assoc-ref outputs "out"))
                   (bin (string-append out "/bin"))
                   (etc (string-append out "/etc/j"))
                   (lib (string-append out "/lib/j"))
                   (libexec (string-append out "/libexec/j"))
                   (share (string-append out "/share/j"))
                   (system (string-append share "/system"))
                   (dev (string-append system "/dev")))
              (mkdir-p bin)
              (copy-file (assoc-ref inputs "ijconsole")
                         (string-append bin "/ijconsole-" ,version-major))
              (mkdir-p lib)
              (for-each
                (lambda (jarch)
                  (let* ((jbin (string-join `("bin" ,platform ,jarch) "/"))
                         (javx-match (string-match "avx.*" jarch))
                         (javx (if (not javx-match) ""
                                 (match:substring javx-match)))
                         (sep (if javx-match "-" ""))
                         (source (string-append jbin "/libj.so"))
                         (dest (format #f "~a/libj~a~a.so" lib sep javx)))
                    (copy-file source dest)))
                (scandir (string-append "bin/" platform)
                         (negate (cut member <> '("." "..")))))
              (install-file (string-append "bin/" platform "/" jbit "/jconsole")
                            libexec)
              (copy-recursively "jlibrary/system" system)
              (for-each
                (lambda (source-dev)
                  (if (access? source-dev R_OK)
                    (copy-recursively source-dev dev)))
                '("jlibrary/dev" "jlibrary/addons/dev"))
              (install-file "jlibrary/bin/profile.ijs" etc)
              (copy-file (assoc-ref inputs "profilex")
                         (string-append etc "/profilex.ijs"))))))))
   (home-page "https://www.jsoftware.com/")
   (synopsis "Ascii-only, array programming language in the APL family")
   (description
    "J is a high-level, general-purpose programming language that is
particularly suited to the mathematical, statistical, and logical analysis of
data.  It is a powerful tool for developing algorithms and exploring problems
that are not already well understood.")
   (license license:gpl3+))))

(define-public jsoftware-j-901
  (make-j
    #:version '("901" . "f")
    #:hash "1776021m0j1aanzwg60by83n53pw7i6afd5wplfzczwk8bywax4p"
    #:patches (search-patches "jsoftware-j901-f-fixes.patch")))

(define j-build-configuration-with-sleef
  `(#:extra-inputs (("sleef" ,sleef))
    #:extra-envars (("USE_SLEEF_SRC" "0")
                    ("LDFLAGS" "-lsleef"))))

(define-public jsoftware-j-902
  (apply make-j
    (append j-build-configuration-with-sleef
      `(#:version ,'("902" . "b")
        #:hash "0j67vgikqflwjqacsdicasvyv1k54s2c8vjgwmf0ix7l41p4xqz0"))))

(define-public jsoftware-j-903
  (apply make-j
    (append j-build-configuration-with-sleef
      `(#:version ,'("903" . "a")
        #:tag "903-release-a"
        #:hash "1fcfl7q7c2vj4fmnqqc8c6hwgsjm20ff93v8xxfniasss1b2fmc4"))))

(define-public (jsoftware-ijconsole-symlink jpkg)
  "Provide bin/ijconsole symlink that points to pkg's bin/ijconsole-<jversion>"
  (package
    (name "jsoftware-ijconsole")
    (version (package-version jpkg))
    (source #f)
    (build-system trivial-build-system)
    (propagated-inputs `(("jpkg" ,jpkg)))
    (arguments
      `(#:modules ((guix build utils)
                   (srfi srfi-26))
        #:builder
        (begin
          (use-modules ((guix build utils) #:select (mkdir-p))
                       ((ice-9 regex) #:select (string-match))
                       ((ice-9 ftw) #:select (scandir))
                       ((srfi srfi-26) #:select (cut)))
          (let* ((out (assoc-ref %outputs "out"))
                 (jpkg (assoc-ref %build-inputs "jpkg"))
                 (ijconsole (car (scandir (string-append jpkg "/bin")
                                       (cut string-match "ijconsole-.*" <>))))
                 (source (string-append jpkg "/bin/" ijconsole))
                 (dest (string-append out "/bin/ijconsole")))
            (mkdir-p (dirname dest))
            (symlink source dest)))))
  (home-page (package-home-page jpkg))
  (synopsis "Provide `ijconsole' symlink to default interpreter version")
  (description
  "The interpreter provided by the J package has a filename like
ijconsole-<version>, which provides support for having multiple, concurrent
versions installed.  This package provides a version-agnostic `ijconsole'
symlink to interpreter version indicated and build time.")
  (license license:gpl3+)))