GNU bug report logs - #77033
Deploy to machine with different architecture

Previous Next

To reply to this bug, email your comments to 77033 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sergey Trofimov <sarg <at> sarg.org.ru>
To: guix-patches <at> gnu.org
Cc: Sergey Trofimov <sarg <at> sarg.org.ru>
Subject: [PATCH] deploy: Support --target and --system.
Date: Sat, 15 Mar 2025 12:59:59 +0100

* guix/scripts/deploy.scm: Support native and cross build options.
---
 guix/scripts/deploy.scm | 30 +++++++++++++++++++++---------
 1 file changed, 21 insertions(+), 9 deletions(-)

diff --git a/guix/scripts/deploy.scm b/guix/scripts/deploy.scm
index e2ef0006e0..94e0d69936 100644
--- a/guix/scripts/deploy.scm
+++ b/guix/scripts/deploy.scm
@@ -26,6 +26,7 @@ (define-module (guix scripts deploy)
   #:use-module (guix scripts)
   #:use-module (guix scripts build)
   #:use-module (guix store)
+  #:use-module (guix utils)
   #:use-module (guix gexp)
   #:use-module (guix ui)
   #:use-module ((guix status) #:select (with-status-verbosity))
@@ -54,6 +55,8 @@ (define (show-help)
   (display (G_ "Usage: guix deploy [OPTION] FILE...
 Perform the deployment specified by FILE.\n"))
   (show-build-options-help)
+  (show-cross-build-options-help)
+  (show-native-build-options-help)
   (newline)
   (display (G_ "
   -h, --help             display this help and exit"))
@@ -93,21 +96,22 @@ (define %options
          (option '(#\x "execute") #f #f
                  (lambda (opt name arg result)
                    (alist-cons 'execute-command? #t result)))
-         (option '(#\s "system") #t #f
-                 (lambda (opt name arg result)
-                   (alist-cons 'system arg
-                               (alist-delete 'system result eq?))))
          (option '(#\v "verbosity") #t #f
                  (lambda (opt name arg result)
                    (let ((level (string->number* arg)))
                      (alist-cons 'verbosity level
                                  (alist-delete 'verbosity result)))))
 
-         %standard-build-options))
+         (append
+          %standard-build-options
+          %standard-native-build-options
+          %standard-cross-build-options)))
 
 (define %default-options
   ;; Alist of default option values.
   `((verbosity . 1)
+    (system . ,(%current-system))
+    (target . #f)
     (debug . 0)
     (graft? . #t)
     (substitutes? . #t)
@@ -186,9 +190,13 @@ (define (deploy-machine* store machine)
             (when (deploy-error-should-roll-back c)
               (info (G_ "rolling back ~a...~%")
                     (machine-display-name machine))
-              (run-with-store store (roll-back-machine machine)))
+              (run-with-store store (roll-back-machine machine)
+                              #:system (%current-system)
+                              #:target (%current-target-system)))
             (apply throw (deploy-error-captured-args c))))
-      (run-with-store store (deploy-machine machine))
+      (run-with-store store (deploy-machine machine)
+           #:system (%current-system)
+           #:target (%current-target-system))
 
     (info (G_ "successfully deployed ~a~%")
           (machine-display-name machine))))
@@ -266,7 +274,9 @@ (define (invoke-command store machine command)
                (loop (cons line lines))))))))
 
   (match (run-with-store store
-           (machine-remote-eval machine invocation))
+           (machine-remote-eval machine invocation)
+           #:system (%current-system)
+           #:target (%current-target-system))
     ((code output)
      (match code
        ((? zero?)
@@ -325,7 +335,9 @@ (define-command (guix-deploy . args)
                                               #:verbosity
                                               (assoc-ref opts 'verbosity)
                                               #:dry-run? dry-run?)
-            (parameterize ((%graft? (assq-ref opts 'graft?)))
+            (parameterize ((%graft? (assq-ref opts 'graft?))
+                           (%current-target-system (assoc-ref opts 'target))
+                           (%current-system (assoc-ref opts 'system)))
               (if execute-command?
                   (match command
                     (("--" command ..1)

base-commit: 412f411d4f8780e6b60b448caae17f01c09be0eb
prerequisite-patch-id: f9cc903b8048c8c6fde576fbf38ab110263020e3
-- 
2.48.1

Message #8 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Roman Scherer <roman.scherer <at> burningswell.com>
To: 77033 <at> debbugs.gnu.org
Cc: ludo <at> gnu.org, sarg <at> sarg.org.ru, maxim.cournoyer <at> gmail.com
Subject: [PATCH] deploy: Support --target and --system.
Date: Sun, 16 Mar 2025 12:42:17 +0100

[Message part 1 (text/plain, inline)]

Hello Sergey,

this looks good to me and is better than what we have now. I'm testing
this right now and will report back when it went through ...

But I'm wondering if the target architecture should ideally be choosen
per machine. Say you have 2 machines with different architectures in a
list of machines you want to deploy.

I don't see a "system" field in the machine or operating system
record. So I'm not sure if we have a way to infer that yet.

Would it make sense to add such a field and chose the target
architecture per machine?

Thanks, Roman.

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Roman Scherer <roman.scherer <at> burningswell.com>
To: 77033 <at> debbugs.gnu.org
Cc: ludo <at> gnu.org, sarg <at> sarg.org.ru, maxim.cournoyer <at> gmail.com
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Mon, 17 Mar 2025 18:31:03 +0100

[Message part 1 (text/plain, inline)]

So, I tried building this machine on an aarch64 system with the qemu-bin
service running now a couple of times ...

```
(use-modules (gnu machine)
             (gnu machine hetzner))

(list (machine
       (operating-system %hetzner-os-x86)
       (environment hetzner-environment-type)
       (configuration (hetzner-configuration
                       (server-type "cx42")
                       (ssh-key "/home/roman/workspace/guix/id_rsa")))))
```

I used this command:

```
[roman <at> m1 guix]$ ./pre-inst-env guix deploy hetzner-minimal.scm --no-offload --target=x86_64-linux-gnu --timeout=864000 --max-silent-time=864000
The following 1 machine will be deployed:
  guix-x86

guix deploy: deploying to guix-x86...
guix deploy: warning: <machine-ssh-configuration> without a 'host-key' is deprecated
substitute: looking for substitutes on 'https://substitutes.asahi-guix.org'... 100.0%
substitute: looking for substitutes on 'https://substitutes.nonguix.org'... 100.0%
substitute: looking for substitutes on 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: looking for substitutes on 'https://ci.guix.gnu.org'... 100.0%
The following derivations will be built:
  /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv
  /gnu/store/8js0zmi3c2zg7ks3qnpiwsckam37vi67-guile-3.0.9.drv

building /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv...
```

And it always "hangs" there. I haven't much investigated this yet. Just
a small update.

Roman

Roman Scherer <roman.scherer <at> burningswell.com> writes:

> Hello Sergey,
>
> this looks good to me and is better than what we have now. I'm testing
> this right now and will report back when it went through ...
>
> But I'm wondering if the target architecture should ideally be choosen
> per machine. Say you have 2 machines with different architectures in a
> list of machines you want to deploy.
>
> I don't see a "system" field in the machine or operating system
> record. So I'm not sure if we have a way to infer that yet.
>
> Would it make sense to add such a field and chose the target
> architecture per machine?
>
> Thanks, Roman.

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Sergey Trofimov <sarg <at> sarg.org.ru>
To: Roman Scherer <roman.scherer <at> burningswell.com>
Cc: ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Mon, 17 Mar 2025 21:33:33 +0100

Hi Roman,

Roman Scherer <roman.scherer <at> burningswell.com> writes:

> So, I tried building this machine on an aarch64 system with the qemu-bin
> service running now a couple of times ...
>
> ```
> (use-modules (gnu machine)
>              (gnu machine hetzner))
>
> (list (machine
>        (operating-system %hetzner-os-x86)
>        (environment hetzner-environment-type)
>        (configuration (hetzner-configuration
>                        (server-type "cx42")
>                        (ssh-key "/home/roman/workspace/guix/id_rsa")))))
> ```
>
> I used this command:
>
> ```
> [roman <at> m1 guix]$ ./pre-inst-env guix deploy hetzner-minimal.scm --no-offload --target=x86_64-linux-gnu --timeout=864000 --max-silent-time=864000
> The following 1 machine will be deployed:
>   guix-x86
>
> guix deploy: deploying to guix-x86...
> guix deploy: warning: <machine-ssh-configuration> without a 'host-key' is deprecated
> substitute: looking for substitutes on 'https://substitutes.asahi-guix.org'... 100.0%
> substitute: looking for substitutes on 'https://substitutes.nonguix.org'... 100.0%
> substitute: looking for substitutes on 'https://bordeaux.guix.gnu.org'... 100.0%
> substitute: looking for substitutes on 'https://ci.guix.gnu.org'... 100.0%
> The following derivations will be built:
>   /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv
>   /gnu/store/8js0zmi3c2zg7ks3qnpiwsckam37vi67-guile-3.0.9.drv
>
> building /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv...
> ```
>
> And it always "hangs" there. I haven't much investigated this yet. Just
> a small update.
>

You could try `guix build
/gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv` to see
what it does and where it hangs.

Also try `--system=x86_64-linux`, it should be fast.

>> I don't see a "system" field in the machine or operating system
>> record. So I'm not sure if we have a way to infer that yet.
>>
>> Would it make sense to add such a field and chose the target
>> architecture per machine?
>>

This totally makes sense to me and is the correct way. However I wonder
if both the `system` and the `target` fields should be added, so that
the user could choose either native or cross build. From my experience
"native" build (as provided by qemu) is much faster. I'll try running
a cross build tomorrow, but it is painfully slow to build the toolchain
on my machine:


--8<---------------cut here---------------start------------->8---
The following 1 machine will be deployed:
  guix-arm

guix deploy: deploying to guix-arm, system=x86_64-linux, target=aarch64-linux-gnu...
guix deploy: warning: <machine-ssh-configuration> without a 'host-key' is deprecated
substitute: looking for substitutes on 'https://substitutes.nonguix.org'... 100.0%
substitute: looking for substitutes on 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: looking for substitutes on 'https://ci.guix.gnu.org'... 100.0%
The following derivations will be built:
  /gnu/store/gxr8v1yisdiyndka0abxrc0xzrra66sv-binutils-cross-aarch64-linux-gnu-2.41.drv
  /gnu/store/lch3711iiczn6smxsr7r3sj991p8avwv-ld-wrapper-aarch64-linux-gnu-0.drv
  /gnu/store/zmsnlbyml0vmphfdxyxw4ps25bgrwz92-gcc-cross-sans-libc-aarch64-linux-gnu-14.2.0.drv
  /gnu/store/57jnlmvqlvk6jkyvqcnrk4psffhmak91-linux-libre-headers-cross-aarch64-linux-gnu-5.15.49.drv
  /gnu/store/b4f1my595ggl7d5qn46vr6qllwx7g49z-glibc-cross-aarch64-linux-gnu-2.39.drv
  /gnu/store/sl5vfnwdarghf9ypbspq1bdlamnz3j2a-gcc-cross-aarch64-linux-gnu-14.2.0.drv
  /gnu/store/3vp8a7mz1576xbk278k9b73nx2zqmzlw-libffi-3.4.4.drv
  /gnu/store/y3hqwsbc8rb2g1mac8c9vsdmaacf20xm-libatomic-ops-7.6.12.drv
  /gnu/store/bd09d178ni5sp9db62w869c6m7d3sh6v-libgc-8.2.4.drv
  /gnu/store/cs7mzhrypgdad8v0v29arafc8brl7ynd-bash-minimal-5.1.16.drv
  /gnu/store/np51g0ak713az6shj6sv9j3wkq4cjvjx-libunistring-1.1.drv
  /gnu/store/rbkb4ig158h9gblbrah5nx5annvfpb4q-libxcrypt-4.4.36.drv
  /gnu/store/lfmamfv5vx690l9n6a1ixbbk6kzw3gsr-guile-3.0.9.drv
--8<---------------cut here---------------end--------------->8---

Message #17 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Sergey Trofimov <sarg <at> sarg.org.ru>
To: Roman Scherer <roman.scherer <at> burningswell.com>
Cc: ludo <at> gnu.org, 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Tue, 18 Mar 2025 21:57:30 +0100

Roman Scherer <roman.scherer <at> burningswell.com> writes:

> Hello Sergey,
>
> this looks good to me and is better than what we have now. I'm testing
> this right now and will report back when it went through ...
>

I think this patch is not necessary at all, your code already sets the
`system` field of `machine-ssh-configuration` to the value appropriate
to one returned by hetzner-api.

It looks to me that the bug is in incorrect parameterization, see the
following patch. I've been able to deploy using it (again
`build-locally?` set to #f as it takes too long otherwise). Please test
this one.

--8<---------------cut here---------------start------------->8---
From 988d192cfe4833761d0714aa8a53392c09dd5c9d Mon Sep 17 00:00:00 2001
Message-ID: <988d192cfe4833761d0714aa8a53392c09dd5c9d.1742331137.git.sarg <at> sarg.org.ru>
From: Sergey Trofimov <sarg <at> sarg.org.ru>
Subject: [PATCH] machine: ssh: Use monadic parameterize for monads.

* gnu/machine/ssh (deploy-managed-host): Use store-parameterize in
monadic context.

Change-Id: I6441a5dac856633f2c2a00b77f21a385bf3a9510
---
 gnu/machine/ssh.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm
index 73d5dc513e..0ce2860236 100644
--- a/gnu/machine/ssh.scm
+++ b/gnu/machine/ssh.scm
@@ -542,8 +542,8 @@ (define (deploy-managed-host machine)
                                         host
                                         (inferior-exception-arguments c))))
                               os))
-          (parameterize ((%current-system system)
-                         (%current-target-system #f))
+          (store-parameterize ((%current-system system)
+                               (%current-target-system #f))
             (with-roll-back #t
               (mbegin %store-monad
                 (upgrade-shepherd-services (eval/error-handling c

base-commit: df799a61ef73451e587cb81942c7bfcbbade9e7e
--
2.48.1

--8<---------------cut here---------------end--------------->8---

Message #20 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Sergey Trofimov <sarg <at> sarg.org.ru>
Cc: ludo <at> gnu.org, Roman Scherer <roman.scherer <at> burningswell.com>,
 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Thu, 20 Mar 2025 16:07:44 +0900

Hi Sergey, Roman,

Sergey Trofimov <sarg <at> sarg.org.ru> writes:

[...]

>>> I don't see a "system" field in the machine or operating system
>>> record. So I'm not sure if we have a way to infer that yet.
>>>
>>> Would it make sense to add such a field and chose the target
>>> architecture per machine?
>>>
>
> This totally makes sense to me and is the correct way. However I wonder
> if both the `system` and the `target` fields should be added, so that
> the user could choose either native or cross build.

We already have platform-system->target and platform-target->system, so
I assume having one or the other could be enough?.  Looking at
machine-ssh-configuration, I also see we already have a 'system' field.

Perhaps 'guix deploy' could have a '--cross-build' option, or the record
could have a 'cross-build?' field, or both, with the CLI taking
precedence over the later, and the target type would be inferred from
the system via platform-system->target?

In my experience, assuming a powerful x86_64 workstation,
cross-compilation > native-compilation (on lower spec ARM machine) >
emulated native compilation (QEMU).

I hope I understood the context correctly :-).

-- 
Thanks,
Maxim

Message #23 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Sergey Trofimov <sarg <at> sarg.org.ru>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: ludo <at> gnu.org, Roman Scherer <roman.scherer <at> burningswell.com>,
 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Thu, 20 Mar 2025 14:43:20 +0100

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:

> Hi Sergey, Roman,
>
> Sergey Trofimov <sarg <at> sarg.org.ru> writes:
>
> [...]
>
>>>> I don't see a "system" field in the machine or operating system
>>>> record. So I'm not sure if we have a way to infer that yet.
>>>>
>>>> Would it make sense to add such a field and chose the target
>>>> architecture per machine?
>>>>
>>
>> This totally makes sense to me and is the correct way. However I wonder
>> if both the `system` and the `target` fields should be added, so that
>> the user could choose either native or cross build.
>
> We already have platform-system->target and platform-target->system, so
> I assume having one or the other could be enough?.  Looking at
> machine-ssh-configuration, I also see we already have a 'system' field.
>
>
> Perhaps 'guix deploy' could have a '--cross-build' option, or the record
> could have a 'cross-build?' field, or both, with the CLI taking
> precedence over the later, and the target type would be inferred from
> the system via platform-system->target?
>

Consider such case:

deployer = x86_64, (aarch64 via binfmt)
builder1 = x86_64, (aarch64 via binfmt)
builder2 = aarch64, (x86_64 via binfmt)
remote1 = x86_64
remote2 = aarch64

The user intends to run `guix deploy all-remotes.scm` on `deployer` to
deploy both `remote1` and `remote2`.

There are many ways how to build the derivations - deployer and builders
could all build native/emulated/cross, remotes could build for
themselves (`build-locally?` to `#f`). I am not sure to what extent this
should be configurable, however here are a couple things I'd change:
- add `platform` to `machine` record
- remove `system` from `machine-ssh-configuration`
- remove `build-locally?` from `machine-ssh-configuration`
- add `build-options` to `machine`
  - `build-host` = `local|offload|remote`
  - `build-type` = `native|cross`

Additionally `guix-daemon` should know all the systems the host
supports. Currently it knows only the native one (`--system` parameter).

>
> In my experience, assuming a powerful x86_64 workstation,
> cross-compilation > native-compilation (on lower spec ARM machine) >
> emulated native compilation (QEMU).
>

I'm on a beaten thinkpad (i5-3320m) and it takes ages to compile the
cross toolchain. It turned out that `(build-locally? #f)` is the fastest
way to deploy an ARM server.

Message #26 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Sergey Trofimov <sarg <at> sarg.org.ru>
Cc: ludo <at> gnu.org, Roman Scherer <roman.scherer <at> burningswell.com>,
 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Fri, 21 Mar 2025 11:12:27 +0900

Hi Sergey,

Sergey Trofimov <sarg <at> sarg.org.ru> writes:

[...]

>> Perhaps 'guix deploy' could have a '--cross-build' option, or the record
>> could have a 'cross-build?' field, or both, with the CLI taking
>> precedence over the later, and the target type would be inferred from
>> the system via platform-system->target?
>>
>
> Consider such case:
>
> deployer = x86_64, (aarch64 via binfmt)
> builder1 = x86_64, (aarch64 via binfmt)
> builder2 = aarch64, (x86_64 via binfmt)
> remote1 = x86_64
> remote2 = aarch64
>
> The user intends to run `guix deploy all-remotes.scm` on `deployer` to
> deploy both `remote1` and `remote2`.
>
> There are many ways how to build the derivations - deployer and builders
> could all build native/emulated/cross, remotes could build for
> themselves (`build-locally?` to `#f`). I am not sure to what extent this
> should be configurable, however here are a couple things I'd change:
> - add `platform` to `machine` record
> - remove `system` from `machine-ssh-configuration`
> - remove `build-locally?` from `machine-ssh-configuration`
> - add `build-options` to `machine`
>   - `build-host` = `local|offload|remote`
>   - `build-type` = `native|cross`

This seems reasonable; I suppose 'remote' means the machine being
deployed to?

`build-type` having just two values, it could be more simply expressed
as a boolean (e.g.; cross-build?)

> Additionally `guix-daemon` should know all the systems the host
> supports. Currently it knows only the native one (`--system` parameter).

I've never used such feature, so I wouldn't know of how useful that
would be :-).  Perhaps useful if the machine is to be used exclusively
for cross-compiling by default?  You could also set that via 'export
GUIX_BUILD_OPTIONS=--target=$gnu-triplet', no?

-- 
Thanks,
Maxim

Message #29 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Sergey Trofimov <sarg <at> sarg.org.ru>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: ludo <at> gnu.org, Roman Scherer <roman.scherer <at> burningswell.com>,
 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Fri, 21 Mar 2025 08:51:44 +0100

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:

> Hi Sergey,
>
> Sergey Trofimov <sarg <at> sarg.org.ru> writes:
>
> [...]
>
>>> Perhaps 'guix deploy' could have a '--cross-build' option, or the record
>>> could have a 'cross-build?' field, or both, with the CLI taking
>>> precedence over the later, and the target type would be inferred from
>>> the system via platform-system->target?
>>>
>>
>> Consider such case:
>>
>> deployer = x86_64, (aarch64 via binfmt)
>> builder1 = x86_64, (aarch64 via binfmt)
>> builder2 = aarch64, (x86_64 via binfmt)
>> remote1 = x86_64
>> remote2 = aarch64
>>
>> The user intends to run `guix deploy all-remotes.scm` on `deployer` to
>> deploy both `remote1` and `remote2`.
>>
>> There are many ways how to build the derivations - deployer and builders
>> could all build native/emulated/cross, remotes could build for
>> themselves (`build-locally?` to `#f`). I am not sure to what extent this
>> should be configurable, however here are a couple things I'd change:
>> - add `platform` to `machine` record
>> - remove `system` from `machine-ssh-configuration`
>> - remove `build-locally?` from `machine-ssh-configuration`
>> - add `build-options` to `machine`
>>   - `build-host` = `local|offload|remote`
>>   - `build-type` = `native|cross`
>
> This seems reasonable; I suppose 'remote' means the machine being
> deployed to?
>

That's right.

> `build-type` having just two values, it could be more simply expressed
> as a boolean (e.g.; cross-build?)
>

It's a matter of preference. Enums are extensible, bools are not.

>> Additionally `guix-daemon` should know all the systems the host
>> supports. Currently it knows only the native one (`--system` parameter).
>
> I've never used such feature, so I wouldn't know of how useful that
> would be :-).  Perhaps useful if the machine is to be used exclusively
> for cross-compiling by default?  You could also set that via 'export
> GUIX_BUILD_OPTIONS=--target=$gnu-triplet', no?

You've mentioned it in the previous mail:

> In my experience, assuming a powerful x86_64 workstation,
> cross-compilation > native-compilation (on lower spec ARM machine) >
> emulated native compilation (QEMU).

Cross compilation should be preferred when deploying from a powerful
machine to a smallest hetzner's ARM VM. The host is not used exclusively
for cross-building in such case.

Message #32 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Roman Scherer <roman.scherer <at> burningswell.com>
To: Sergey Trofimov <sarg <at> sarg.org.ru>
Cc: ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Fri, 21 Mar 2025 09:20:39 +0100

[Message part 1 (text/plain, inline)]

Hi Sergey,

I tried building libxcrypt on an aarch64 system with the following command:

```
guix build --system=x86_64-linux /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv -v 5
The following derivation will be built:
  /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv
building /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv...
qemu-x86_64: QEMU internal SIGSEGV {code=MAPERR, addr=0x20}
```

Not sure, why it's crashing. I haven't found the logs for it on my
system. The log file for the derivation is empty. I see this in my
guix-daemon.log file:

```
2025-03-21 09:17:08 SIGPOLL
2025-03-21 09:17:08 unexpected build daemon error: interrupted by the user
2025-03-21 09:17:10 accepted connection from pid 27118, user roman
```

Anything else I could try?

Roman

Sergey Trofimov <sarg <at> sarg.org.ru> writes:

> Hi Roman,
>
> Roman Scherer <roman.scherer <at> burningswell.com> writes:
>
>> So, I tried building this machine on an aarch64 system with the qemu-bin
>> service running now a couple of times ...
>>
>> ```
>> (use-modules (gnu machine)
>>              (gnu machine hetzner))
>>
>> (list (machine
>>        (operating-system %hetzner-os-x86)
>>        (environment hetzner-environment-type)
>>        (configuration (hetzner-configuration
>>                        (server-type "cx42")
>>                        (ssh-key "/home/roman/workspace/guix/id_rsa")))))
>> ```
>>
>> I used this command:
>>
>> ```
>> [roman <at> m1 guix]$ ./pre-inst-env guix deploy hetzner-minimal.scm --no-offload --target=x86_64-linux-gnu --timeout=864000 --max-silent-time=864000
>> The following 1 machine will be deployed:
>>   guix-x86
>>
>> guix deploy: deploying to guix-x86...
>> guix deploy: warning: <machine-ssh-configuration> without a 'host-key' is deprecated
>> substitute: looking for substitutes on 'https://substitutes.asahi-guix.org'... 100.0%
>> substitute: looking for substitutes on 'https://substitutes.nonguix.org'... 100.0%
>> substitute: looking for substitutes on 'https://bordeaux.guix.gnu.org'... 100.0%
>> substitute: looking for substitutes on 'https://ci.guix.gnu.org'... 100.0%
>> The following derivations will be built:
>>   /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv
>>   /gnu/store/8js0zmi3c2zg7ks3qnpiwsckam37vi67-guile-3.0.9.drv
>>
>> building /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv...
>> ```
>>
>> And it always "hangs" there. I haven't much investigated this yet. Just
>> a small update.
>>
>
> You could try `guix build
> /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv` to see
> what it does and where it hangs.
>
> Also try `--system=x86_64-linux`, it should be fast.
>
>>> I don't see a "system" field in the machine or operating system
>>> record. So I'm not sure if we have a way to infer that yet.
>>>
>>> Would it make sense to add such a field and chose the target
>>> architecture per machine?
>>>
>
> This totally makes sense to me and is the correct way. However I wonder
> if both the `system` and the `target` fields should be added, so that
> the user could choose either native or cross build. From my experience
> "native" build (as provided by qemu) is much faster. I'll try running
> a cross build tomorrow, but it is painfully slow to build the toolchain
> on my machine:
>
>
> --8<---------------cut here---------------start------------->8---
> The following 1 machine will be deployed:
>   guix-arm
>
> guix deploy: deploying to guix-arm, system=x86_64-linux, target=aarch64-linux-gnu...
> guix deploy: warning: <machine-ssh-configuration> without a 'host-key' is deprecated
> substitute: looking for substitutes on 'https://substitutes.nonguix.org'... 100.0%
> substitute: looking for substitutes on 'https://bordeaux.guix.gnu.org'... 100.0%
> substitute: looking for substitutes on 'https://ci.guix.gnu.org'... 100.0%
> The following derivations will be built:
>   /gnu/store/gxr8v1yisdiyndka0abxrc0xzrra66sv-binutils-cross-aarch64-linux-gnu-2.41.drv
>   /gnu/store/lch3711iiczn6smxsr7r3sj991p8avwv-ld-wrapper-aarch64-linux-gnu-0.drv
>   /gnu/store/zmsnlbyml0vmphfdxyxw4ps25bgrwz92-gcc-cross-sans-libc-aarch64-linux-gnu-14.2.0.drv
>   /gnu/store/57jnlmvqlvk6jkyvqcnrk4psffhmak91-linux-libre-headers-cross-aarch64-linux-gnu-5.15.49.drv
>   /gnu/store/b4f1my595ggl7d5qn46vr6qllwx7g49z-glibc-cross-aarch64-linux-gnu-2.39.drv
>   /gnu/store/sl5vfnwdarghf9ypbspq1bdlamnz3j2a-gcc-cross-aarch64-linux-gnu-14.2.0.drv
>   /gnu/store/3vp8a7mz1576xbk278k9b73nx2zqmzlw-libffi-3.4.4.drv
>   /gnu/store/y3hqwsbc8rb2g1mac8c9vsdmaacf20xm-libatomic-ops-7.6.12.drv
>   /gnu/store/bd09d178ni5sp9db62w869c6m7d3sh6v-libgc-8.2.4.drv
>   /gnu/store/cs7mzhrypgdad8v0v29arafc8brl7ynd-bash-minimal-5.1.16.drv
>   /gnu/store/np51g0ak713az6shj6sv9j3wkq4cjvjx-libunistring-1.1.drv
>   /gnu/store/rbkb4ig158h9gblbrah5nx5annvfpb4q-libxcrypt-4.4.36.drv
>   /gnu/store/lfmamfv5vx690l9n6a1ixbbk6kzw3gsr-guile-3.0.9.drv
> --8<---------------cut here---------------end--------------->8---

[signature.asc (application/pgp-signature, inline)]

Message #35 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Roman Scherer <roman.scherer <at> burningswell.com>
To: Sergey Trofimov <sarg <at> sarg.org.ru>
Cc: ludo <at> gnu.org, 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Fri, 21 Mar 2025 09:24:49 +0100

[Message part 1 (text/plain, inline)]

Hi Sergey,

Sergey Trofimov <sarg <at> sarg.org.ru> writes:

> Roman Scherer <roman.scherer <at> burningswell.com> writes:
>
>> Hello Sergey,
>>
>> this looks good to me and is better than what we have now. I'm testing
>> this right now and will report back when it went through ...
>>
>
> I think this patch is not necessary at all, your code already sets the
> `system` field of `machine-ssh-configuration` to the value appropriate
> to one returned by hetzner-api.

Oops, looks like I forgot about this. I was looking at this at the
machine and/or OS level.

> It looks to me that the bug is in incorrect parameterization, see the
> following patch. I've been able to deploy using it (again
> `build-locally?` set to #f as it takes too long otherwise). Please test
> this one.

I give it a try and will report back.

>
> --8<---------------cut here---------------start------------->8---
> From 988d192cfe4833761d0714aa8a53392c09dd5c9d Mon Sep 17 00:00:00 2001
> Message-ID: <988d192cfe4833761d0714aa8a53392c09dd5c9d.1742331137.git.sarg <at> sarg.org.ru>
> From: Sergey Trofimov <sarg <at> sarg.org.ru>
> Subject: [PATCH] machine: ssh: Use monadic parameterize for monads.
>
> * gnu/machine/ssh (deploy-managed-host): Use store-parameterize in
> monadic context.
>
> Change-Id: I6441a5dac856633f2c2a00b77f21a385bf3a9510
> ---
>  gnu/machine/ssh.scm | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm
> index 73d5dc513e..0ce2860236 100644
> --- a/gnu/machine/ssh.scm
> +++ b/gnu/machine/ssh.scm
> @@ -542,8 +542,8 @@ (define (deploy-managed-host machine)
>                                          host
>                                          (inferior-exception-arguments c))))
>                                os))
> -          (parameterize ((%current-system system)
> -                         (%current-target-system #f))
> +          (store-parameterize ((%current-system system)
> +                               (%current-target-system #f))
>              (with-roll-back #t
>                (mbegin %store-monad
>                  (upgrade-shepherd-services (eval/error-handling c
>
> base-commit: df799a61ef73451e587cb81942c7bfcbbade9e7e

[signature.asc (application/pgp-signature, inline)]

Message #38 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Roman Scherer <roman.scherer <at> burningswell.com>
To: Sergey Trofimov <sarg <at> sarg.org.ru>
Cc: ludo <at> gnu.org, 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Fri, 21 Mar 2025 10:15:51 +0100

[Message part 1 (text/plain, inline)]

Hi Sergey,

I just tried this patch with the following machine config:

```
(use-modules (gnu machine)
             (gnu machine hetzner))

(list
 (machine
  (operating-system %hetzner-os-x86)
  (environment hetzner-environment-type)
  (configuration (hetzner-configuration
                  (server-type "cpx41")
                  (ssh-key "/home/roman/workspace/guix/id_rsa")))))
```

I deployed it with the following command:

```
./pre-inst-env guix deploy hetzner-minimal.scm --no-offload --timeout=864000 --max-silent-time=864000
```

The server gets created, the Guix bootstrap system is installed on it,
but when it tries to apply the final operating system config it "hangs"
at applying grafts:

```
...
guix system: bootloader successfully installed on /dev/sda
successfully installed guix operating system on 'guix-x86'
rebooting server for 'guix-x86'...
successfully rebooted server for 'guix-x86'
connecting via SSH to '138.201.175.151' using '/home/roman/workspace/guix/id_rsa'...
SSH connection to '138.201.175.151' port 22 failed: Timeout connecting to 138.201.175.151
guix deploy: warning: <machine-ssh-configuration> without a 'host-key' is deprecated
substitute: looking for substitutes on 'https://substitutes.asahi-guix.org'... 100.0%
substitute: looking for substitutes on 'https://substitutes.nonguix.org'... 100.0%
substitute: looking for substitutes on 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: looking for substitutes on 'https://ci.guix.gnu.org'... 100.0%
The following derivations will be built:
  /gnu/store/f3s4as9imjy6mbpgq8796q64hm82ykaf-remote-assertion.scm.drv
  /gnu/store/14cba7yngalx7375gdh5325ba5ss1igh-module-import-compiled.drv
  /gnu/store/ipqcb48n8jhc11y7b1pbxaf4nxdv27iv-remote-assertion.scm.drv
  /gnu/store/q49sc3kk2f3430dmcsd4lbqgran1w8il-remote-assertion.scm.drv
  /gnu/store/8wdvxf30pn3m0i7six8s5v02iw0h8hl4-remote-exp.scm.drv

applying 2 grafts for bash-minimal-5.1.16 ...
-
```

SSHing into the machine doesn't show much activity. The "-" on the last line
is also not spinning, so I'm wondering if it actually does something or if it
waits on something. The "-" not spinning I also observed when I did the cross
build with your other patch.

Applying grafts doesn't take so long in what I have observed.

Deploying this arm machine from an aarch64 system works with your patch though.

```
(list (machine
       (operating-system %hetzner-os-arm)
       (environment hetzner-environment-type)
       (configuration (hetzner-configuration
                       (server-type "cax21")
                       (ssh-key "/home/roman/workspace/guix/id_rsa")))))
```

Any ideas why it is hanging at applying the grafts?

Roman

Sergey Trofimov <sarg <at> sarg.org.ru> writes:

> Roman Scherer <roman.scherer <at> burningswell.com> writes:
>
>> Hello Sergey,
>>
>> this looks good to me and is better than what we have now. I'm testing
>> this right now and will report back when it went through ...
>>
>
> I think this patch is not necessary at all, your code already sets the
> `system` field of `machine-ssh-configuration` to the value appropriate
> to one returned by hetzner-api.
>
> It looks to me that the bug is in incorrect parameterization, see the
> following patch. I've been able to deploy using it (again
> `build-locally?` set to #f as it takes too long otherwise). Please test
> this one.
>
> --8<---------------cut here---------------start------------->8---
> From 988d192cfe4833761d0714aa8a53392c09dd5c9d Mon Sep 17 00:00:00 2001
> Message-ID: <988d192cfe4833761d0714aa8a53392c09dd5c9d.1742331137.git.sarg <at> sarg.org.ru>
> From: Sergey Trofimov <sarg <at> sarg.org.ru>
> Subject: [PATCH] machine: ssh: Use monadic parameterize for monads.
>
> * gnu/machine/ssh (deploy-managed-host): Use store-parameterize in
> monadic context.
>
> Change-Id: I6441a5dac856633f2c2a00b77f21a385bf3a9510
> ---
>  gnu/machine/ssh.scm | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm
> index 73d5dc513e..0ce2860236 100644
> --- a/gnu/machine/ssh.scm
> +++ b/gnu/machine/ssh.scm
> @@ -542,8 +542,8 @@ (define (deploy-managed-host machine)
>                                          host
>                                          (inferior-exception-arguments c))))
>                                os))
> -          (parameterize ((%current-system system)
> -                         (%current-target-system #f))
> +          (store-parameterize ((%current-system system)
> +                               (%current-target-system #f))
>              (with-roll-back #t
>                (mbegin %store-monad
>                  (upgrade-shepherd-services (eval/error-handling c
>
> base-commit: df799a61ef73451e587cb81942c7bfcbbade9e7e

[signature.asc (application/pgp-signature, inline)]

Message #41 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Roman Scherer <roman.scherer <at> burningswell.com>
Cc: ludo <at> gnu.org, Sergey Trofimov <sarg <at> sarg.org.ru>, 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Sat, 22 Mar 2025 16:39:44 +0900

Hi Roman,

Roman Scherer <roman.scherer <at> burningswell.com> writes:

> Hi Sergey,
>
> I tried building libxcrypt on an aarch64 system with the following command:
>
> ```
> guix build --system=x86_64-linux /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv -v 5
> The following derivation will be built:
>   /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv
> building /gnu/store/0ma7qrdd5pz8421rka75mvmvhkvn1rld-libxcrypt-4.4.36.drv...
> qemu-x86_64: QEMU internal SIGSEGV {code=MAPERR, addr=0x20}
> ```

That looks like a QEMU bug.  If you'd like it resolved, I'd open the
issue with them, at <https://gitlab.com/qemu-project/qemu/-/issues>.

> Not sure, why it's crashing. I haven't found the logs for it on my
> system. The log file for the derivation is empty. I see this in my
> guix-daemon.log file:
>
> ```
> 2025-03-21 09:17:08 SIGPOLL
> 2025-03-21 09:17:08 unexpected build daemon error: interrupted by the user
> 2025-03-21 09:17:10 accepted connection from pid 27118, user roman
> ```
>
> Anything else I could try?

An actual x86_64 machine to use as an offload machine, if you have
access to one :-).  You could also try '--target=x86_64-linux-gnu'
instead of '--system', to attempt cross-compiling it for x86_64.  If
supported by the involved build systems in Guix, that should be more
reliable than QEMU.

-- 
Thanks,
Maxim

Message #44 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Sergey Trofimov <sarg <at> sarg.org.ru>
Cc: ludo <at> gnu.org, Roman Scherer <roman.scherer <at> burningswell.com>,
 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Sat, 22 Mar 2025 16:42:25 +0900

Hi Sergey,

Sergey Trofimov <sarg <at> sarg.org.ru> writes:

[...]

>>> Additionally `guix-daemon` should know all the systems the host
>>> supports. Currently it knows only the native one (`--system`
>>> parameter).
>>
>> I've never used such feature, so I wouldn't know of how useful that
>> would be :-).  Perhaps useful if the machine is to be used
>> exclusively
>> for cross-compiling by default?  You could also set that via 'export
>> GUIX_BUILD_OPTIONS=--target=$gnu-triplet', no?
>
> You've mentioned it in the previous mail:
>
>> In my experience, assuming a powerful x86_64 workstation,
>> cross-compilation > native-compilation (on lower spec ARM machine) >
>> emulated native compilation (QEMU).

I was answering the specific implementation details you seemed to allude
to, regarding that guix-daemon should know also about the non-native
(cross-compiled) target systems supported.

I didn't understand that part.  Why would we need to teach the daemon
something new?  It already can cross-compile things as it is, even it
lacks such explicit knowledge, it seems.

-- 
Thanks,
Maxim

Message #47 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Sergey Trofimov <sarg <at> sarg.org.ru>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: ludo <at> gnu.org, Roman Scherer <roman.scherer <at> burningswell.com>,
 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Sat, 22 Mar 2025 10:09:29 +0100

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:

> [...]
>
>>>> Additionally `guix-daemon` should know all the systems the host
>>>> supports. Currently it knows only the native one (`--system`
>>>> parameter).
>>>
>>> I've never used such feature, so I wouldn't know of how useful that
>>> would be :-).  Perhaps useful if the machine is to be used
>>> exclusively
>>> for cross-compiling by default?  You could also set that via 'export
>>> GUIX_BUILD_OPTIONS=--target=$gnu-triplet', no?
>>
>> You've mentioned it in the previous mail:
>>
>>> In my experience, assuming a powerful x86_64 workstation,
>>> cross-compilation > native-compilation (on lower spec ARM machine) >
>>> emulated native compilation (QEMU).
>
> I was answering the specific implementation details you seemed to allude
> to, regarding that guix-daemon should know also about the non-native
> (cross-compiled) target systems supported.
>
> I didn't understand that part.  Why would we need to teach the daemon
> something new?  It already can cross-compile things as it is, even it
> lacks such explicit knowledge, it seems.

`build-machine` declares the systems it supports and local host is in
fact also a build machine. Having explicit supported system declaration
would:
1. allow better error messages ("can't build for `targetX`" instead of
failing to build actual derivations)
2. provide more data to the offloading algorithm. It could then select
the current host when necessary
3. potentially allow to get rid of the `--system` build parameter (or
replace it with `--build-type=native|emulated|cross`).

Message #52 received at 77033 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Sergey Trofimov <sarg <at> sarg.org.ru>
Cc: ludo <at> gnu.org, Roman Scherer <roman.scherer <at> burningswell.com>,
 77033 <at> debbugs.gnu.org
Subject: Re: [PATCH] deploy: Support --target and --system.
Date: Sat, 22 Mar 2025 21:32:30 +0900

Hi Sergey,

Sergey Trofimov <sarg <at> sarg.org.ru> writes:

> Hi Maxim,
>
> Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:
>
>> [...]
>>
>>>>> Additionally `guix-daemon` should know all the systems the host
>>>>> supports. Currently it knows only the native one (`--system`
>>>>> parameter).
>>>>
>>>> I've never used such feature, so I wouldn't know of how useful that
>>>> would be :-).  Perhaps useful if the machine is to be used
>>>> exclusively
>>>> for cross-compiling by default?  You could also set that via 'export
>>>> GUIX_BUILD_OPTIONS=--target=$gnu-triplet', no?
>>>
>>> You've mentioned it in the previous mail:
>>>
>>>> In my experience, assuming a powerful x86_64 workstation,
>>>> cross-compilation > native-compilation (on lower spec ARM machine) >
>>>> emulated native compilation (QEMU).
>>
>> I was answering the specific implementation details you seemed to allude
>> to, regarding that guix-daemon should know also about the non-native
>> (cross-compiled) target systems supported.
>>
>> I didn't understand that part.  Why would we need to teach the daemon
>> something new?  It already can cross-compile things as it is, even it
>> lacks such explicit knowledge, it seems.
>
> `build-machine` declares the systems it supports and local host is in
> fact also a build machine. Having explicit supported system declaration
> would:
> 1. allow better error messages ("can't build for `targetX`" instead of
> failing to build actual derivations)
>
> 2. provide more data to the offloading algorithm. It could then select
> the current host when necessary

I think one way to tackle both points above could be to allow defining
localhost as a build machine in /etc/guix/machines.scm (perhaps there
could be an option to make this implicit/automatic), though we would
need to special case it so that ssh is not used to communicate with it.

> 3. potentially allow to get rid of the `--system` build parameter (or
> replace it with `--build-type=native|emulated|cross`).

I'm not sure about this one, or at least I don't understand how that'd
capture all the necessary information.  Perhaps you are considering only
in the narrow context of 'guix deploy' ?  I think there's value that it
works the same everywhere.

-- 
Thanks,
Maxim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.