GNU bug report logs - #43650
[PATCH 0/8] Assorted childhurd improvements

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 43650 in the body.
You can then email your comments to 43650 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>, janneke <at> gnu.org
Subject: [PATCH 0/8] Assorted childhurd improvements
Date: Sun, 27 Sep 2020 17:29:32 +0200

Hello Guix!

Here are assorted improvements to childhurds!

There’s one thing missing to allow ‘hurd-vm-service-type’ to
automatically enable offloading to the local childhurd:
declarative ACL and declarative machines.scm.

Feedback welcome!  :-)

Ludo’.

PS: It’s GNU’s 37th birthday! \o/

Ludovic Courtès (8):
  services: hurd-vm: Run QEMU as an unprivileged user.
  services: childhurd: Tweak description.
  secret-service: Clarify the origin of messages.
  services: hurd-vm: Check whether /dev/kvm exists at run time.
  services: guix: Generate key pair if needed during activation.
  services: hurd-vm: Initialize the guest's SSH/Guix keys at activation
    time.
  services: hurd-vm: Pass "-no-reboot" when spawning the Hurd VM.
  secret-service: Add a timeout when waiting for a client.

 doc/guix.texi                   |  44 +++++++++--
 gnu/build/secret-service.scm    |  48 +++++++-----
 gnu/services/base.scm           |  13 +++-
 gnu/services/virtualization.scm | 131 +++++++++++++++++++++++++++-----
 4 files changed, 187 insertions(+), 49 deletions(-)

-- 
2.28.0

Message #8 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 43650 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>, janneke <at> gnu.org
Subject: [PATCH 1/8] services: hurd-vm: Run QEMU as an unprivileged user.
Date: Sun, 27 Sep 2020 17:32:14 +0200

Until qemu was running as "root", which is unnecessary.

* gnu/services/virtualization.scm (%hurd-vm-accounts): New variable.
(hurd-vm-service-type)[extensions]: Add ACCOUNT-SERVICE-TYPE extension.
---
 gnu/services/virtualization.scm | 43 +++++++++++++++++++++++----------
 1 file changed, 30 insertions(+), 13 deletions(-)

diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 20e104f48c..55a19d7af9 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -959,28 +959,45 @@ is added to the OS specified in CONFIG."
        (with-imported-modules
            (source-module-closure '((gnu build secret-service)
                                     (guix build utils)))
-         #~(let ((spawn (make-forkexec-constructor #$vm-command)))
-             (lambda _
-               (let ((pid (spawn))
-                     (port #$(hurd-vm-port config %hurd-vm-secrets-port))
-                     (root #$(hurd-vm-configuration-secret-root config)))
-                 (catch #t
-                   (lambda _
-                     (secret-service-send-secrets port root))
-                   (lambda (key . args)
-                     (kill (- pid) SIGTERM)
-                     (apply throw key args)))
-                 pid)))))
+         #~(lambda ()
+             (let ((pid  (fork+exec-command #$vm-command
+                                            #:user "childhurd"
+                                            #:group "childhurd"
+                                            #:environment-variables
+                                            ;; QEMU tries to write to /var/tmp
+                                            ;; by default.
+                                            '("TMPDIR=/tmp")))
+                   (port #$(hurd-vm-port config %hurd-vm-secrets-port))
+                   (root #$(hurd-vm-configuration-secret-root config)))
+               (catch #t
+                 (lambda _
+                   (secret-service-send-secrets port root)
+                   pid)
+                 (lambda (key . args)
+                   (kill (- pid) SIGTERM)
+                   (apply throw key args)))))))
       (modules `((gnu build secret-service)
                  (guix build utils)
                  ,@%default-modules))
       (stop  #~(make-kill-destructor))))))
 
+(define %hurd-vm-accounts
+  (list (user-group (name "childhurd") (system? #t))
+        (user-account
+         (name "childhurd")
+         (group "childhurd")
+         (comment "Privilege separation user for the childhurd")
+         (home-directory "/var/empty")
+         (shell (file-append shadow "/sbin/nologin"))
+         (system? #t))))
+
 (define hurd-vm-service-type
   (service-type
    (name 'hurd-vm)
    (extensions (list (service-extension shepherd-root-service-type
-                                        hurd-vm-shepherd-service)))
+                                        hurd-vm-shepherd-service)
+                     (service-extension account-service-type
+                                        (const %hurd-vm-accounts))))
    (default-value (hurd-vm-configuration))
    (description
     "Provide a Virtual Machine running the GNU/Hurd.")))
-- 
2.28.0

Message #11 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 43650 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>, janneke <at> gnu.org
Subject: [PATCH 2/8] services: childhurd: Tweak description.
Date: Sun, 27 Sep 2020 17:32:15 +0200

* gnu/services/virtualization.scm (hurd-vm-service-type)[description]:
Mention "childhurd".
---
 gnu/services/virtualization.scm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 55a19d7af9..d184eea746 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -1000,4 +1000,5 @@ is added to the OS specified in CONFIG."
                                         (const %hurd-vm-accounts))))
    (default-value (hurd-vm-configuration))
    (description
-    "Provide a Virtual Machine running the GNU/Hurd.")))
+    "Provide a virtual machine (VM) running GNU/Hurd, also known as a
+@dfn{childhurd}.")))
-- 
2.28.0

Message #14 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 43650 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>, janneke <at> gnu.org
Subject: [PATCH 3/8] secret-service: Clarify the origin of messages.
Date: Sun, 27 Sep 2020 17:32:16 +0200

* gnu/build/secret-service.scm (secret-service-send-secrets)
(secret-service-receive-secrets): Prefix messages by "secret service".
---
 gnu/build/secret-service.scm | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/gnu/build/secret-service.scm b/gnu/build/secret-service.scm
index 781651e90d..aafb1684b5 100644
--- a/gnu/build/secret-service.scm
+++ b/gnu/build/secret-service.scm
@@ -54,11 +54,14 @@ local PORT.  If connect fails, sleep 1s and retry RETRY times."
         (lambda (key . args)
           (when (zero? retry)
             (apply throw key args))
-          (format (current-error-port) "retrying connection~%")
+          (format (current-error-port)
+                  "secret service: retrying connection [~a attempts left]~%"
+                  (- retry 1))
           (sleep 1)
           (loop (1- retry)))))
 
-    (format (current-error-port) "connected!  sending files in ~s %~"
+    (format (current-error-port)
+            "secret service: connected; sending files in ~s~%"
             secret-root)
     (let* ((files (if secret-root (find-files secret-root) '()))
            (files-sizes-modes (map file->file+size+mode files))
@@ -82,11 +85,12 @@ Write them to the file system."
       (bind sock AF_INET INADDR_ANY port)
       (listen sock 1)
       (format (current-error-port)
-              "waiting for secrets on port ~a...~%"
+              "secret service: waiting for secrets on port ~a...~%"
               port)
       (match (accept sock)
         ((client . address)
-         (format (current-error-port) "client connection from ~a~%"
+         (format (current-error-port)
+                 "secret service: client connection from ~a~%"
                  (inet-ntop (sockaddr:fam address)
                             (sockaddr:addr address)))
          (close-port sock)
@@ -116,7 +120,8 @@ Write them to the file system."
                  ('files ((files sizes modes) ...)))
        (for-each (lambda (file size mode)
                    (format (current-error-port)
-                           "installing file '~a' (~a bytes)...~%"
+                           "secret service: \
+installing file '~a' (~a bytes)...~%"
                            file size)
                    (mkdir-p (dirname file))
                    (call-with-output-file file
@@ -126,7 +131,7 @@ Write them to the file system."
                  files sizes modes))
       (_
        (format (current-error-port)
-               "invalid secrets received~%")
+               "secret service: invalid secrets received~%")
        #f)))
 
   (let* ((port (wait-for-client port))
-- 
2.28.0

Message #17 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 43650 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>, janneke <at> gnu.org
Subject: [PATCH 4/8] services: hurd-vm: Check whether /dev/kvm exists at run
 time.
Date: Sun, 27 Sep 2020 17:32:17 +0200

This change allows a childhurd to run within Guix System in a VM.

* gnu/services/virtualization.scm (hurd-vm-shepherd-service)[vm-command]:
Stage the 'file-exists?' call.
---
 gnu/services/virtualization.scm | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index d184eea746..b84203ad18 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -937,13 +937,14 @@ is added to the OS specified in CONFIG."
         (provisions  '(hurd-vm childhurd)))
 
     (define vm-command
-      #~(list
-         (string-append #$qemu "/bin/qemu-system-i386")
-         #$@(if (file-exists? "/dev/kvm") '("--enable-kvm") '())
-         "-m" (number->string #$memory-size)
-         #$@net-options
-         #$@options
-         "--hda" #+image))
+      #~(append (list #$(file-append qemu "/bin/qemu-system-i386")
+                      "-m" (number->string #$memory-size)
+                      #$@net-options
+                      #$@options
+                      "--hda" #+image)
+                (if (file-exists? "/dev/kvm")
+                    '("--enable-kvm")
+                    '())))
 
     (list
      (shepherd-service
-- 
2.28.0

Message #20 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 43650 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>, janneke <at> gnu.org
Subject: [PATCH 5/8] services: guix: Generate key pair if needed during
 activation.
Date: Sun, 27 Sep 2020 17:32:18 +0200

* gnu/services/base.scm (guix-activation): Invoke "guix archive
--generate-key".
* doc/guix.texi (Invoking guix archive)
(Invoking guix deploy): Mention that 'guix-service-type' takes care of
generating the key pair.
---
 doc/guix.texi         | 11 +++++++----
 gnu/services/base.scm | 13 +++++++++----
 2 files changed, 16 insertions(+), 8 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 82241b010a..885f7fcf97 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5048,9 +5048,11 @@ the store.
 @item --generate-key[=@var{parameters}]
 @cindex signing, archives
 Generate a new key pair for the daemon.  This is a prerequisite before
-archives can be exported with @option{--export}.  Note that this
-operation usually takes time, because it needs to gather enough entropy
-to generate the key pair.
+archives can be exported with @option{--export}.  This
+operation is usually instantaneous but it can take time if the system's
+entropy pool needs to be refilled.  On Guix System,
+@code{guix-service-type} takes care of generating this key pair the
+first boot.
 
 The generated key pair is typically stored under @file{/etc/guix}, in
 @file{signing-key.pub} (public key) and @file{signing-key.sec} (private
@@ -29531,7 +29533,8 @@ a Virtual Private Server (VPS) provider.  In such a case, a different
 
 Do note that you first need to generate a key pair on the coordinator machine
 to allow the daemon to export signed archives of files from the store
-(@pxref{Invoking guix archive}).
+(@pxref{Invoking guix archive}), though this step is automatic on Guix
+System:
 
 @example
 # guix archive --generate-key
diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index bef4eef241..04bc991356 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -1653,10 +1653,15 @@ proxy of 'guix-daemon'...~%")
      ;; otherwise call 'chown' here, but the problem is that on a COW overlayfs,
      ;; chown leads to an entire copy of the tree, which is a bad idea.
 
-     ;; Optionally authorize substitute server keys.
-     (if authorize-key?
-         (substitute-key-authorization keys guix)
-         #~#f))))
+     ;; Generate a key pair and optionally authorize substitute server keys.
+     #~(begin
+         (unless (file-exists? "/etc/guix/signing-key.pub")
+           (system* #$(file-append guix "/bin/guix") "archive"
+                    "--generate-key"))
+
+         #$(if authorize-key?
+               (substitute-key-authorization keys guix)
+               #~#f)))))
 
 (define* (references-file item #:optional (name "references"))
   "Return a file that contains the list of references of ITEM."
-- 
2.28.0

Message #23 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 43650 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>, janneke <at> gnu.org
Subject: [PATCH 6/8] services: hurd-vm: Initialize the guest's SSH/Guix keys
 at activation time.
Date: Sun, 27 Sep 2020 17:32:19 +0200

* gnu/services/virtualization.scm (initialize-hurd-vm-substitutes)
(hurd-vm-activation): New procedures.
(hurd-vm-service-type)[extensions]: Add ACTIVATION-SERVICE-TYPE
extension.
* doc/guix.texi (Transparent Emulation with QEMU): Mention GNU/Hurd.
(The Hurd in a Virtual Machine): Explain which files are automatically
installed and mention offloading.
---
 doc/guix.texi                   | 33 ++++++++++++++--
 gnu/services/virtualization.scm | 67 ++++++++++++++++++++++++++++++++-
 2 files changed, 96 insertions(+), 4 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 885f7fcf97..851afe843d 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -25342,6 +25342,8 @@ emulation of program binaries built for different architectures---e.g.,
 it allows you to transparently execute an ARMv7 program on an x86_64
 machine.  It achieves this by combining the @uref{https://www.qemu.org,
 QEMU} emulator and the @code{binfmt_misc} feature of the kernel Linux.
+This feature only allows you to emulate GNU/Linux on a different
+architecture, but see below for GNU/Hurd support.
 
 @defvr {Scheme Variable} qemu-binfmt-service-type
 This is the type of the QEMU/binfmt service for transparent emulation.
@@ -25544,10 +25546,11 @@ If the @file{/etc/childhurd} directory does not exist, the
 @code{secret-service} running in the Childhurd will be sent an empty
 list of secrets.
 
-Typical use to populate @file{"/etc/childhurd"} with a tree of
-non-volatile secrets, like so
+By default, the service automatically populates @file{/etc/childhurd}
+with the following non-volatile secrets, unless they already exist:
 
 @example
+/etc/childhurd/etc/guix/acl
 /etc/childhurd/etc/guix/signing-key.pub
 /etc/childhurd/etc/guix/signing-key.sec
 /etc/childhurd/etc/ssh/ssh_host_ed25519_key
@@ -25556,8 +25559,32 @@ non-volatile secrets, like so
 /etc/childhurd/etc/ssh/ssh_host_ecdsa_key.pub
 @end example
 
-to be sent to the Childhurd, including permissions.
+These files are automatically sent to the guest Hurd VM when it boots,
+including permissions.
 
+@cindex childhurd, offloading
+@cindex Hurd, offloading
+Having these files in place means that only a couple of things are
+missing to allow the host to offload @code{i586-gnu} builds to the
+childhurd:
+
+@enumerate
+@item
+Authorizing the childhurd's key on the host so that the host accepts
+build results coming from the childhurd, which can be done like so:
+
+@example
+guix archive --authorize < \
+  /etc/childhurd/etc/guix/signing-key.pub
+@end example
+
+@item
+Adding the childhurd to @file{/etc/guix/machines.scm} (@pxref{Daemon
+Offload Setup}).
+@end enumerate
+
+We're working towards making that happen automatically---get in touch
+with us at @email{guix-devel@@gnu.org} to discuss it!
 @end table
 @end deftp
 
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index b84203ad18..c639fa3741 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -23,6 +23,7 @@
   #:use-module (gnu bootloader grub)
   #:use-module (gnu image)
   #:use-module (gnu packages admin)
+  #:use-module (gnu packages package-management)
   #:use-module (gnu packages ssh)
   #:use-module (gnu packages virtualization)
   #:use-module (gnu services base)
@@ -992,13 +993,77 @@ is added to the OS specified in CONFIG."
          (shell (file-append shadow "/sbin/nologin"))
          (system? #t))))
 
+(define (initialize-hurd-vm-substitutes)
+  "Initialize the Hurd VM's key pair and ACL and store it on the host."
+  (define run
+    (with-imported-modules '((guix build utils))
+      #~(begin
+          (use-modules (guix build utils)
+                       (ice-9 match))
+
+          (define host-key
+            "/etc/guix/signing-key.pub")
+
+          (define host-acl
+            "/etc/guix/acl")
+
+          (match (command-line)
+            ((_ guest-config-directory)
+             (setenv "GUIX_CONFIGURATION_DIRECTORY"
+                     guest-config-directory)
+             (invoke #+(file-append guix "/bin/guix") "archive"
+                     "--generate-key")
+
+             (when (file-exists? host-acl)
+               ;; Copy the host ACL.
+               (copy-file host-acl
+                          (string-append guest-config-directory
+                                         "/acl")))
+
+             (when (file-exists? host-key)
+               ;; Add the host key to the childhurd's ACL.
+               (let ((key (open-fdes host-key O_RDONLY)))
+                 (close-fdes 0)
+                 (dup2 key 0)
+                 (execl #+(file-append guix "/bin/guix")
+                        "guix" "archive" "--authorize"))))))))
+
+  (program-file "initialize-hurd-vm-substitutes" run))
+
+(define (hurd-vm-activation config)
+  "Return a gexp to activate the Hurd VM according to CONFIG."
+  (with-imported-modules '((guix build utils))
+    #~(begin
+        (use-modules (guix build utils))
+
+        (define secret-directory
+          #$(hurd-vm-configuration-secret-root config))
+
+        (define ssh-directory
+          (string-append secret-directory "/etc/ssh"))
+
+        (define guix-directory
+          (string-append secret-directory "/etc/guix"))
+
+        (unless (file-exists? ssh-directory)
+          ;; Generate SSH host keys under SSH-DIRECTORY.
+          (mkdir-p ssh-directory)
+          (invoke #$(file-append openssh "/bin/ssh-keygen")
+                  "-A" "-f" secret-directory))
+
+        (unless (file-exists? guix-directory)
+          (invoke #$(initialize-hurd-vm-substitutes)
+                  guix-directory)))))
+
 (define hurd-vm-service-type
   (service-type
    (name 'hurd-vm)
    (extensions (list (service-extension shepherd-root-service-type
                                         hurd-vm-shepherd-service)
                      (service-extension account-service-type
-                                        (const %hurd-vm-accounts))))
+                                        (const %hurd-vm-accounts))
+                     (service-extension activation-service-type
+                                        hurd-vm-activation)))
    (default-value (hurd-vm-configuration))
    (description
     "Provide a virtual machine (VM) running GNU/Hurd, also known as a
-- 
2.28.0

Message #26 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 43650 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>, janneke <at> gnu.org
Subject: [PATCH 7/8] services: hurd-vm: Pass "-no-reboot" when spawning the
 Hurd VM.
Date: Sun, 27 Sep 2020 17:32:20 +0200

* gnu/services/virtualization.scm (hurd-vm-shepherd-service)[vm-command]:
Add "--no-reboot".
---
 gnu/services/virtualization.scm | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index c639fa3741..a50cf8b733 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -942,7 +942,12 @@ is added to the OS specified in CONFIG."
                       "-m" (number->string #$memory-size)
                       #$@net-options
                       #$@options
-                      "--hda" #+image)
+                      "--hda" #+image
+
+                      ;; Cause the service to be respawned if the guest
+                      ;; reboots (it can reboot for instance if it did not
+                      ;; receive valid secrets, or if it crashed.)
+                      "--no-reboot")
                 (if (file-exists? "/dev/kvm")
                     '("--enable-kvm")
                     '())))
-- 
2.28.0

Message #29 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 43650 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>, janneke <at> gnu.org
Subject: [PATCH 8/8] secret-service: Add a timeout when waiting for a client.
Date: Sun, 27 Sep 2020 17:32:21 +0200

* gnu/build/secret-service.scm (secret-service-receive-secrets)
[wait-for-client]: Call 'select' with a 60s timeout before 'accept'.
Return #f upon timeout.
[read-secrets]: Return FILES on success.
Adjust caller of 'wait-for-client' to handle #f.
---
 gnu/build/secret-service.scm | 33 +++++++++++++++++++++------------
 1 file changed, 21 insertions(+), 12 deletions(-)

diff --git a/gnu/build/secret-service.scm b/gnu/build/secret-service.scm
index aafb1684b5..40c24abf09 100644
--- a/gnu/build/secret-service.scm
+++ b/gnu/build/secret-service.scm
@@ -75,7 +75,8 @@ local PORT.  If connect fails, sleep 1s and retry RETRY times."
 
 (define (secret-service-receive-secrets port)
   "Listen to local PORT and wait for a secret service client to send secrets.
-Write them to the file system."
+Write them to the file system.  Return the list of files installed on success,
+and #f otherwise."
 
   (define (wait-for-client port)
     ;; Wait for a TCP connection on PORT.  Note: We cannot use the
@@ -87,14 +88,20 @@ Write them to the file system."
       (format (current-error-port)
               "secret service: waiting for secrets on port ~a...~%"
               port)
-      (match (accept sock)
-        ((client . address)
+      (match (select (list sock) '() '() 60)
+        (((_) () ())
+         (match (accept sock)
+           ((client . address)
+            (format (current-error-port)
+                    "secret service: client connection from ~a~%"
+                    (inet-ntop (sockaddr:fam address)
+                               (sockaddr:addr address)))
+            (close-port sock)
+            client)))
+        ((() () ())
          (format (current-error-port)
-                 "secret service: client connection from ~a~%"
-                 (inet-ntop (sockaddr:fam address)
-                            (sockaddr:addr address)))
-         (close-port sock)
-         client))))
+                 "secret service: did not receive any secrets; time out~%")
+         #f))))
 
   ;; TODO: Remove when (@ (guix build utils) dump-port) has a 'size'
   ;; parameter.
@@ -128,15 +135,17 @@ installing file '~a' (~a bytes)...~%"
                      (lambda (output)
                        (dump port output size)
                        (chmod file mode))))
-                 files sizes modes))
+                 files sizes modes)
+       files)
       (_
        (format (current-error-port)
                "secret service: invalid secrets received~%")
        #f)))
 
-  (let* ((port (wait-for-client port))
-         (result (read-secrets port)))
-    (close-port port)
+  (let* ((port   (wait-for-client port))
+         (result (and=> port read-secrets)))
+    (when port
+      (close-port port))
     result))
 
 ;;; secret-service.scm ends here
-- 
2.28.0

Message #32 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 43650 <at> debbugs.gnu.org
Subject: Re: [PATCH 1/8] services: hurd-vm: Run QEMU as an unprivileged user.
Date: Mon, 28 Sep 2020 18:57:00 +0200

Ludovic Courtès writes:

Hello!

> Until qemu was running as "root", which is unnecessary.

Well...I can't get this to work; my childhurd does not run.  Did you
test it?  Any special tricks needed, adding "childhurd" to "kvm" maybe?

I do like the idea...

Greetings,
Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #35 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 43650 <at> debbugs.gnu.org
Subject: Re: [PATCH 4/8] services: hurd-vm: Check whether /dev/kvm exists at
 run time.
Date: Mon, 28 Sep 2020 19:02:02 +0200

[Message part 1 (text/plain, inline)]

Ludovic Courtès writes:

Hi!

> This change allows a childhurd to run within Guix System in a VM.

Ah, this

> * gnu/services/virtualization.scm (hurd-vm-shepherd-service)[vm-command]:
> Stage the 'file-exists?' call.
> ---
>  gnu/services/virtualization.scm | 15 ++++++++-------
>  1 file changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
> index d184eea746..b84203ad18 100644
[..]
> -      #~(list
> -         (string-append #$qemu "/bin/qemu-system-i386")
> -         #$@(if (file-exists? "/dev/kvm") '("--enable-kvm") '())

ungexp'ed IF is certainly a bug!

> +      #~(append (list #$(file-append qemu "/bin/qemu-system-i386")
> +                      "-m" (number->string #$memory-size)
> +                      #$@net-options
> +                      #$@options
> +                      "--hda" #+image)
> +                (if (file-exists? "/dev/kvm")
> +                    '("--enable-kvm")
> +                    '())))

Looks good!  However...I tried adding a childhurd to a VM (see
attached), but it keeps looping...

--8<---------------cut here---------------start------------->8---
VNC server running on 127.0.0.1:5900
secret service: connected; sending files in "/etc/childhurd"
qemusystem-i386: Slirp: Failed to send packet, ret: -1
sending secrets to 11004
secret service: retrying connection [59 attempts left]
--8<---------------cut here---------------end--------------->8---

Greetings,
Janneke

[bare+childhurd.tmpl (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #38 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 43650 <at> debbugs.gnu.org
Subject: Re: [bug#43650] [PATCH 0/8] Assorted childhurd improvements
Date: Mon, 28 Sep 2020 19:10:01 +0200

Ludovic Courtès writes:

Hello!

> Here are assorted improvements to childhurds!

Oh, lovely!

> There’s one thing missing to allow ‘hurd-vm-service-type’ to
> automatically enable offloading to the local childhurd:
> declarative ACL and declarative machines.scm.
>
> Feedback welcome!  :-)

Took me much longer than I hoped to...:It's broke, for me ;)
(Well, privilege separatation breaks it, for me).

I especially hoped that childhurd in a Guix System VM would work, but in
the end reverted to reconfiguring and rebooting until I found the
problem.

(In the end, I'm pretty sure that rebooting is not necessary,
reconfiguring should be enough.)

The the rest of the patch set LreallyGTM, thanks!

> PS: It’s GNU’s 37th birthday! \o/

\o/ ...well, GNU needs to wait for their birthday present :-(

Greetings,
Janneke

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #41 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: 43650 <at> debbugs.gnu.org
Subject: Re: [bug#43650] [PATCH 0/8] Assorted childhurd improvements
Date: Mon, 28 Sep 2020 22:47:29 +0200

Hi!

Jan Nieuwenhuizen <janneke <at> gnu.org> skribis:

> Took me much longer than I hoped to...:It's broke, for me ;)
> (Well, privilege separatation breaks it, for me).
>
> I especially hoped that childhurd in a Guix System VM would work, but in
> the end reverted to reconfiguring and rebooting until I found the
> problem.

Oh, what exactly is broken for you?

I was able to “guix system vm” my laptop’s config, which includes an
instance of ‘hurd-vm-service-type’, and to connect with SSH or vncviewer
to the childhurd (running as non-root).

Does that fail for you?

Thanks for taking a look!

Ludo’.

Message #44 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: 43650 <at> debbugs.gnu.org
Subject: Re: [PATCH 1/8] services: hurd-vm: Run QEMU as an unprivileged user.
Date: Tue, 29 Sep 2020 00:19:34 +0200

Hi,

Jan Nieuwenhuizen <janneke <at> gnu.org> skribis:

>> Until qemu was running as "root", which is unnecessary.
>
> Well...I can't get this to work; my childhurd does not run.  Did you
> test it?  Any special tricks needed, adding "childhurd" to "kvm" maybe?

I did test it, but it seems there’s “something” that sometimes leads to
a startup failure and subsequent respawn of the Shepherd service (it can
be seen in the output of “herd status childhurd”).  Typically if I “herd
restart childhurd” it then proceeds and works.

To be continued…

Thanks for testing!

Ludo’.

Message #47 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 43650 <at> debbugs.gnu.org
Subject: Re: [bug#43650] [PATCH 1/8] services: hurd-vm: Run QEMU as an
 unprivileged user.
Date: Tue, 29 Sep 2020 10:06:24 +0300

[Message part 1 (text/plain, inline)]

On Mon, Sep 28, 2020 at 06:57:00PM +0200, Jan Nieuwenhuizen wrote:
> Ludovic Courtès writes:
> 
> Hello!
> 
> > Until qemu was running as "root", which is unnecessary.
> 
> Well...I can't get this to work; my childhurd does not run.  Did you
> test it?  Any special tricks needed, adding "childhurd" to "kvm" maybe?
> 
> I do like the idea...
> 
> Greetings,
> Janneke
> 

Shot in the dark, do the permissions/ownership on /var/empty matter?
childhurd is far from the only user claiming /var/empty as home.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #50 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: 43650 <at> debbugs.gnu.org
Subject: Re: [bug#43650] [PATCH 4/8] services: hurd-vm: Check whether
 /dev/kvm exists at run time.
Date: Tue, 29 Sep 2020 12:10:50 +0200

Hi!

I’ve pushed ‘wip-childhurd’ with a few additional commits.

The flaky startup issue appears to be fixed by:

  88946005d7 * services: secret-service: Add initial client/server handshake.

Before that, what would happen is that:

  1. The host would connect(2) to QEMU as soon as QEMU is running;
     connect(2) would succeed immediately and so the host would send its
     secrets right away, disconnect, and move on.

     However, at that point, the guest is still booting and its secret
     service server is not even accept(2)ing yet.  Looks like QEMU’s
     SLIRP would more or less buffer the packets the host sent, “more or
     less” being the important point.

  2. The guest would eventually accept(2), which would succeed.  Then it
     would sometimes receive stuff, sometimes not, depending on what
     happened with the SLIRP buffering I suppose.

The fix is to have the server in the guest send a “hello” message.  The
client in the host waits for that message before sending its secrets.

Consequently, it can take ~20s for the ‘start’ method of the childhurd
to succeed.  Eventually, when shepherd runs on Fibers or similar, it
won’t be a problem, but for now it means that PID 1 remains stuck in
select(2) for this many seconds.

Ludo’.

Message #53 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Efraim Flashner <efraim <at> flashner.co.il>
Cc: 43650 <at> debbugs.gnu.org, Jan Nieuwenhuizen <janneke <at> gnu.org>
Subject: Re: [bug#43650] [PATCH 1/8] services: hurd-vm: Run QEMU as an
 unprivileged user.
Date: Tue, 29 Sep 2020 12:23:19 +0200

Efraim Flashner <efraim <at> flashner.co.il> skribis:

> Shot in the dark, do the permissions/ownership on /var/empty matter?
> childhurd is far from the only user claiming /var/empty as home.

I don’t think so.  There’s code somewhere that ensures that /var/empty
is root-owned and read-only.

Ludo’.

Message #56 received at 43650 <at> debbugs.gnu.org (full text, mbox):

From: Jan Nieuwenhuizen <janneke <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 43650 <at> debbugs.gnu.org
Subject: Re: [bug#43650] [PATCH 4/8] services: hurd-vm: Check whether
 /dev/kvm exists at run time.
Date: Tue, 29 Sep 2020 16:22:02 +0200

[Message part 1 (text/plain, inline)]

Ludovic Courtès writes:

Hello,

> I’ve pushed ‘wip-childhurd’ with a few additional commits.

Great, this works/fixes it for me!  Using the attached
bare+childhurd.tmpl, I can build and start a Guix VM with a childhurd:

--8<---------------cut here---------------start------------->8---
$ $(./pre-inst-env guix system vm gnu/system/examples/bare+childhurd.tmpl) \
  -m 1G --nographic --net nic \
  --net user,hostfwd=tcp:127.0.0.1:12022-:2222,hostfwd=tcp:127.0.0.1:13022-:10022
--8<---------------cut here---------------end--------------->8---

and then, after half a minute or so:

--8<---------------cut here---------------start------------->8---
$ ssh -p 13022 localhost


  This is the GNU Hurd.  Welcome.

root <at> childhurd ~#
--8<---------------cut here---------------end--------------->8---

> The flaky startup issue appears to be fixed by:
>
>   88946005d7 * services: secret-service: Add initial client/server handshake.
>
> Before that, what would happen is that:
>
>   1. The host would connect(2) to QEMU as soon as QEMU is running;
>      connect(2) would succeed immediately and so the host would send its
>      secrets right away, disconnect, and move on.
>
>      However, at that point, the guest is still booting and its secret
>      service server is not even accept(2)ing yet.  Looks like QEMU’s
>      SLIRP would more or less buffer the packets the host sent, “more or
>      less” being the important point.
>
>   2. The guest would eventually accept(2), which would succeed.  Then it
>      would sometimes receive stuff, sometimes not, depending on what
>      happened with the SLIRP buffering I suppose.

Ah, thanks for the explanation...that makes sense.

> Consequently, it can take ~20s for the ‘start’ method of the childhurd
> to succeed.  Eventually, when shepherd runs on Fibers or similar, it
> won’t be a problem, but for now it means that PID 1 remains stuck in
> select(2) for this many seconds.

Yeah...Anyway LGTM!

Greetings,
Janneke

[bare+childhurd.tmpl (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

-- 
Jan Nieuwenhuizen <janneke <at> gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Message #61 received at 43650-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Jan Nieuwenhuizen <janneke <at> gnu.org>
Cc: 43650-done <at> debbugs.gnu.org
Subject: Re: [bug#43650] [PATCH 4/8] services: hurd-vm: Check whether
 /dev/kvm exists at run time.
Date: Tue, 29 Sep 2020 22:13:42 +0200

Hi!

Jan Nieuwenhuizen <janneke <at> gnu.org> skribis:

>> I’ve pushed ‘wip-childhurd’ with a few additional commits.
>
> Great, this works/fixes it for me!  Using the attached
> bare+childhurd.tmpl, I can build and start a Guix VM with a childhurd:
>
> $ $(./pre-inst-env guix system vm gnu/system/examples/bare+childhurd.tmpl) \
>   -m 1G --nographic --net nic \
>   --net user,hostfwd=tcp:127.0.0.1:12022-:2222,hostfwd=tcp:127.0.0.1:13022-:10022
>
>
> and then, after half a minute or so:
>
> $ ssh -p 13022 localhost
> 
> 
>   This is the GNU Hurd.  Welcome.
>
> root <at> childhurd ~#

Thanks for testing again.  I’ve pushed this to ‘master’ as commit
c11c19bd4d0dc4ec56b949647057dbf00567f2ae, along with a new system test
that ensures the childhurd’s SSH server is up and running in the end:

  https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c11c19bd4d0dc4ec56b949647057dbf00567f2ae

You can run it with:

  make check-system TESTS=childhurd

Thank you!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.