GNU bug report logs - #28288
[PATCH] Test and fix the ISO Image Installer

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 28288 in the body.
You can then email your comments to 28288 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: guix-patches <at> gnu.org
Subject: [PATCH] Test and fix the ISO Image Installer
Date: Wed, 30 Aug 2017 07:17:14 +0100

[Message part 1 (text/plain, inline)]

Hey,

So at the GNU Hackers Meeting, I attempted to build the installer system
as an ISO image, and test installing Guix on a Bytemark VM with this. I
encountered problems, and therefore set about writing a system test to
see if this could be reproduced.

The failures did seem reproducible locally, and Ludo helped with some
code changes. I'll send the work in progress patches to this bug.

[Message part 2 (application/pgp-signature, inline)]

Message #8 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28288 <at> debbugs.gnu.org
Subject: [PATCH 1/2] tests: Add test for installing from an ISO Image.
Date: Wed, 30 Aug 2017 07:20:09 +0100

* gnu/tests/install.scm (%test-iso-image-installer): New variable.
  (run-install): Add #:installation-disk-image-file-system-type as a keyword
  argument.
---
 gnu/tests/install.scm | 48 +++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 37 insertions(+), 11 deletions(-)

diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index 866bf885c..7596347a3 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -37,6 +37,7 @@
   #:use-module (guix utils)
   #:export (%test-installed-os
             %test-installed-extlinux-os
+            %test-iso-image-installer
             %test-separate-store-os
             %test-separate-home-os
             %test-raid-root-os
@@ -196,6 +197,7 @@ reboot\n")
                              (kernel-arguments '("console=ttyS0")))
                            #:imported-modules '((gnu services herd)
                                                 (guix combinators))))
+                      (installation-disk-image-file-system-type "ext4")
                       (target-size (* 1200 MiB)))
   "Run SCRIPT (a shell script following the GuixSD installation procedure) in
 OS to install TARGET-OS.  Return a VM image of TARGET-SIZE bytes containing
@@ -213,7 +215,9 @@ packages defined in installation-os."
                        (image  (system-disk-image
                                 (operating-system-with-gc-roots
                                  os (list target))
-                                #:disk-image-size (* 1500 MiB))))
+                                #:disk-image-size (* 1500 MiB)
+                                #:file-system-type
+                                installation-disk-image-file-system-type)))
     (define install
       (with-imported-modules '((guix build utils)
                                (gnu build marionette))
@@ -229,16 +233,24 @@ packages defined in installation-os."
 
             (define marionette
               (make-marionette
-               (cons (which #$(qemu-command system))
-                     (cons* "-no-reboot" "-m" "800"
-                            "-drive"
-                            (string-append "file=" #$image
-                                           ",if=virtio,readonly")
-                            "-drive"
-                            (string-append "file=" #$output ",if=virtio")
-                            (if (file-exists? "/dev/kvm")
-                                '("-enable-kvm")
-                                '())))))
+               `(,(which #$(qemu-command system))
+                 "-no-reboot"
+                 "-m" "800"
+                 ,@(case installation-disk-image-file-system-type
+                     (("ext4")
+                      `("-drive"
+                        ,(string-append "file=" #$image
+                                        ",if=virtio,readonly")))
+                     (("iso9660") '("-cdrom" #$image))
+                     (else
+                      (error
+                       "unsupported installation-disk-image-file-system-type"
+                       installation-disk-image-file-system-type)))
+                 "-drive"
+                 ,(string-append "file=" #$output ",if=virtio")
+                 ,@(if (file-exists? "/dev/kvm")
+                       '("-enable-kvm")
+                       '()))))
 
             (pk 'uname (marionette-eval '(uname) marionette))
 
@@ -312,6 +324,20 @@ per %test-installed-os, this test is expensive in terms of CPU and storage.")
       (run-basic-test %minimal-extlinux-os command
                       "installed-extlinux-os")))))
 
+(define %test-iso-image-installer
+  (system-test
+   (name "iso-image-installer")
+   (description
+    "")
+   (value
+    (mlet* %store-monad ((image   (run-install
+                                   %minimal-os
+                                   %minimal-os-source
+                                   #:installation-disk-image-file-system-type
+                                   "iso9660"))
+                         (command (qemu-command/writable-image image)))
+      (run-basic-test %minimal-os command name)))))
+
 
 ;;;
 ;;; Separate /home.
-- 
2.14.1

Message #11 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28288 <at> debbugs.gnu.org
Subject: [PATCH 2/2] WIP
Date: Wed, 30 Aug 2017 07:20:10 +0100

---
 gnu/build/vm.scm  | 19 +++++++++++++++++--
 gnu/system/vm.scm | 16 +++++++++++-----
 2 files changed, 28 insertions(+), 7 deletions(-)

diff --git a/gnu/build/vm.scm b/gnu/build/vm.scm
index 727494ad9..e2659cb5e 100644
--- a/gnu/build/vm.scm
+++ b/gnu/build/vm.scm
@@ -366,12 +366,27 @@ SYSTEM-DIRECTORY is the name of the directory of the 'system' derivation."
       (error "failed to create GRUB EFI image"))))
 
 (define* (make-iso9660-image grub config-file os-drv target
-                             #:key (volume-id "GuixSD_image") (volume-uuid #f))
+                             #:key (volume-id "GuixSD_image") (volume-uuid #f)
+                             register-closures? (closures '()))
   "Given a GRUB package, creates an iso image as TARGET, using CONFIG-FILE as
 GRUB configuration and OS-DRV as the stuff in it."
-  (let ((grub-mkrescue (string-append grub "/bin/grub-mkrescue")))
+  (let ((grub-mkrescue (string-append grub "/bin/grub-mkrescue"))
+        (target-store  (string-append "/tmp/root" (%store-directory))))
     (mkdir-p "/tmp/root/var/run")
     (mkdir-p "/tmp/root/run")
+
+    ;; XXX: 'guix-register' wants to palpate the things it registers, so
+    ;; bind-mount the store on the target.
+    (mkdir-p target-store)
+    (mount (%store-directory) target-store "" MS_BIND)
+
+    (when register-closures?
+      (display "registering closures...\n")
+      (for-each (lambda (closure)
+                  (register-closure target
+                                    (string-append "/xchg/" closure)))
+                closures))
+
     (unless (zero? (apply system*
                           `(,grub-mkrescue "-o" ,target
                             ,(string-append "boot/grub/grub.cfg=" config-file)
diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index 4494af003..b106dff0a 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -192,6 +192,7 @@ made available under the /xchg CIFS share."
                         os-drv
                         bootcfg-drv
                         bootloader
+                        register-closures?
                         (inputs '()))
   "Return a bootable, stand-alone iso9660 image.
 
@@ -207,8 +208,13 @@ INPUTS is a list of inputs (as for packages)."
          (let ((inputs
                 '#$(append (list qemu parted e2fsprogs dosfstools xorriso)
                            (map canonical-package
-                                (list sed grep coreutils findutils gawk))))
+                                (list sed grep coreutils findutils gawk))
+                           (if register-closures? (list guix) '())))
 
+
+               (graphs     '#$(match inputs
+                                   (((names . _) ...)
+                                    names)))
                ;; This variable is unused but allows us to add INPUTS-TO-COPY
                ;; as inputs.
                (to-register
@@ -222,6 +228,8 @@ INPUTS is a list of inputs (as for packages)."
                                #$bootcfg-drv
                                #$os-drv
                                "/xchg/guixsd.iso"
+                               #:register-closures? #$register-closures?
+                               #:closures graphs
                                #:volume-id #$file-system-label
                                #:volume-uuid #$file-system-uuid)
            (reboot))))
@@ -391,6 +399,7 @@ to USB sticks meant to be read-only."
                          #:file-system-label root-label
                          #:file-system-uuid #f
                          #:os-drv os-drv
+                         #:register-closures? #t
                          #:bootcfg-drv bootcfg
                          #:bootloader (bootloader-configuration-bootloader
                                         (operating-system-bootloader os))
@@ -403,10 +412,7 @@ to USB sticks meant to be read-only."
                                     (operating-system-bootloader os))
                       #:disk-image-size disk-image-size
                       #:disk-image-format "raw"
-                      #:file-system-type (if (string=? "iso9660"
-                                                       file-system-type)
-                                             "ext4"
-                                             file-system-type)
+                      #:file-system-type file-system-type
                       #:file-system-label root-label
                       #:copy-inputs? #t
                       #:register-closures? #t
-- 
2.14.1

Message #14 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28288 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: Re: [bug#28288] [PATCH 2/2] WIP
Date: Wed, 30 Aug 2017 07:31:13 +0100

[Message part 1 (text/plain, inline)]

Currently, I get the following error when running this test.

registering closures...
error: initialising database schema: disk I/O error
ERROR: In procedure scm-error:
ERROR: failed to register store items "/xchg/system"


I also need the patch from
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28273 otherwise I can't
even run the test.

Danny, I guess you have been looking at installing from an ISO image as
well, as I spotted you added it to the release target. Have you been
able to successfully use it yet?

When attempting to use it initially, I encountered a problem where a
glibc in the store was partially deleted. /var/guix/db/db.sqlite also
initially didn't exist, which the WIP patch here attempts to address. 

[Message part 2 (application/pgp-signature, inline)]

Message #17 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 2/2] WIP
Date: Wed, 30 Aug 2017 10:38:00 +0200

Hi Christopher,

On Wed, 30 Aug 2017 07:31:13 +0100
Christopher Baines <mail <at> cbaines.net> wrote:

> Currently, I get the following error when running this test.
> 
> registering closures...
> error: initialising database schema: disk I/O error
> ERROR: In procedure scm-error:
> ERROR: failed to register store items "/xchg/system"
> 
> 
> I also need the patch from
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28273 otherwise I can't
> even run the test.
> 
> Danny, I guess you have been looking at installing from an ISO image as
> well, as I spotted you added it to the release target. Have you been
> able to successfully use it yet?

Does that mean with the wip-installer-2 branch?

Otherwise in master I always tested it using gnu/system/install.scm until it has booted up successfully, which it did.

I've asked about the installation part before but there was no response.

> When attempting to use it initially, I encountered a problem where a
> glibc in the store was partially deleted. /var/guix/db/db.sqlite also
> initially didn't exist, which the WIP patch here attempts to address. 

Hmm, do you have logs of the release creation process?  Maybe xorriso complained?

Message #20 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 2/2] WIP
Date: Wed, 30 Aug 2017 20:26:37 +0200

> +    ;; XXX: 'guix-register' wants to palpate the things it registers, so
> +    ;; bind-mount the store on the target.
> +    (mkdir-p target-store)
> +    (mount (%store-directory) target-store "" MS_BIND)

What does this do?

gnu/build/vm.scm make-iso9660-image already provides /gnu/store a few lines down:

    (unless (zero? (apply system*
                          `(,grub-mkrescue "-o" ,target
                            ,(string-append "boot/grub/grub.cfg=" config-file)
                            ,(string-append "gnu/store=" os-drv "/..")
                                            ^^^^

Maybe I'm missing something...

Otherwise LGTM!

Message #23 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: ludo <at> gnu.org, 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 2/2] WIP
Date: Wed, 30 Aug 2017 23:56:47 +0100

[Message part 1 (text/plain, inline)]

On Wed, 30 Aug 2017 10:38:00 +0200
Danny Milosavljevic <dannym <at> scratchpost.org> wrote:

> Hi Christopher,
> 
> On Wed, 30 Aug 2017 07:31:13 +0100
> Christopher Baines <mail <at> cbaines.net> wrote:
> 
> > Currently, I get the following error when running this test.
> > 
> > registering closures...
> > error: initialising database schema: disk I/O error
> > ERROR: In procedure scm-error:
> > ERROR: failed to register store items "/xchg/system"
> > 
> > 
> > I also need the patch from
> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28273 otherwise I
> > can't even run the test.
> > 
> > Danny, I guess you have been looking at installing from an ISO
> > image as well, as I spotted you added it to the release target.
> > Have you been able to successfully use it yet?  
> 
> Does that mean with the wip-installer-2 branch?

I've just been testing with the master branch, and with the changes I
sent to this bug. Are there any changes on the wip-installer-2 that are
not solely related to the installer (which if I understand correctly is
something more complicated than running guix system init).

> Otherwise in master I always tested it using gnu/system/install.scm
> until it has booted up successfully, which it did.
> 
> I've asked about the installation part before but there was no
> response.

Ok, that's good to know. I would guess it's pretty close now, great
work on moving the ISO stuff forward!

> > When attempting to use it initially, I encountered a problem where a
> > glibc in the store was partially deleted. /var/guix/db/db.sqlite
> > also initially didn't exist, which the WIP patch here attempts to
> > address.   
> 
> Hmm, do you have logs of the release creation process?  Maybe xorriso
> complained?

From talking to Ludo at the GHM, I think there is some extra steps
needed to make installing from the ISO possible, but I don't yet
understand this. I'll CC Ludo, as he might be able to explain better.

[Message part 2 (application/pgp-signature, inline)]

Message #28 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: ludo <at> gnu.org, 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 2/2] WIP
Date: Thu, 31 Aug 2017 15:04:48 +0200

On Wed, 30 Aug 2017 23:56:47 +0100
Christopher Baines <mail <at> cbaines.net> wrote:

> I've just been testing with the master branch, and with the changes I
> sent to this bug. Are there any changes on the wip-installer-2 that are
> not solely related to the installer 

No

>(which if I understand correctly is something more complicated than running guix system init).

Yes

Message #31 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 2/2] WIP
Date: Thu, 31 Aug 2017 22:47:28 +0100

[Message part 1 (text/plain, inline)]

On Thu, 31 Aug 2017 15:04:48 +0200
Danny Milosavljevic <dannym <at> scratchpost.org> wrote:

> On Wed, 30 Aug 2017 23:56:47 +0100
> Christopher Baines <mail <at> cbaines.net> wrote:
> 
> > I've just been testing with the master branch, and with the changes
> > I sent to this bug. Are there any changes on the wip-installer-2
> > that are not solely related to the installer   
> 
> No
> 
> >(which if I understand correctly is something more complicated than
> >running guix system init).  
> 
> Yes

Great, ok, good to know I haven't missed anything.

I'll hopefully be able to make some time soonish to take a look at
getting the ISO image in to a state where running guix system init from
it works, unless someone beats me to it!

[Message part 2 (application/pgp-signature, inline)]

Message #34 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28288 <at> debbugs.gnu.org
Subject: [PATCH 3/5] vm: Add support for registering closures to iso9660-image.
Date: Sun,  3 Sep 2017 11:50:39 +0100

* gnu/system/vm.scm (iso9660-image): Add support for registering closures.
---
 gnu/system/vm.scm | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index 4494af003..fc55935aa 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -192,6 +192,7 @@ made available under the /xchg CIFS share."
                         os-drv
                         bootcfg-drv
                         bootloader
+                        register-closures?
                         (inputs '()))
   "Return a bootable, stand-alone iso9660 image.
 
@@ -207,8 +208,13 @@ INPUTS is a list of inputs (as for packages)."
          (let ((inputs
                 '#$(append (list qemu parted e2fsprogs dosfstools xorriso)
                            (map canonical-package
-                                (list sed grep coreutils findutils gawk))))
+                                (list sed grep coreutils findutils gawk))
+                           (if register-closures? (list guix) '())))
 
+
+               (graphs     '#$(match inputs
+                                   (((names . _) ...)
+                                    names)))
                ;; This variable is unused but allows us to add INPUTS-TO-COPY
                ;; as inputs.
                (to-register
@@ -222,6 +228,8 @@ INPUTS is a list of inputs (as for packages)."
                                #$bootcfg-drv
                                #$os-drv
                                "/xchg/guixsd.iso"
+                               #:register-closures? #$register-closures?
+                               #:closures graphs
                                #:volume-id #$file-system-label
                                #:volume-uuid #$file-system-uuid)
            (reboot))))
-- 
2.14.1

Message #37 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28288 <at> debbugs.gnu.org
Subject: [PATCH 1/5] vm: Add support for registering closures to
 make-iso9660-image.
Date: Sun,  3 Sep 2017 11:50:37 +0100

This mimics the functionality in the root-partition-initializer used in
creating the QEMU image. This helps when trying to run guix system init from
the generated ISO image.

* gnu/build/vm.scm (make-iso9660-image): Add support for registering closures.
---
 gnu/build/vm.scm | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/gnu/build/vm.scm b/gnu/build/vm.scm
index 727494ad9..606257d8c 100644
--- a/gnu/build/vm.scm
+++ b/gnu/build/vm.scm
@@ -366,12 +366,27 @@ SYSTEM-DIRECTORY is the name of the directory of the 'system' derivation."
       (error "failed to create GRUB EFI image"))))
 
 (define* (make-iso9660-image grub config-file os-drv target
-                             #:key (volume-id "GuixSD_image") (volume-uuid #f))
+                             #:key (volume-id "GuixSD_image") (volume-uuid #f)
+                             register-closures? (closures '()))
   "Given a GRUB package, creates an iso image as TARGET, using CONFIG-FILE as
 GRUB configuration and OS-DRV as the stuff in it."
-  (let ((grub-mkrescue (string-append grub "/bin/grub-mkrescue")))
+  (let ((grub-mkrescue (string-append grub "/bin/grub-mkrescue"))
+        (target-store  (string-append "/tmp/root" (%store-directory))))
     (mkdir-p "/tmp/root/var/run")
     (mkdir-p "/tmp/root/run")
+
+    (mkdir-p target-store)
+    (mount (%store-directory) target-store "" MS_BIND)
+
+    (when register-closures?
+      (display "registering closures...\n")
+      (for-each (lambda (closure)
+                  (register-closure
+                   "/tmp/root"
+                   (string-append "/xchg/" closure)
+                   #:deduplicate? #f))
+                closures))
+
     (unless (zero? (apply system*
                           `(,grub-mkrescue "-o" ,target
                             ,(string-append "boot/grub/grub.cfg=" config-file)
-- 
2.14.1

Message #40 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28288 <at> debbugs.gnu.org
Subject: [PATCH 2/5] vm: Create /mnt in the generated ISO image in
 make-iso9660-image.
Date: Sun,  3 Sep 2017 11:50:38 +0100

This is used in the installation process, as the mountpoint for the target
filesystem.

* gnu/build/vm.scm (make-iso9660-image): Create /mnt within the generated ISO
  image.
---
 gnu/build/vm.scm | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/gnu/build/vm.scm b/gnu/build/vm.scm
index 606257d8c..f6228b40b 100644
--- a/gnu/build/vm.scm
+++ b/gnu/build/vm.scm
@@ -374,6 +374,7 @@ GRUB configuration and OS-DRV as the stuff in it."
         (target-store  (string-append "/tmp/root" (%store-directory))))
     (mkdir-p "/tmp/root/var/run")
     (mkdir-p "/tmp/root/run")
+    (mkdir-p "/tmp/root/mnt")
 
     (mkdir-p target-store)
     (mount (%store-directory) target-store "" MS_BIND)
@@ -393,6 +394,10 @@ GRUB configuration and OS-DRV as the stuff in it."
                             ,(string-append "gnu/store=" os-drv "/..")
                             "var=/tmp/root/var"
                             "run=/tmp/root/run"
+                            ;; /mnt is used as part of the installation
+                            ;; process, as the mount point for the target
+                            ;; filesystem, so create it.
+                            "mnt=/tmp/root/mnt"
                             "--"
                             ;; Store two copies of the headers.
                             ;; The resulting ISO-9660 image has a DOS MBR and
-- 
2.14.1

Message #43 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28288 <at> debbugs.gnu.org
Subject: [PATCH 5/5] tests: Add test for installing from an ISO Image.
Date: Sun,  3 Sep 2017 11:50:41 +0100

* gnu/tests/install.scm (%test-iso-image-installer): New variable.
  (run-install): Add #:installation-disk-image-file-system-type as a keyword
  argument.
---
 gnu/tests/install.scm | 110 +++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 99 insertions(+), 11 deletions(-)

diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index 866bf885c..93c8a89cd 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -37,6 +37,7 @@
   #:use-module (guix utils)
   #:export (%test-installed-os
             %test-installed-extlinux-os
+            %test-iso-image-installer
             %test-separate-store-os
             %test-separate-home-os
             %test-raid-root-os
@@ -196,6 +197,7 @@ reboot\n")
                              (kernel-arguments '("console=ttyS0")))
                            #:imported-modules '((gnu services herd)
                                                 (guix combinators))))
+                      (installation-disk-image-file-system-type "ext4")
                       (target-size (* 1200 MiB)))
   "Run SCRIPT (a shell script following the GuixSD installation procedure) in
 OS to install TARGET-OS.  Return a VM image of TARGET-SIZE bytes containing
@@ -213,7 +215,9 @@ packages defined in installation-os."
                        (image  (system-disk-image
                                 (operating-system-with-gc-roots
                                  os (list target))
-                                #:disk-image-size (* 1500 MiB))))
+                                #:disk-image-size (* 1500 MiB)
+                                #:file-system-type
+                                installation-disk-image-file-system-type)))
     (define install
       (with-imported-modules '((guix build utils)
                                (gnu build marionette))
@@ -229,16 +233,25 @@ packages defined in installation-os."
 
             (define marionette
               (make-marionette
-               (cons (which #$(qemu-command system))
-                     (cons* "-no-reboot" "-m" "800"
-                            "-drive"
-                            (string-append "file=" #$image
-                                           ",if=virtio,readonly")
-                            "-drive"
-                            (string-append "file=" #$output ",if=virtio")
-                            (if (file-exists? "/dev/kvm")
-                                '("-enable-kvm")
-                                '())))))
+               `(,(which #$(qemu-command system))
+                 "-no-reboot"
+                 "-m" "800"
+                 #$@(cond
+                     ((string=? "ext4" installation-disk-image-file-system-type)
+                      `("-drive"
+                        ,(file-append "file=" image
+                                      ",if=virtio,readonly")))
+                     ((string=? "iso9660" installation-disk-image-file-system-type)
+                      `("-cdrom" ,image))
+                     (else
+                      (error
+                       "unsupported installation-disk-image-file-system-type:"
+                       installation-disk-image-file-system-type)))
+                 "-drive"
+                 ,(string-append "file=" #$output ",if=virtio")
+                 ,@(if (file-exists? "/dev/kvm")
+                       '("-enable-kvm")
+                       '()))))
 
             (pk 'uname (marionette-eval '(uname) marionette))
 
@@ -313,6 +326,81 @@ per %test-installed-os, this test is expensive in terms of CPU and storage.")
                       "installed-extlinux-os")))))
 
 
+;;;
+;;; Installation through an ISO image.
+;;;
+
+(define-os-with-source (%minimal-os-on-vda %minimal-os-on-vda-source)
+  ;; The OS we want to install.
+  (use-modules (gnu) (gnu tests) (srfi srfi-1))
+
+  (operating-system
+    (host-name "liberigilo")
+    (timezone "Europe/Paris")
+    (locale "en_US.UTF-8")
+
+    (bootloader (grub-configuration (target "/dev/vda")))
+    (kernel-arguments '("console=ttyS0"))
+    (file-systems (cons (file-system
+                          (device "my-root")
+                          (title 'label)
+                          (mount-point "/")
+                          (type "ext4"))
+                        %base-file-systems))
+    (users (cons (user-account
+                  (name "alice")
+                  (comment "Bob's sister")
+                  (group "users")
+                  (supplementary-groups '("wheel" "audio" "video"))
+                  (home-directory "/home/alice"))
+                 %base-user-accounts))
+    (services (cons (service marionette-service-type
+                             (marionette-configuration
+                              (imported-modules '((gnu services herd)
+                                                  (guix combinators)))))
+                    %base-services))))
+
+(define %simple-installation-script-for-/dev/vda
+  ;; Shell script of a simple installation.
+  "\
+. /etc/profile
+set -e -x
+guix --version
+
+export GUIX_BUILD_OPTIONS=--no-grafts
+guix build isc-dhcp
+parted --script /dev/vda mklabel gpt \\
+  mkpart primary ext2 1M 3M \\
+  mkpart primary ext2 3M 1G \\
+  set 1 boot on \\
+  set 1 bios_grub on
+mkfs.ext4 -L my-root /dev/vda2
+mount /dev/vda2 /mnt
+df -h /mnt
+herd start cow-store /mnt
+mkdir /mnt/etc
+cp /etc/target-config.scm /mnt/etc/config.scm
+guix system init /mnt/etc/config.scm /mnt --no-substitutes
+sync
+reboot\n")
+
+(define %test-iso-image-installer
+  (system-test
+   (name "iso-image-installer")
+   (description
+    "")
+   (value
+    (mlet* %store-monad ((image   (run-install
+                                   %minimal-os-on-vda
+                                   %minimal-os-on-vda-source
+                                   #:script
+                                   %simple-installation-script-for-/dev/vda
+                                   #:installation-disk-image-file-system-type
+                                   "iso9660"))
+                         (command (qemu-command/writable-image image)))
+      (run-basic-test %minimal-os-on-vda command name)))))
+
+
 ;;;
 ;;; Separate /home.
 ;;;
-- 
2.14.1

Message #46 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28288 <at> debbugs.gnu.org
Subject: [PATCH 4/5] vm: Call iso9660-image with #:register-closures? as #t.
Date: Sun,  3 Sep 2017 11:50:40 +0100

* gnu/system/vm.scm (system-disk-image): Call iso9660-image with
  #:register-closures? as #t.
---
 gnu/system/vm.scm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index fc55935aa..9cd973385 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -399,6 +399,7 @@ to USB sticks meant to be read-only."
                          #:file-system-label root-label
                          #:file-system-uuid #f
                          #:os-drv os-drv
+                         #:register-closures? #t
                          #:bootcfg-drv bootcfg
                          #:bootloader (bootloader-configuration-bootloader
                                         (operating-system-bootloader os))
-- 
2.14.1

Message #49 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 5/5] tests: Add test for installing from an
 ISO Image.
Date: Sun, 3 Sep 2017 11:58:38 +0100

[Message part 1 (text/plain, inline)]

On Sun,  3 Sep 2017 11:50:41 +0100
Christopher Baines <mail <at> cbaines.net> wrote:

> * gnu/tests/install.scm (%test-iso-image-installer): New variable.
>   (run-install): Add #:installation-disk-image-file-system-type as a
> keyword argument.
> ---
>  gnu/tests/install.scm | 110
> +++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 99
> insertions(+), 11 deletions(-)

This test now passes! I've also successfully generated an ISO image
that I used to install GuixSD to a Bytemark VM [1].

Most of the changes are educated guesses, mostly from copying the code
relating to QEMU images, so it would good to get some review of these.

1: https://www.bytemark.co.uk/cloud-hosting/

[Message part 2 (application/pgp-signature, inline)]

Message #52 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 4/5] vm: Call iso9660-image with
 #:register-closures? as #t.
Date: Tue, 5 Sep 2017 14:56:54 +0200

LGTM!

Message #55 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 3/5] vm: Add support for registering
 closures to iso9660-image.
Date: Tue, 5 Sep 2017 14:57:36 +0200

LGTM!

Message #58 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 2/5] vm: Create /mnt in the generated ISO
 image in make-iso9660-image.
Date: Tue, 5 Sep 2017 14:58:04 +0200

LGTM!

Message #61 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 1/5] vm: Add support for registering closures
 to make-iso9660-image.
Date: Tue, 05 Sep 2017 15:17:32 +0200

Hi!

Christopher Baines <mail <at> cbaines.net> skribis:

> This mimics the functionality in the root-partition-initializer used in
> creating the QEMU image. This helps when trying to run guix system init from
> the generated ISO image.
>
> * gnu/build/vm.scm (make-iso9660-image): Add support for registering closures.

[...]

> +    (when register-closures?
> +      (display "registering closures...\n")
> +      (for-each (lambda (closure)
> +                  (register-closure
> +                   "/tmp/root"
> +                   (string-append "/xchg/" closure)
> +                   #:deduplicate? #f))
> +                closures))

Was there any specific reason for #:deduplicate? #f here?

Otherwise LGTM!

Ludo’.

Message #64 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 5/5] tests: Add test for installing from an
 ISO Image.
Date: Tue, 05 Sep 2017 15:18:22 +0200

Christopher Baines <mail <at> cbaines.net> skribis:

> On Sun,  3 Sep 2017 11:50:41 +0100
> Christopher Baines <mail <at> cbaines.net> wrote:
>
>> * gnu/tests/install.scm (%test-iso-image-installer): New variable.
>>   (run-install): Add #:installation-disk-image-file-system-type as a
>> keyword argument.
>> ---
>>  gnu/tests/install.scm | 110
>> +++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 99
>> insertions(+), 11 deletions(-)
>
> This test now passes! I've also successfully generated an ISO image
> that I used to install GuixSD to a Bytemark VM [1].

Woohoo, really nice!  \o/

Thanks for persevering!

Ludo’.

Message #67 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: Christopher Baines <mail <at> cbaines.net>, 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 4/5] vm: Call iso9660-image with
 #:register-closures? as #t.
Date: Tue, 05 Sep 2017 15:18:39 +0200

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> LGTM!

+1!

Excellent.

Ludo'.

Message #70 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 1/5] vm: Add support for registering
 closures to make-iso9660-image.
Date: Wed, 6 Sep 2017 08:05:32 +0100

[Message part 1 (text/plain, inline)]

On Tue, 05 Sep 2017 15:17:32 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Hi!
> 
> Christopher Baines <mail <at> cbaines.net> skribis:
> 
> > This mimics the functionality in the root-partition-initializer
> > used in creating the QEMU image. This helps when trying to run guix
> > system init from the generated ISO image.
> >
> > * gnu/build/vm.scm (make-iso9660-image): Add support for
> > registering closures.  
> 
> [...]
> 
> > +    (when register-closures?
> > +      (display "registering closures...\n")
> > +      (for-each (lambda (closure)
> > +                  (register-closure
> > +                   "/tmp/root"
> > +                   (string-append "/xchg/" closure)
> > +                   #:deduplicate? #f))
> > +                closures))  
> 
> Was there any specific reason for #:deduplicate? #f here?

Yep, the image creation process fails if this is not specified.

error: cannot link
`/tmp/root/gnu/store/.links/1wj8bqv0ygz606a0day0gk7w37prsnbg5z2im2yf025r0y6lqfs0'
to
`/tmp/root/gnu/store/00lgsgxvv1hfhl22yqmzsa3lrs5qa9my-firmware/lib/firmware/b43-open':
Invalid cross-device link

There is probably another way around this though. I'm not sure what
impact skipping the deduplication has on the size of the image.

> Otherwise LGTM!
> 
> Ludo’.

[Message part 2 (application/pgp-signature, inline)]

Message #75 received at 28288-done <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: ludo <at> gnu.org (Ludovic Courtès), Danny Milosavljevic
 <dannym <at> scratchpost.org>
Cc: 28288-done <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 5/5] tests: Add test for installing from an
 ISO Image.
Date: Wed, 6 Sep 2017 08:46:49 +0100

[Message part 1 (text/plain, inline)]

On Tue, 05 Sep 2017 15:18:22 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Christopher Baines <mail <at> cbaines.net> skribis:
> 
> > On Sun,  3 Sep 2017 11:50:41 +0100
> > Christopher Baines <mail <at> cbaines.net> wrote:
> >  
> >> * gnu/tests/install.scm (%test-iso-image-installer): New variable.
> >>   (run-install): Add #:installation-disk-image-file-system-type as
> >> a keyword argument.
> >> ---
> >>  gnu/tests/install.scm | 110
> >> +++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed,
> >> 99 insertions(+), 11 deletions(-)  
> >
> > This test now passes! I've also successfully generated an ISO image
> > that I used to install GuixSD to a Bytemark VM [1].  
> 
> Woohoo, really nice!  \o/
> 
> Thanks for persevering!

Danny, Ludo, thanks for reviewing :) I've now pushed these changes.

[Message part 2 (application/pgp-signature, inline)]

Message #78 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 1/5] vm: Add support for registering closures
 to make-iso9660-image.
Date: Wed, 06 Sep 2017 15:20:44 +0200

Christopher Baines <mail <at> cbaines.net> skribis:

> On Tue, 05 Sep 2017 15:17:32 +0200
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> Hi!
>> 
>> Christopher Baines <mail <at> cbaines.net> skribis:
>> 
>> > This mimics the functionality in the root-partition-initializer
>> > used in creating the QEMU image. This helps when trying to run guix
>> > system init from the generated ISO image.
>> >
>> > * gnu/build/vm.scm (make-iso9660-image): Add support for
>> > registering closures.  
>> 
>> [...]
>> 
>> > +    (when register-closures?
>> > +      (display "registering closures...\n")
>> > +      (for-each (lambda (closure)
>> > +                  (register-closure
>> > +                   "/tmp/root"
>> > +                   (string-append "/xchg/" closure)
>> > +                   #:deduplicate? #f))
>> > +                closures))  
>> 
>> Was there any specific reason for #:deduplicate? #f here?
>
> Yep, the image creation process fails if this is not specified.
>
> error: cannot link
> `/tmp/root/gnu/store/.links/1wj8bqv0ygz606a0day0gk7w37prsnbg5z2im2yf025r0y6lqfs0'
> to
> `/tmp/root/gnu/store/00lgsgxvv1hfhl22yqmzsa3lrs5qa9my-firmware/lib/firmware/b43-open':
> Invalid cross-device link

Hmm, OK.  Looks like it’s trying to deduplicate between /tmp/root and
some other store.  (Or are we using a bind mount here?)

> There is probably another way around this though. I'm not sure what
> impact skipping the deduplication has on the size of the image.

It probably doesn’t matter much, so it’s not a blocker IMO.

Maybe leave an XXX comment mentioning the cross-device link issue.

Thanks!

Ludo’.

Message #81 received at 28288 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28288 <at> debbugs.gnu.org
Subject: Re: [bug#28288] [PATCH 1/5] vm: Add support for registering
 closures to make-iso9660-image.
Date: Sun, 10 Sep 2017 11:45:32 +0100

[Message part 1 (text/plain, inline)]

On Wed, 06 Sep 2017 15:20:44 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Christopher Baines <mail <at> cbaines.net> skribis:
> 
> > On Tue, 05 Sep 2017 15:17:32 +0200
> > ludo <at> gnu.org (Ludovic Courtès) wrote:
> >  
> >> Hi!
> >> 
> >> Christopher Baines <mail <at> cbaines.net> skribis:
> >>   
> >> > This mimics the functionality in the root-partition-initializer
> >> > used in creating the QEMU image. This helps when trying to run
> >> > guix system init from the generated ISO image.
> >> >
> >> > * gnu/build/vm.scm (make-iso9660-image): Add support for
> >> > registering closures.    
> >> 
> >> [...]
> >>   
> >> > +    (when register-closures?
> >> > +      (display "registering closures...\n")
> >> > +      (for-each (lambda (closure)
> >> > +                  (register-closure
> >> > +                   "/tmp/root"
> >> > +                   (string-append "/xchg/" closure)
> >> > +                   #:deduplicate? #f))
> >> > +                closures))    
> >> 
> >> Was there any specific reason for #:deduplicate? #f here?  
> >
> > Yep, the image creation process fails if this is not specified.
> >
> > error: cannot link
> > `/tmp/root/gnu/store/.links/1wj8bqv0ygz606a0day0gk7w37prsnbg5z2im2yf025r0y6lqfs0'
> > to
> > `/tmp/root/gnu/store/00lgsgxvv1hfhl22yqmzsa3lrs5qa9my-firmware/lib/firmware/b43-open':
> > Invalid cross-device link  
> 
> Hmm, OK.  Looks like it’s trying to deduplicate between /tmp/root and
> some other store.  (Or are we using a bind mount here?)

I think there is a bind mount in use here, so maybe that has something
to do with it.

> > There is probably another way around this though. I'm not sure what
> > impact skipping the deduplication has on the size of the image.  
> 
> It probably doesn’t matter much, so it’s not a blocker IMO.
> 
> Maybe leave an XXX comment mentioning the cross-device link issue.

Yep, I'll put in a comment :)

[Message part 2 (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.