GNU bug report logs - #40207
[PATCH 0/2] Add a desktop graphical installer test.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 40207 in the body.
You can then email your comments to 40207 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: guix-patches <at> gnu.org
Cc: Mathieu Othacehe <m.othacehe <at> gmail.com>
Subject: [PATCH 0/2] Add a desktop graphical installer test.
Date: Tue, 24 Mar 2020 09:34:43 +0100

Hello,

Here are two patches that allow testing the graphical installation of a Guix
System with all desktop-environments included.

Thanks,

Mathieu

Mathieu Othacehe (2):
  installer: tests: Use a filter to select desktop-environments.
  tests: install: Add %test-gui-installed-desktop-os-encrypted.

 gnu/installer/tests.scm | 11 +++--
 gnu/tests/install.scm   | 90 +++++++++++++++++++++++++++++++++--------
 2 files changed, 81 insertions(+), 20 deletions(-)

-- 
2.25.1

Message #8 received at 40207 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: 40207 <at> debbugs.gnu.org
Cc: Mathieu Othacehe <m.othacehe <at> gmail.com>
Subject: [PATCH 1/2] installer: tests: Use a filter to select
 desktop-environments.
Date: Tue, 24 Mar 2020 09:36:25 +0100

* gnu/installer/tests.scm (choose-services): Turn desktop-environments list
into a choose-desktop-environment procedure. This way, it is easier to select
all desktop-environments or none, in the same way as choose-network-service?
and choose-network-management-tool? arguments.
---
 gnu/installer/tests.scm | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/gnu/installer/tests.scm b/gnu/installer/tests.scm
index 6f5393e3ab..07f3121083 100644
--- a/gnu/installer/tests.scm
+++ b/gnu/installer/tests.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2020 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2020 Mathieu Othacehe <m.othacehe <at> gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -211,7 +212,7 @@ ROOT-PASSWORD, and USERS."
 
 (define* (choose-services port
                           #:key
-                          (desktop-environments '("GNOME"))
+                          (choose-desktop-environment? (const #f))
                           (choose-network-service?
                            (lambda (service)
                              (or (string-contains service "SSH")
@@ -220,10 +221,14 @@ ROOT-PASSWORD, and USERS."
                            (lambda (service)
                              (string-contains service "DHCP"))))
   "Converse over PORT to choose networking services."
+  (define desktop-environments '())
+
   (converse port
     ((checkbox-list (title "Desktop environment") (text _)
-                    (items _))
-     desktop-environments)
+                    (items ,services))
+     (let ((services* (filter choose-desktop-environment? services)))
+       (set! desktop-environments services*)
+       services*))
     ((checkbox-list (title "Network service") (text _)
                     (items ,services))
      (filter choose-network-service? services))
-- 
2.25.1

Message #11 received at 40207 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: 40207 <at> debbugs.gnu.org
Cc: Mathieu Othacehe <m.othacehe <at> gmail.com>
Subject: [PATCH 2/2] tests: install: Add
 %test-gui-installed-desktop-os-encrypted.
Date: Tue, 24 Mar 2020 09:36:26 +0100

* gnu/tests/install.scm (run-install): Make sure that the default target-size
is used if #f is passed,
(gui-test-program): add a desktop? argument, and pass it to choose-services,
(guided-installation-test): add desktop? and target-size arguments. If
desktop? is #t, make sure that all desktop-environments are available. Pass
target-size to run-install call and desktop? to gui-test-program call.
(%test-gui-installed-desktop-os-encrypted): New variable.
---
 gnu/tests/install.scm | 90 +++++++++++++++++++++++++++++++++++--------
 1 file changed, 73 insertions(+), 17 deletions(-)

diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index 83988873c2..9a4b36d5e7 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -32,15 +32,23 @@
   #:use-module (gnu packages cryptsetup)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages ocr)
+  #:use-module (gnu packages openbox)
   #:use-module (gnu packages package-management)
+  #:use-module (gnu packages ratpoison)
+  #:use-module (gnu packages suckless)
   #:use-module (gnu packages virtualization)
+  #:use-module (gnu packages wm)
+  #:use-module (gnu packages xorg)
+  #:use-module (gnu services desktop)
   #:use-module (gnu services networking)
+  #:use-module (gnu services xorg)
   #:use-module (guix store)
   #:use-module (guix monads)
   #:use-module (guix packages)
   #:use-module (guix grafts)
   #:use-module (guix gexp)
   #:use-module (guix utils)
+  #:use-module (srfi srfi-1)
   #:export (%test-installed-os
             %test-installed-extlinux-os
             %test-iso-image-installer
@@ -52,7 +60,8 @@
             %test-jfs-root-os
 
             %test-gui-installed-os
-            %test-gui-installed-os-encrypted))
+            %test-gui-installed-os-encrypted
+            %test-gui-installed-desktop-os-encrypted))
 
 ;;; Commentary:
 ;;;
@@ -203,13 +212,14 @@ reboot\n")
                                                 (gnu installer tests)
                                                 (guix combinators))))
                       (installation-disk-image-file-system-type "ext4")
-                      (target-size (* 2200 MiB)))
+                      (target-size #f))
   "Run SCRIPT (a shell script following the system installation procedure) in
 OS to install TARGET-OS.  Return a VM image of TARGET-SIZE bytes containing
 the installed system.  The packages specified in PACKAGES will be appended to
 packages defined in installation-os."
 
-  (mlet* %store-monad ((_      (set-grafting #f))
+  (mlet* %store-monad ((target-size -> (or target-size (* 2200 MiB)))
+                       (_      (set-grafting #f))
                        (system (current-system))
                        (target (operating-system-derivation target-os))
 
@@ -941,7 +951,10 @@ build (current-guix) and then store a couple of full system images.")
 
 (define %root-password "foo")
 
-(define* (gui-test-program marionette #:key (encrypted? #f))
+(define* (gui-test-program marionette
+                           #:key
+                           (desktop? #f)
+                           (encrypted? #f))
   #~(let ()
       (define (screenshot file)
         (marionette-control (string-append "screendump " file)
@@ -998,7 +1011,8 @@ build (current-guix) and then store a couple of full system images.")
       (screenshot "installer-services.ppm")
 
       (marionette-eval* '(choose-services installer-socket
-                                          #:desktop-environments '()
+                                          #:choose-desktop-environment?
+                                          (const #$desktop?)
                                           #:choose-network-service?
                                           (const #f))
                         #$marionette)
@@ -1038,7 +1052,11 @@ build (current-guix) and then store a couple of full system images.")
                         (gnu installer tests)
                         (guix combinators))))
 
-(define* (guided-installation-test name #:key encrypted?)
+(define* (guided-installation-test name
+                                   #:key
+                                   (desktop? #f)
+                                   encrypted?
+                                   (target-size #f))
   (define os
     (operating-system
       (inherit %minimal-os)
@@ -1055,26 +1073,56 @@ build (current-guix) and then store a couple of full system images.")
                             (supplementary-groups
                              '("wheel" "audio" "video"))))
                      %base-user-accounts))
+      (keyboard-layout (and desktop?
+                            (keyboard-layout "us" "altgr-intl")))
       ;; The installer does not create a swap device in guided mode with
       ;; encryption support.
       (swap-devices (if encrypted? '() '("/dev/vdb2")))
-      (services (cons (service dhcp-client-service-type)
-                      (operating-system-user-services %minimal-os)))))
+
+      ;; Make sure that all the packages and services that may be used by the
+      ;; graphical installer are available.
+      (packages (append
+                 (if desktop?
+                     (list openbox awesome i3-wm i3status
+                           dmenu st ratpoison xterm)
+                     '())
+                 %base-packages))
+      (services
+       (if desktop?
+           (append
+            (list (service gnome-desktop-service-type)
+                  (service xfce-desktop-service-type)
+                  (service mate-desktop-service-type)
+                  (service enlightenment-desktop-service-type)
+                  (set-xorg-configuration
+                   (xorg-configuration
+                    (keyboard-layout keyboard-layout)))
+                  (service marionette-service-type
+                           (marionette-configuration
+                            (imported-modules '((gnu services herd)
+                                                (guix build utils)
+                                                (guix combinators))))))
+            %desktop-services)
+           (cons (service dhcp-client-service-type)
+                 (operating-system-user-services %minimal-os))))))
 
   (system-test
    (name name)
    (description
     "Install an OS using the graphical installer and test it.")
    (value
-    (mlet* %store-monad ((image   (run-install os '(this is unused)
-                                               #:script #f
-                                               #:os installation-os-for-gui-tests
-                                               #:gui-test
-                                               (lambda (marionette)
-                                                 (gui-test-program
-                                                  marionette
-                                                  #:encrypted? encrypted?))))
-                         (command (qemu-command/writable-image image)))
+    (mlet* %store-monad
+        ((image   (run-install os '(this is unused)
+                               #:script #f
+                               #:os installation-os-for-gui-tests
+                               #:target-size target-size
+                               #:gui-test
+                               (lambda (marionette)
+                                 (gui-test-program
+                                  marionette
+                                  #:desktop? desktop?
+                                  #:encrypted? encrypted?))))
+         (command (qemu-command/writable-image image)))
       (run-basic-test os command name
                       #:initialization (and encrypted? enter-luks-passphrase)
                       #:root-password %root-password)))))
@@ -1087,4 +1135,12 @@ build (current-guix) and then store a couple of full system images.")
   (guided-installation-test "gui-installed-os-encrypted"
                             #:encrypted? #t))
 
+;; Building a desktop image is very time and space consuming. Install all
+;; desktop environments in a single test to reduce the overhead.
+(define %test-gui-installed-desktop-os-encrypted
+  (guided-installation-test "gui-installed-desktop-os-encrypted"
+                            #:desktop? #t
+                            #:encrypted? #t
+                            #:target-size (* 9000 MiB)))
+
 ;;; install.scm ends here
-- 
2.25.1

Message #14 received at 40207 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <m.othacehe <at> gmail.com>
Cc: 40207 <at> debbugs.gnu.org
Subject: Re: [bug#40207] [PATCH 1/2] installer: tests: Use a filter to select
 desktop-environments.
Date: Tue, 24 Mar 2020 11:31:58 +0100

Hi Mathieu,

Mathieu Othacehe <m.othacehe <at> gmail.com> skribis:

> * gnu/installer/tests.scm (choose-services): Turn desktop-environments list
> into a choose-desktop-environment procedure. This way, it is easier to select
> all desktop-environments or none, in the same way as choose-network-service?
> and choose-network-management-tool? arguments.

[...]

> +     (let ((services* (filter choose-desktop-environment? services)))
> +       (set! desktop-environments services*)
> +       services*))

Maybe rename ‘services*’ to ‘desktops’?

‘set!’ hurts my eyes :-) but let’s get over it.  Eventually we can
change convert to run expressions in ‘%state-monad’ or something.

Ludo’.

Message #17 received at 40207 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 40207 <at> debbugs.gnu.org
Subject: Re: [bug#40207] [PATCH 1/2] installer: tests: Use a filter to select
 desktop-environments.
Date: Tue, 24 Mar 2020 11:35:31 +0100

Hey!

> Maybe rename ‘services*’ to ‘desktops’?
>
> ‘set!’ hurts my eyes :-) but let’s get over it.  Eventually we can
> change convert to run expressions in ‘%state-monad’ or something.

Heh, also feel terribly guilty when using it :)

Thanks for reviewing,

Mathieu

Message #20 received at 40207 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <m.othacehe <at> gmail.com>
Cc: 40207 <at> debbugs.gnu.org
Subject: Re: [bug#40207] [PATCH 2/2] tests: install: Add
 %test-gui-installed-desktop-os-encrypted.
Date: Tue, 24 Mar 2020 11:38:12 +0100

Mathieu Othacehe <m.othacehe <at> gmail.com> skribis:

> * gnu/tests/install.scm (run-install): Make sure that the default target-size
> is used if #f is passed,
> (gui-test-program): add a desktop? argument, and pass it to choose-services,
> (guided-installation-test): add desktop? and target-size arguments. If
> desktop? is #t, make sure that all desktop-environments are available. Pass
> target-size to run-install call and desktop? to gui-test-program call.
> (%test-gui-installed-desktop-os-encrypted): New variable.

[...]

> -                      (target-size (* 2200 MiB)))
> +                      (target-size #f))
>    "Run SCRIPT (a shell script following the system installation procedure) in
>  OS to install TARGET-OS.  Return a VM image of TARGET-SIZE bytes containing
>  the installed system.  The packages specified in PACKAGES will be appended to
>  packages defined in installation-os."
>  
> -  (mlet* %store-monad ((_      (set-grafting #f))
> +  (mlet* %store-monad ((target-size -> (or target-size (* 2200 MiB)))

[...]

> +(define* (guided-installation-test name
> +                                   #:key
> +                                   (desktop? #f)
> +                                   encrypted?
> +                                   (target-size #f))

What about giving #:target-size a default value here so that the hunk
above in unnecessary?

>    (define os
>      (operating-system
>        (inherit %minimal-os)
> @@ -1055,26 +1073,56 @@ build (current-guix) and then store a couple of full system images.")
>                              (supplementary-groups
>                               '("wheel" "audio" "video"))))
>                       %base-user-accounts))
> +      (keyboard-layout (and desktop?
> +                            (keyboard-layout "us" "altgr-intl")))
>        ;; The installer does not create a swap device in guided mode with
>        ;; encryption support.
>        (swap-devices (if encrypted? '() '("/dev/vdb2")))
> -      (services (cons (service dhcp-client-service-type)
> -                      (operating-system-user-services %minimal-os)))))
> +
> +      ;; Make sure that all the packages and services that may be used by the
> +      ;; graphical installer are available.
> +      (packages (append
> +                 (if desktop?
> +                     (list openbox awesome i3-wm i3status
> +                           dmenu st ratpoison xterm)
> +                     '())
> +                 %base-packages))
> +      (services
> +       (if desktop?
> +           (append
> +            (list (service gnome-desktop-service-type)
> +                  (service xfce-desktop-service-type)
> +                  (service mate-desktop-service-type)
> +                  (service enlightenment-desktop-service-type)
> +                  (set-xorg-configuration
> +                   (xorg-configuration
> +                    (keyboard-layout keyboard-layout)))
> +                  (service marionette-service-type
> +                           (marionette-configuration
> +                            (imported-modules '((gnu services herd)
> +                                                (guix build utils)
> +                                                (guix combinators))))))
> +            %desktop-services)
> +           (cons (service dhcp-client-service-type)
> +                 (operating-system-user-services %minimal-os))))))

What about making a separate OS definition for the desktop installation
case?  We’d move the ‘os’ variable to the top-level and add an ‘os’
parameter to ‘guided-installation-test’.

Having two separate OS definitions (one for desktop install, one for
“bare-bones” install) should be clearer than having a single definition
with conditionals.

> +;; Building a desktop image is very time and space consuming. Install all
> +;; desktop environments in a single test to reduce the overhead.
> +(define %test-gui-installed-desktop-os-encrypted
> +  (guided-installation-test "gui-installed-desktop-os-encrypted"
> +                            #:desktop? #t
> +                            #:encrypted? #t
> +                            #:target-size (* 9000 MiB)))

Wo0t!

Thanks a lot for working on it!  You can stop by on #guix if you feel
like it so we’re all less lonely during confinement.  :-)

Ludo’.

Message #23 received at 40207 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 40207 <at> debbugs.gnu.org
Subject: Re: [bug#40207] [PATCH 2/2] tests: install: Add
 %test-gui-installed-desktop-os-encrypted.
Date: Wed, 25 Mar 2020 18:57:00 +0100

Hey,

> What about making a separate OS definition for the desktop installation
> case?  We’d move the ‘os’ variable to the top-level and add an ‘os’
> parameter to ‘guided-installation-test’.
>
> Having two separate OS definitions (one for desktop install, one for
> “bare-bones” install) should be clearer than having a single definition
> with conditionals.

I fixed all your remarks. But there's still an issue, the size of the
installation image is guessed badly in "run-install" for
"%test-gui-installed-desktop-os-encrypted"

--8<---------------cut here---------------start------------->8---
(image  (system-disk-image
         (operating-system-with-gc-roots
          os (list target))
         #:disk-image-size 'guess ;guess is too low.
         #:file-system-type
         installation-disk-image-file-system-type)))
--8<---------------cut here---------------end--------------->8---

applying the dirty snippet below fixes it, but you may have a better idea?

--8<---------------cut here---------------start------------->8---
--- a/gnu/build/vm.scm
+++ b/gnu/build/vm.scm
@@ -233,7 +233,7 @@ deduplicates files common to CLOSURE and the rest of PREFIX."
   "Return the estimated size of a partition that can store the store items
 given by GRAPHS, a list of file names produced by #:references-graphs."
   ;; Simply add a 25% overhead.
-  (round (* 1.25 (closure-size graphs))))
+  (round (* 1.30 (closure-size graphs))))

 (define* (initialize-partition-table device partitions
--8<---------------cut here---------------end--------------->8---

Thanks,

Mathieu

Message #26 received at 40207 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <m.othacehe <at> gmail.com>
Cc: 40207 <at> debbugs.gnu.org
Subject: Re: [bug#40207] [PATCH 2/2] tests: install: Add
 %test-gui-installed-desktop-os-encrypted.
Date: Wed, 25 Mar 2020 22:41:11 +0100

Hi!

Mathieu Othacehe <m.othacehe <at> gmail.com> skribis:

>> What about making a separate OS definition for the desktop installation
>> case?  We’d move the ‘os’ variable to the top-level and add an ‘os’
>> parameter to ‘guided-installation-test’.
>>
>> Having two separate OS definitions (one for desktop install, one for
>> “bare-bones” install) should be clearer than having a single definition
>> with conditionals.
>
> I fixed all your remarks. But there's still an issue, the size of the
> installation image is guessed badly in "run-install" for
> "%test-gui-installed-desktop-os-encrypted"
>
> (image  (system-disk-image
>          (operating-system-with-gc-roots
>           os (list target))
>          #:disk-image-size 'guess ;guess is too low.
>          #:file-system-type
>          installation-disk-image-file-system-type)))
>
>
> applying the dirty snippet below fixes it, but you may have a better idea?
>
> --- a/gnu/build/vm.scm
> +++ b/gnu/build/vm.scm
> @@ -233,7 +233,7 @@ deduplicates files common to CLOSURE and the rest of PREFIX."
>    "Return the estimated size of a partition that can store the store items
>  given by GRAPHS, a list of file names produced by #:references-graphs."
>    ;; Simply add a 25% overhead.
> -  (round (* 1.25 (closure-size graphs))))
> +  (round (* 1.30 (closure-size graphs))))

Weird, the 25% hack had always worked fine.

I don’t have a better idea though.  Perhaps it’s better to pass an
explicit size in ‘run-install’ than to modify the 25% hack above?

Thanks,
Ludo’.

Message #29 received at 40207 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <m.othacehe <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 40207 <at> debbugs.gnu.org
Subject: Re: [bug#40207] [PATCH 2/2] tests: install: Add
 %test-gui-installed-desktop-os-encrypted.
Date: Thu, 26 Mar 2020 11:54:50 +0100

Hey,

> I don’t have a better idea though.  Perhaps it’s better to pass an
> explicit size in ‘run-install’ than to modify the 25% hack above?

Seems fair, fixed and pushed!

Thanks,

Mathieu

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.