Subject: [PATCH] Allow booting from a Btrfs subvolume.

From: Maxim Cournoyer <[email-masked]>
Date: Thu, 05 Sep 2019 09:20:02 +0900

Hello!

I'm sending this patch series to add support for booting off Btrfs
subvolumes.  There was some interested shown on #guix, so hopefully
someone can test it on their system :-)

Before this change, it wasn't possible to pass the required options to
the Linux kernel as our init script would ignore them.

I'm not including system tests yet, as this will take a bit more time
and is starting to be a big change in itself.

From 6858efa540d89c54ce377bfa6a6882e551cd2e56 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <[email-masked]>
Date: Sun, 14 Jul 2019 20:50:23 +0900
Subject: [PATCH 1/4] bootloader: grub: Allow booting from a Btrfs subvolume.

* gnu/bootloader/grub.scm (prepend-subvol, arguments->subvol): New procedures.
(grub-configuration-file): Use ARGUMENTS->SUBVOL to extract the subvolume name
from the kernel arguments, and prepend it to the kernel and initrd paths using
the PREPEND-SUBVOL procedure.
* tests/grub.scm: Add tests for the "arguments->subvol" procedure.
* doc/guix.texi (File Systems, (Bootloader Configuration): Document the use of
Btrfs subvolumes.
---
 doc/guix.texi               | 29 ++++++++++++++++++++-
 gnu/bootloader/grub.scm     | 43 +++++++++++++++++++++++++------
 gnu/build/linux-boot.scm    |  7 +++--
 gnu/build/linux-modules.scm |  3 ++-
 tests/grub.scm              | 51 +++++++++++++++++++++++++++++++++++++
 5 files changed, 121 insertions(+), 12 deletions(-)
 create mode 100644 tests/grub.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index 707c2ba700..cc7c91ac92 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -48,7 +48,7 @@ Copyright @copyright{} 2017 humanitiesNerd@*
 Copyright @copyright{} 2017 Christopher Allan Webber@*
 Copyright @copyright{} 2017, 2018 Marius Bakke@*
 Copyright @copyright{} 2017 Hartmut Goebel@*
-Copyright @copyright{} 2017 Maxim Cournoyer@*
+Copyright @copyright{} 2017, 2019 Maxim Cournoyer@*
 Copyright @copyright{} 2017, 2018, 2019 Tobias Geerinckx-Rice@*
 Copyright @copyright{} 2017 George Clemmer@*
 Copyright @copyright{} 2017 Andy Wingo@*
@@ -10802,6 +10802,20 @@ using the @code{file-system} form, like this:
   (type "ext4"))
 @end example
 
+@cindex Btrfs subvolume, file system
+Below is a more complex example, making use of a Btrfs subvolume, named
+@code{rootfs}, which parent Btrfs file system is labeled @code{my-btrfs-pool},
+on an encrypted device (hence the dependency on @code{mapped-devices}):
+
+@example
+(file-system
+  (device (file-system-label "my-btrfs-pool"))
+  (mount-point "/")
+  (type "btrfs")
+  (options "defaults,subvol=rootfs")
+  (dependencies mapped-devices))
+@end example
+
 As usual, some of the fields are mandatory---those shown in the example
 above---while others can be omitted.  These are described below.
 
@@ -24868,6 +24882,19 @@ when you boot it on your system.
 @code{grub-bootloader} allows you to boot in particular Intel-based machines
 in ``legacy'' BIOS mode.
 
+@cindex rootflags, Grub
+@cindex Btrfs root subvolume, Grub
+To allow using a Btrfs @emph{subvolume} for the root partition, the Grub-based
+bootloaders can use a subvolume @emph{name} passed using the @code{rootflags}
+kernel argument, e.g.:
+
+@example
+(kernel-arguments '("rootflags=subvol=@var{root-subvolume-name}"))
+@end example
+
+to correctly populate the @code{kernel} and @code{initrd} fields of the Grub
+configuration file.
+
 @cindex ARM, bootloaders
 @cindex AArch64, bootloaders
 Available bootloaders are described in @code{(gnu bootloader @dots{})}
diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm
index d984d5f5e3..dee6028dd0 100644
--- a/gnu/bootloader/grub.scm
+++ b/gnu/bootloader/grub.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2016 Chris Marusich <[email-masked]>
 ;;; Copyright © 2017 Leo Famulari <[email-masked]>
 ;;; Copyright © 2017 Mathieu Othacehe <[email-masked]>
+;;; Copyright © 2019 Maxim Cournoyer <[email-masked]>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -25,6 +26,8 @@
   #:use-module (guix gexp)
   #:use-module (gnu artwork)
   #:use-module (gnu bootloader)
+  #:use-module ((gnu build linux-modules) #:select (%not-comma))
+  #:use-module ((gnu build linux-boot) #:select (find-long-option))
   #:use-module (gnu system uuid)
   #:use-module (gnu system file-systems)
   #:use-module (gnu system keyboard)
@@ -34,6 +37,7 @@
   #:use-module (ice-9 match)
   #:use-module (ice-9 regex)
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
   #:export (grub-image
             grub-image?
             grub-image-aspect-ratio
@@ -73,6 +77,14 @@ denoting a file name."
                  file))))
     (#f file)))
 
+(define (prepend-subvol subvol file)
+  "Prepend SUBVOL from FILE, which is a gexp or other lowerable object
+denoting a file name."
+  (match subvol
+    ((? string? subvol)
+     #~(string-append "/" #$subvol #$file))
+    (#f file)))
+
 (define-record-type* <grub-image>
   grub-image make-grub-image
   grub-image?
@@ -313,6 +325,13 @@ code."
         ((or #f (? string?))
          #~(format #f "search --file --set ~a" #$file)))))
 
+(define (arguments->subvol arguments)
+  "Return any \"subvol\" value given as an option to the \"rootflags\"
+argument, or #f on failure."
+  (let* ((rootflags (find-long-option "rootflags" arguments))
+         (rootflags-options (and=> rootflags (cut string-tokenize <> %not-comma))))
+    (and=> rootflags-options (cut find-long-option "subvol" <>))))
+
 (define* (grub-configuration-file config entries
                                   #:key
                                   (system (%current-system))
@@ -324,18 +343,26 @@ entries corresponding to old generations of the system."
   (define all-entries
     (append entries (bootloader-configuration-menu-entries config)))
   (define (menu-entry->gexp entry)
-    (let ((device (menu-entry-device entry))
-          (device-mount-point (menu-entry-device-mount-point entry))
-          (label (menu-entry-label entry))
-          (kernel (menu-entry-linux entry))
-          (arguments (menu-entry-linux-arguments entry))
-          (initrd (menu-entry-initrd entry)))
+    (let* ((device (menu-entry-device entry))
+           (device-mount-point (menu-entry-device-mount-point entry))
+           (label (menu-entry-label entry))
+           (kernel (menu-entry-linux entry))
+           (arguments (menu-entry-linux-arguments entry))
+           (subvol (arguments->subvol arguments))
+           (initrd (menu-entry-initrd entry)))
       ;; Here DEVICE is the store and DEVICE-MOUNT-POINT is its mount point.
       ;; Use the right file names for KERNEL and INITRD in case
       ;; DEVICE-MOUNT-POINT is not "/", meaning that the store is on a
       ;; separate partition.
-      (let ((kernel  (strip-mount-point device-mount-point kernel))
-            (initrd  (strip-mount-point device-mount-point initrd)))
+
+      ;; Also, in case a subvolume name could be extracted from the "subvol"
+      ;; option given to the "rootflags" argument of the kernel, it is
+      ;; prepended to the kernel and initrd paths, to allow booting from
+      ;; a Btrfs subvolume.
+      (let ((kernel (prepend-subvol subvol (strip-mount-point
+                                            device-mount-point kernel)))
+            (initrd (prepend-subvol subvol (strip-mount-point
+                                            device-mount-point initrd))))
         #~(format port "menuentry ~s {
   ~a
   linux ~a ~a
diff --git a/gnu/build/linux-boot.scm b/gnu/build/linux-boot.scm
index f273957d78..7a30ebcef1 100644
--- a/gnu/build/linux-boot.scm
+++ b/gnu/build/linux-boot.scm
@@ -92,9 +92,12 @@
 
 (define (find-long-option option arguments)
   "Find OPTION among ARGUMENTS, where OPTION is something like \"--load\".
-Return the value associated with OPTION, or #f on failure."
+Return the value associated with OPTION, or #f on failure.  Any non-string
+arguments are ignored."
   (let ((opt (string-append option "=")))
-    (and=> (find (cut string-prefix? opt <>)
+    (and=> (find (lambda (arg)
+                   (and (string? arg)
+                        (string-prefix? opt arg)))
                  arguments)
            (lambda (arg)
              (substring arg (+ 1 (string-index arg #\=)))))))
diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm
index a149eff329..5f2efe7484 100644
--- a/gnu/build/linux-modules.scm
+++ b/gnu/build/linux-modules.scm
@@ -32,7 +32,8 @@
   #:use-module (ice-9 match)
   #:use-module (ice-9 rdelim)
   #:autoload   (ice-9 pretty-print) (pretty-print)
-  #:export (dot-ko
+  #:export (%not-comma
+            dot-ko
             ensure-dot-ko
             module-formal-name
             module-aliases
diff --git a/tests/grub.scm b/tests/grub.scm
new file mode 100644
index 0000000000..81453b00ab
--- /dev/null
+++ b/tests/grub.scm
@@ -0,0 +1,51 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2019 Maxim Cournoyer <[email-masked]>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (test-grub)
+  #:use-module (gnu bootloader grub)
+  #:use-module (srfi srfi-64))
+
+;;; Local bindings to private procedures that are to be tested.
+(define arguments->subvol (@@ (gnu bootloader grub) arguments->subvol))
+
+
+(test-begin "grub")
+
+(test-equal "Get subvolume name from arguments, multiple options"
+  "@"
+  (arguments->subvol '("ro" "debug"
+                       "rootflags=foo=bar,subvol=@,bar=baz")))
+
+(test-equal "No subvolume from arguments"
+  #f
+  (arguments->subvol '("rootflags=foo=bar")))
+
+(test-equal "No rootflags argument"
+  #f
+  (arguments->subvol '("ro" "debug")))
+
+(test-equal "Empty arguments list"
+  #f
+  (arguments->subvol '()))
+
+;;; The other types would typically be gexp objects.
+(test-equal "Argument list may contain other types than string"
+  "rootfs"
+  (arguments->subvol '(#f "rootflags=subvol=rootfs")))
+
+(test-end "grub")
-- 
2.23.0

From 3a628d1e731b2857a4c964484213cce980cb596f Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <[email-masked]>
Date: Tue, 16 Jul 2019 18:09:38 +0900
Subject: [PATCH 2/4] build: initrd: Fix "write-cpio-archive" return value.

* gnu/build/linux-initrd.scm (write-cpio-archive): Really return OUTPUT on
success, even when compression is disabled.
---
 gnu/build/linux-initrd.scm | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/gnu/build/linux-initrd.scm b/gnu/build/linux-initrd.scm
index 3aaa06d3a0..ea7de58553 100644
--- a/gnu/build/linux-initrd.scm
+++ b/gnu/build/linux-initrd.scm
@@ -71,8 +71,7 @@ COMPRESS? is true, compress it using GZIP.  On success, return OUTPUT."
       (cpio:write-cpio-archive files port
                                #:file->header cpio:file->cpio-header*)))
 
-  (or (not compress?)
-
+  (if compress?
       ;; Gzip insists on adding a '.gz' suffix and does nothing if the input
       ;; file already has that suffix.  Shuffle files around to placate it.
       (let* ((gz-suffix? (string-suffix? ".gz" output))
@@ -88,7 +87,6 @@ COMPRESS? is true, compress it using GZIP.  On success, return OUTPUT."
                (unless gz-suffix?
                  (rename-file (string-append output ".gz") output))
                output)))
-
       output))
 
 (define (cache-compiled-file-name file)
-- 
2.23.0

From 49ffe9a2645252bb708995169a9f1749f3982385 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <[email-masked]>
Date: Thu, 18 Jul 2019 07:23:48 +0900
Subject: [PATCH 3/4] linux-boot: Fix typo.

* gnu/build/linux-boot.scm (mount-root-file-system): Fix typo.
---
 gnu/build/linux-boot.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/build/linux-boot.scm b/gnu/build/linux-boot.scm
index 7a30ebcef1..b4e6421b27 100644
--- a/gnu/build/linux-boot.scm
+++ b/gnu/build/linux-boot.scm
@@ -362,8 +362,9 @@ the last argument of `mknod'."
 (define* (mount-root-file-system root type
                                  #:key volatile-root?)
   "Mount the root file system of type TYPE at device ROOT.  If VOLATILE-ROOT?
-is true, mount ROOT read-only and make it a overlay with a writable tmpfs
-using the kernel build-in overlayfs."
+is true, mount ROOT read-only and make it an overlay with a writable tmpfs
+using the kernel built-in overlayfs."
+
   (if volatile-root?
       (begin
         (mkdir-p "/real-root")
-- 
2.23.0

From b56aea9c62b015c8a8b48827f9587b1578c83af3 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <[email-masked]>
Date: Thu, 18 Jul 2019 04:59:25 +0900
Subject: [PATCH 4/4] linux-boot: Honor "rootflags" kernel argument.

* gnu/build/linux-boot.scm (mount-root-file-system): Add the optional FLAGS
and OPTIONS arguments; and document them.  Pass those to the `mount' calls.
(boot-system): Parse the "rootflags" kernel argument, and use it when calling
`mount-root-file-system'.
* doc/guix.texi (Initial RAM Disk): Document the use of the "rootflags"
argument.
---
 doc/guix.texi            | 19 +++++++++++++++++++
 gnu/build/linux-boot.scm | 22 +++++++++++++---------
 2 files changed, 32 insertions(+), 9 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index cc7c91ac92..1e093b38a0 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -24761,6 +24761,25 @@ Instruct the initial RAM disk as well as the @command{modprobe} command
 must be a comma-separated list of module names---e.g.,
 @code{usbkbd,9pnet}.
 
+@item rootflags=@var{options}@dots{}
+@cindex mount options, passed to initrd
+@cindex rootflags, initrd
+This argument allows passing one or multiple file system specific mount
+options to the @code{mount} procedure used by the init script.  @var{options}
+must be a comma-separated list of option names or option-value pairs.  The
+following example instructs the initial RAM disk to mount the Btrfs subvolume
+named ``rootfs'' as the root file system, and to enable automatic file
+defragmentation:
+
+@example
+rootflags=subvol=rootfs,autodefrag
+@end example
+
+Specifying the subvolume to mount by its name, as shown above, is also used in
+Guix to produce a working Grub configuration for the Grub-based bootloaders
+when using a Btrfs subvolume for the root file system (@xref{Bootloader
+Configuration}).
+
 @item --repl
 Start a read-eval-print loop (REPL) from the initial RAM disk before it
 tries to load kernel modules and to mount the root file system.  Our
diff --git a/gnu/build/linux-boot.scm b/gnu/build/linux-boot.scm
index b4e6421b27..b2d8f74a71 100644
--- a/gnu/build/linux-boot.scm
+++ b/gnu/build/linux-boot.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès <[email-masked]>
 ;;; Copyright © 2017 Mathieu Othacehe <[email-masked]>
+;;; Copyright © 2019 Maxim Cournoyer <[email-masked]>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -360,15 +361,16 @@ the last argument of `mknod'."
           (filter-map string->number (scandir "/proc")))))
 
 (define* (mount-root-file-system root type
+                                 #:optional (flags 0) options
                                  #:key volatile-root?)
-  "Mount the root file system of type TYPE at device ROOT.  If VOLATILE-ROOT?
-is true, mount ROOT read-only and make it an overlay with a writable tmpfs
-using the kernel built-in overlayfs."
-
+  "Mount the root file system of type TYPE at device ROOT.  The optional FLAGS
+and OPTIONS arguments behave the same as for the `mount' procedure.  If
+VOLATILE-ROOT?  is true, mount ROOT read-only and make it an overlay with a
+writable tmpfs using the kernel built-in overlayfs."
   (if volatile-root?
       (begin
         (mkdir-p "/real-root")
-        (mount root "/real-root" type MS_RDONLY)
+        (mount root "/real-root" type (logior MS_RDONLY flags) options)
         (mkdir-p "/rw-root")
         (mount "none" "/rw-root" "tmpfs")
 
@@ -385,11 +387,11 @@ using the kernel built-in overlayfs."
                "lowerdir=/real-root,upperdir=/rw-root/upper,workdir=/rw-root/work"))
       (begin
         (check-file-system root type)
-        (mount root "/root" type)))
+        (mount root "/root" type flags options)))
 
   ;; Make sure /root/etc/mtab is a symlink to /proc/self/mounts.
   (false-if-exception
-    (delete-file "/root/etc/mtab"))
+   (delete-file "/root/etc/mtab"))
   (mkdir-p "/root/etc")
   (symlink "/proc/self/mounts" "/root/etc/mtab"))
 
@@ -483,7 +485,8 @@ upon error."
      (mount-essential-file-systems)
      (let* ((args    (linux-command-line))
             (to-load (find-long-option "--load" args))
-            (root    (find-long-option "--root" args)))
+            (root    (find-long-option "--root" args))
+            (rootflags (find-long-option "rootflags" args)))
 
        (when (member "--repl" args)
          (start-repl))
@@ -526,7 +529,8 @@ upon error."
                              ((uuid root) => identity)
                              (else (file-system-label root)))))
              (mount-root-file-system (canonicalize-device-spec root)
-                                     root-fs-type
+                                     root-fs-type 0
+                                     rootflags
                                      #:volatile-root? volatile-root?))
            (mount "none" "/root" "tmpfs"))
 
-- 
2.23.0