Subject: Re: [PATCH] gnu, doc: Create and document procedure to compile ZFS for specific kernel.

From: raid5atemyhomework <[email-masked]>
Date: Sat, 02 Jan 2021 10:22:25 +0000

Did some more testing and finally got ZFS working inside a VM!

Turns out the `zfs` package definition was buggy --- it was using `util-linux:lib`, but trying to execute `mount` and `umount` from it.  So I modified the package definition as well to include `util-linux:lib` as `"util-linux-lib"` and gave it `util-linux` directly as well.

I also created a new service to load ZFS kernel module *and* import all ZFS pools at boot and mount them, which is the behavior ZFS has on other distros and platforms, so we should emulate it as well.

ZFS as root is not supported yet. I presume it requires `initrd` hacking, plus it also has to be supported somehow in the installer or something as well.

Here's the patch!

----

>From d02b9a69ad8c261e7f7c7bf145117327d1217d1c Mon Sep 17 00:00:00 2001
From: raid5atemyhomework <[email-masked]>
Date: Fri, 1 Jan 2021 20:26:42 +0800
Subject: [PATCH] gnu, doc: Create and document procedure to compile ZFS for
 specific kernel.

---
 doc/guix.texi                 | 98 +++++++++++++++++++++++++++++++++++
 gnu/packages/file-systems.scm | 25 ++++++++-
 gnu/services/file-systems.scm | 65 +++++++++++++++++++++++
 3 files changed, 186 insertions(+), 2 deletions(-)
 create mode 100644 gnu/services/file-systems.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index 1081ed26a3..620dc48ff5 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -13885,6 +13885,104 @@ a file system declaration such as:
 compress-force=zstd,space_cache=v2"))
 @end lisp

+@node ZFS file system
+@subsection ZFS file system
+
+The ZFS on Linux file system cannot legally be downloaded as part of the
+Linux kernel, but you as a user can do anything on hardware you own,
+including download ZFS as source code, compile ZFS as a kernel module,
+and link it into Linux.
+
+As a large and complex kernel module, ZFS has to be compiled for a
+specific version of Linux. Often the latest ZFS package available in Guix
+cannot be compiled with the latest Linux kernel available in Guix, so
+installing the @code{zfs} package in your system configuration file will
+fail.
+
+Instead, you have to check the
+@url{https://github.com/openzfs/zfs/releases,OpenZFS release notes} for
+the specific version of ZFS that Guix has packaged to determine what
+Linux kernels you can use, then check the @code{linux-libre} packages
+that Guix has packaged, and select one you can use on your system.
+
+Then, you have to modify your system configuration file, and create a
+ZFS package that compiles using the specific Linux version you chose.
+Below is a sketch of how you would modify your @code{operating-system}
+declaration in order to set up ZFS:
+
+@lisp
+(use-modules (gnu))
+;; @dots{}
+(use-service-modules
+  ; @dots{}
+  linux
+  file-systems)
+(use-package-modules
+  ; @dots{}
+  linux
+  file-systems)
+
+;; @dots{}
+
+;;; (1) Select a specific kernel.
+(define system-kernel linux-libre-5.4)
+;;; (2) Define a ZFS package for your kernel.
+(define system-zfs (make-zfs-package system-kernel))
+
+;; @dots{}
+
+(operating-system
+  ;;; (3) Specify your selected kernel.
+  (kernel system-kernel)
+  ;;; (4) Add the "module" output of the system ZFS package to
+  ;;; the kernel-loadable modules.
+  (kernel-loadable-modules (list (list system-zfs "module")))
+
+  ;; @dots{}
+
+  (packages
+    ;;; (5) Add the system ZFS package to global packages so that
+    ;;; "zfs", "zpool" etc. commands are available.
+    (cons* system-zfs
+           ; @dots{}
+           %base-packages))
+
+  ;; @dots{}
+
+  (services
+    ;;; (6) Add a service that loads ZFS and all ZFS pools at boot.
+    (cons* (service zfs-loader-service-type
+                    system-zfs)
+           ; @dots{}
+           %base-services))
+  ;; @dots{}
+  )
+@end lisp
+
+@deffn (Scheme Procedure) make-zfs-package @var{kernel}
+This procedure creates a package which, when included as a package
+in your system, can be loaded as a kernel module for the specified
+@var{kernel}, a Linux kernel package.
+It is intended to create a system-specific ZFS kernel module for
+the Linux kernel you will use in your system.
+@end deffn
+
+@deffn (Scheme Variable) zfs-loader-service-type
+This is the service that loads the ZFS kernel module and imports all
+ZFS pools. The value.associated with @code{zfs-loader-servie-type}
+services must be the ZFS package you defined for your system, and
+whose @code{"module"} output you have included as a loadable kernel
+module.
+@end deffn
+
+When you have modified your @code{operating-system} to include ZFS
+as in the above and reconfigured and rebooted, you will now be able
+to create, mount, and manage ZFS pools. At each boot, ZFS will
+automatically open any ZFS pools it detects in your system, and will
+mount ZFS pools that have a non-@code{legacy} mountpoint.
+
+ZFS as root filesystem is not supported yet.
+
 @node Mapped Devices
 @section Mapped Devices

diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scm
index 895ad069c5..e8f49a59fa 100644
--- a/gnu/packages/file-systems.scm
+++ b/gnu/packages/file-systems.scm
@@ -944,16 +944,37 @@ APFS.")
        ("openssl" ,openssl)
        ("python" ,python)
        ("python-cffi" ,python-cffi)
-       ("util-linux" ,util-linux "lib")
+       ("util-linux" ,util-linux)
+       ("util-linux-lib" ,util-linux "lib")
        ("zlib" ,zlib)))
     (home-page "https://zfsonlinux.org/")
     (synopsis "Native ZFS on Linux")
     (description
      "ZFS on Linux is an advanced file system and volume manager which was
 originally developed for Solaris and is now maintained by the OpenZFS
-community.")
+community.
+
+DO NOT INSTALL THIS PACKAGE. Instead, refer to the 'ZFS file system' section
+of the Guix info manual for how to install ZFS.")
     (license license:cddl1.0)))

+(define-public (make-zfs-package kernel)
+  (package
+    (inherit zfs)
+    (name (string-append "zfs-for-"
+                         (package-name kernel)
+                         "-"
+                         (package-version kernel)
+                         "-version"))
+    (arguments
+      (cons* #:linux kernel (package-arguments zfs)))
+    (description
+     "ZFS on Linux is an advanced file system and volume manager which was
+originally developed for Solaris and is now maintained by the OpenZFS
+community.
+
+This package has been compiled for a specific Linux kernel.")))
+
 (define-public mergerfs
   (package
     (name "mergerfs")
diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm
new file mode 100644
index 0000000000..97132cc53d
--- /dev/null
+++ b/gnu/services/file-systems.scm
@@ -0,0 +1,65 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 raid5atemyhomework <[email-masked]>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services file-systems)
+  #:use-module (guix gexp)
+  #:use-module (gnu services)
+  #:use-module (gnu services shepherd))
+
+;;; ZFS
+(define (zfs-loader-shepherd-service system-zfs)
+  (let* ((zpool (file-append system-zfs "/sbin/zpool")))
+    (list
+      (shepherd-service
+       (documentation "Load ZFS kernel module and import ZFS pools.")
+       (provision '(zfs))
+       (requirement '(file-systems))
+       (one-shot? #t)
+       (modules `((srfi srfi-1)
+                  (srfi srfi-34)
+                  (srfi srfi-35)
+                  (rnrs io ports)
+                  ,@%default-modules))
+       (start #~(lambda _
+                  (if (not (file-exists? "/proc/sys/kernel/modprobe"))
+                      (begin (format (current-error-port) "error: ~a~%"
+                                     "Kernel is missing loadable module support.")
+                             #f)
+                      (and
+                        (let ((modprobe (call-with-input-file
+                                          "/proc/sys/kernel/modprobe" get-line)))
+                          (guard (c ((message-condition? c)
+                                     (format (current-error-port)
+                                             "error loading 'zfs' kernel module: ~a~%"
+                                             (condition-message c))
+                                     #f))
+                            (invoke/quiet modprobe "--" "zfs")))
+                        (guard (c ((message-condition? c)
+                                   (format (current-error-port)
+                                           "error importing zpools: ~a~%"
+                                           (condition-message c))
+                                   #f))
+                          (invoke/quiet #$zpool "import" "-a"))))))))))
+
+(define-public zfs-loader-service-type
+  (service-type
+   (name 'zfs-loader)
+   (description "Load ZFS kernel module and import ZFS pools.")
+   (extensions
+    (list (service-extension shepherd-root-service-type zfs-loader-shepherd-service)))))
+
--
2.29.2