GNU bug report logs - #74837
[PATCH 0/2] Add resize-fs service

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 74837 in the body.
You can then email your comments to 74837 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Richard Sent <richard <at> freakingpenguin.com>
To: guix-patches <at> gnu.org
Cc: Richard Sent <richard <at> freakingpenguin.com>,
 Gabriel Wicki <gabriel <at> erlikon.ch>
Subject: [PATCH 0/2] Add resize-fs service
Date: Thu, 12 Dec 2024 15:16:33 -0500

Hi all,

This patch series adds a resize-fs service, a service designed to resize a
file system. It does this by utilizing cloud-utils growpart and an appropriate
file system utility.

This is loosely based on code from Gabriel Wicki in #69090. While it has been
substantially changed, I figured I'd preserve the copyright notice.

I opted to use a file-system object directly and resolve the device path when
the service runs instead of having the user hardcode the parent device path
and partition number ahead of time. This hopefully makes it a bit easier to
use, although the user still needs to ensure the desired partition is at the
end of any image.

I tested this with a removable flash drive and an ext4 file system. I'd
greatly appreciate additional help; it's quite a pain! I have not tried it
using a system image on a VPS.

One possible limitation is that this service only supports resizing one file
system and cannot be extended. On the other hand I'm struggling to come up
with use cases where someone would need to extend multiple file systems
simultaneously.

I expect this patch will need some feedback. :)

Richard Sent (2):
  gnu: services: Add resize-fs-service.
  packages: cloud-utils: Add missing growpart programs to path.

 doc/guix.texi          |  50 +++++++++++++++
 gnu/packages/admin.scm |  15 ++++-
 gnu/services/admin.scm | 135 ++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 196 insertions(+), 4 deletions(-)


base-commit: a9003b8e6b40b59c9545ae87bb441d3549630db7
-- 
2.46.0

Message #8 received at 74837 <at> debbugs.gnu.org (full text, mbox):

From: Richard Sent <richard <at> freakingpenguin.com>
To: 74837 <at> debbugs.gnu.org
Cc: Richard Sent <richard <at> freakingpenguin.com>,
 Gabriel Wicki <gabriel <at> erlikon.ch>
Subject: [PATCH 1/2] gnu: services: Add resize-fs-service.
Date: Thu, 12 Dec 2024 15:18:59 -0500

* gnu/services/admin.scm (resize-fs-configuration): New configuration
type.
(resize-fs-shepherd-service): New procedure.
(resize-fs-service-type): New variable.
* doc/guix.texi (Miscallaneous Services): Document it.

Change-Id: Icae2fefc9a8d936d4c3add47520258b341f689a4
---
 doc/guix.texi          |  50 +++++++++++++++
 gnu/services/admin.scm | 135 ++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 184 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index a2915de954..5636eb23fb 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41891,6 +41891,56 @@ Miscellaneous Services
 
 @c End of auto-generated fail2ban documentation.
 
+@cindex resize-fs
+@subsubheading Resize File System service
+
+This service type lets you resize a live file system during boot, which
+can be convenient if a Guix image is flashed on an SD Card (e.g. for an
+embedded device) or uploaded to a VPS.  In both cases the medium the
+image will reside upon may be larger than the image you want to produce.
+
+For an embedded device booting from an SD card you may use something like:
+@lisp
+(service resize-fs-service-type
+  (resize-fs-configuration
+    (file-system
+     (device (file-system-label "root"))
+     (type "ext4"))))
+@end lisp
+
+Be extra cautious to use the correct device and type.  The service has
+little error handling of its own and relies on the underlying tools.
+Wrong use could end in loss of data or the corruption of the operating
+system.
+
+Partitions and file systems are grown to the maximum size available.
+File systems can only grow when they are on the last partition on a
+device and have empty space available.
+
+This service supports the ext2, ext3, ext4, btrfs, and bcachefs file
+systems.
+
+@table @asis
+
+@item @code{file-system} (default: @code{#f}) (type: file-system)
+The file-system object to resize.  This object must have the device and
+type fields set.  The others are ignored.
+
+@item @code{cloud-utils} (default: @code{cloud-utils}) (type: file-like)
+The cloud-utils package to use.
+
+@item @code{e2fsprogs} (default: @code{e2fsprogs}) (type: file-like)
+The e2fsprogs package to use, used for resizing ext2, ext3, and ext4
+file systems.
+
+@item @code{btrfs-progs} (default: @code{btrfs-progs}) (type: file-like)
+The btrfs-progs package to use, used for resizing the btrfs file system.
+
+@item @code{bcachefs-tools} (default: @code{bcachefs-tools}) (type: file-like)
+The bcachefs-tools package to use, used for resizing the bcachefs file system.
+
+@end table
+
 @cindex Backup
 @subsubheading Backup Services
 
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index 24ff659a01..f754012e1e 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -3,6 +3,8 @@
 ;;; Copyright © 2016-2023 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2020 Brice Waegeneire <brice <at> waegenei.re>
 ;;; Copyright © 2023 Giacomo Leidi <goodoldpaul <at> autistici.org>
+;;; Copyright © 2024 Gabriel Wicki <gabriel <at> erlikon.ch>
+;;; Copyright © 2024 Richard Sent <richard <at> freakingpenguin.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -20,11 +22,15 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu services admin)
+  #:use-module (gnu system file-systems)
   #:use-module (gnu packages admin)
   #:use-module ((gnu packages base)
                 #:select (canonical-package findutils coreutils sed))
+  #:use-module (gnu packages file-systems)
   #:use-module (gnu packages certs)
+  #:use-module (gnu packages disk)
   #:use-module (gnu packages package-management)
+  #:use-module (gnu packages linux)
   #:use-module (gnu services)
   #:use-module (gnu services configuration)
   #:use-module (gnu services mcron)
@@ -93,7 +99,18 @@ (define-module (gnu services admin)
             unattended-upgrade-configuration-services-to-restart
             unattended-upgrade-configuration-system-expiration
             unattended-upgrade-configuration-maximum-duration
-            unattended-upgrade-configuration-log-file))
+            unattended-upgrade-configuration-log-file
+
+            unattended-upgrade-configuration-log-file
+
+            resize-fs-configuration
+            resize-fs-configuration?
+            resize-fs-configuration-parted
+            resize-fs-configuration-e2fsprogs
+            resize-fs-configuration-device
+            resize-fs-configuration-partition
+            resize-fs-configuration-end
+            resize-fs-service-type))
 
 ;;; Commentary:
 ;;;
@@ -550,4 +567,120 @@ (define unattended-upgrade-service-type
     "Periodically upgrade the system from the current configuration.")
    (default-value (unattended-upgrade-configuration))))
 
+;;;
+;;; Resize file system.
+;;;
+
+(define-record-type* <resize-fs-configuration>
+  resize-fs-configuration make-resize-fs-configuration
+  resize-fs-configuration?
+  (file-system    resize-fs-file-system
+                  (default #f))
+  (cloud-utils    resize-fs-cloud-utils
+                  (default cloud-utils))
+  (e2fsprogs      resize-fs-e2fsprogs
+                  (default e2fsprogs))
+  (btrfs-progs    resize-fs-btrfs-progs
+                  (default btrfs-progs))
+  (bcachefs-tools resize-fs-bcachefs-tools
+                  (default bcachefs-tools)))
+
+(define (resize-fs-shepherd-service config)
+  "Returns a <shepherd-service> for resize-fs-service for CONFIG."
+  (match-record config <resize-fs-configuration>
+                (file-system cloud-utils e2fsprogs btrfs-progs
+                             bcachefs-tools)
+    (let ((fs-spec (file-system->spec file-system)))
+      (shepherd-service
+       (documentation "Resize a file system. Intended for Guix Systems that
+are booted from a system image flashed onto a larger medium.")
+       ;; XXX: This could be extended with file-system info.
+       (provision '(resize-fs))
+       (requirement '(user-processes))
+       (one-shot? #t)
+       (respawn? #f)
+       (modules '((guix build utils)
+                  (gnu build file-systems)
+                  (gnu system file-systems)
+                  (ice-9 control)
+                  (ice-9 match)
+                  (ice-9 ftw)
+                  (ice-9 rdelim)
+                  (srfi srfi-34)))
+       (start (with-imported-modules (source-module-closure
+                                      '((guix build utils)
+                                        (gnu build file-systems)
+                                        (gnu system file-systems)))
+                #~(lambda _
+                    (use-modules (guix build utils)
+                                 (gnu build file-systems)
+                                 (gnu system file-systems)
+                                 (ice-9 control)
+                                 (ice-9 match)
+                                 (ice-9 ftw)
+                                 (ice-9 rdelim)
+                                 (srfi srfi-34))
+
+                    (define file-system
+                      (spec->file-system '#$fs-spec))
+
+                    ;; Shepherd recommends the start constructor takes <1
+                    ;; minute, canonicalize-device-spec will hang for up to
+                    ;; max-trials seconds (20 seconds) if an invalid device is
+                    ;; connected. Revisit this if max-trials increases.
+                    (define device (canonicalize-device-spec
+                                    (file-system-device file-system)))
+
+                    (define grow-partition-command
+                      (let* ((sysfs-device
+                              (string-append "/sys/class/block/"
+                                             (basename device)))
+                             (partition-number
+                              (with-input-from-file
+                                  (string-append sysfs-device
+                                                 "/partition")
+                                read-line))
+                             (parent (string-append
+                                      "/dev/"
+                                      (basename (dirname (readlink sysfs-device))))))
+                        (list #$(file-append cloud-utils "/bin/growpart")
+                              parent partition-number)))
+
+                    (define grow-filesystem-command
+                      (match (file-system-type file-system)
+                        ((or "ext2" "ext3" "ext4")
+                         (list #$(file-append e2fsprogs "/sbin/resize2fs") device))
+                        ("btrfs"
+                         (list #$(file-append btrfs-progs "/bin/btrfs")
+                               "filesystem" "resize" device))
+                        ("bcachefs"
+                         (list #$(file-append bcachefs-tools "/sbin/bcachefs")
+                               "device" "resize" device))
+                        (e (error "Unsupported filesystem type" e))))
+
+                    (let/ec return
+                      (guard (c ((and (invoke-error? c)
+                                      ;; growpart NOCHANGE exits with 1. It is
+                                      ;; unlikely the partition was resized
+                                      ;; while the file system was not. Just
+                                      ;; exit.
+                                      (equal? (invoke-error-exit-status c) 1))
+                                 (format (current-error-port)
+                                         "The device ~a is already resized.~%" device)
+                                 ;; Must return something or Shepherd considers
+                                 ;; the service perpetually starting.
+                                 (return 0)))
+                        (apply invoke grow-partition-command))
+                      (apply invoke grow-filesystem-command)))))))))
+
+(define resize-fs-service-type
+  (service-type
+   (name 'resize-fs)
+   (description "Resize a partition during boot.")
+   (extensions
+    (list
+     (service-extension shepherd-root-service-type
+                        (compose list resize-fs-shepherd-service))))
+   (default-value (resize-fs-configuration))))
+
 ;;; admin.scm ends here
-- 
2.46.0

Message #11 received at 74837 <at> debbugs.gnu.org (full text, mbox):

From: Richard Sent <richard <at> freakingpenguin.com>
To: 74837 <at> debbugs.gnu.org
Cc: Richard Sent <richard <at> freakingpenguin.com>,
 Gabriel Wicki <gabriel <at> erlikon.ch>
Subject: [PATCH 2/2] packages: cloud-utils: Add missing growpart programs to
 path.
Date: Thu, 12 Dec 2024 15:19:00 -0500

This fixes a bug in resize-fs-service.

* gnu/packages/admin.scm (cloud-utils): Augment growpart PATH with necessary
programs.

Change-Id: Idf0ca9e14e4c9d82208fbf715dd0b235cd5a371b
---
 gnu/packages/admin.scm | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/admin.scm b/gnu/packages/admin.scm
index b1889ceea0..d39bc5adc5 100644
--- a/gnu/packages/admin.scm
+++ b/gnu/packages/admin.scm
@@ -581,12 +581,21 @@ (define-public cloud-utils
                    ,(map dirname
                          (map (cut search-input-file inputs <>)
                               (list "bin/readlink"
-                                    "sbin/sfdisk")))))))))))
+                                    "sbin/sfdisk"
+                                    "bin/sed"
+                                    "bin/grep"
+                                    "bin/awk"
+                                    "bin/udevadm"
+                                    "bin/flock")))))))))))
     (inputs
      (list bash-minimal                 ;for wrap-program
-           coreutils                    ;for readlink
+           coreutils                    ;for readlink and cat
+           sed                          ;growpart
+           grep                         ;growpart
+           gawk                         ;awk for growpart
+           eudev                        ;udevadm for growpart
            python
-           util-linux))                 ;sfdisk for growpart
+           util-linux))                 ;sfdisk, flock, partx, blkid for growpart
     (home-page "https://launchpad.net/cloud-utils")
     (synopsis "Set of utilities for cloud computing environments")
     (description
-- 
2.46.0

Message #14 received at 74837 <at> debbugs.gnu.org (full text, mbox):

From: Richard Sent <richard <at> freakingpenguin.com>
To: 74837 <at> debbugs.gnu.org
Cc: Richard Sent <richard <at> freakingpenguin.com>
Subject: [PATCH v2 1/2] gnu: services: Add resize-fs-service.
Date: Thu, 12 Dec 2024 16:15:29 -0500

* gnu/services/admin.scm (resize-fs-configuration): New configuration
type.
(resize-fs-shepherd-service): New procedure.
(resize-fs-service-type): New variable.
* doc/guix.texi (Miscallaneous Services): Document it.

Change-Id: Icae2fefc9a8d936d4c3add47520258b341f689a4
---
Fixing up the export list.

 doc/guix.texi          |  50 ++++++++++++++++
 gnu/services/admin.scm | 133 ++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 182 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index a2915de954..5636eb23fb 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41891,6 +41891,56 @@ Miscellaneous Services
 
 @c End of auto-generated fail2ban documentation.
 
+@cindex resize-fs
+@subsubheading Resize File System service
+
+This service type lets you resize a live file system during boot, which
+can be convenient if a Guix image is flashed on an SD Card (e.g. for an
+embedded device) or uploaded to a VPS.  In both cases the medium the
+image will reside upon may be larger than the image you want to produce.
+
+For an embedded device booting from an SD card you may use something like:
+@lisp
+(service resize-fs-service-type
+  (resize-fs-configuration
+    (file-system
+     (device (file-system-label "root"))
+     (type "ext4"))))
+@end lisp
+
+Be extra cautious to use the correct device and type.  The service has
+little error handling of its own and relies on the underlying tools.
+Wrong use could end in loss of data or the corruption of the operating
+system.
+
+Partitions and file systems are grown to the maximum size available.
+File systems can only grow when they are on the last partition on a
+device and have empty space available.
+
+This service supports the ext2, ext3, ext4, btrfs, and bcachefs file
+systems.
+
+@table @asis
+
+@item @code{file-system} (default: @code{#f}) (type: file-system)
+The file-system object to resize.  This object must have the device and
+type fields set.  The others are ignored.
+
+@item @code{cloud-utils} (default: @code{cloud-utils}) (type: file-like)
+The cloud-utils package to use.
+
+@item @code{e2fsprogs} (default: @code{e2fsprogs}) (type: file-like)
+The e2fsprogs package to use, used for resizing ext2, ext3, and ext4
+file systems.
+
+@item @code{btrfs-progs} (default: @code{btrfs-progs}) (type: file-like)
+The btrfs-progs package to use, used for resizing the btrfs file system.
+
+@item @code{bcachefs-tools} (default: @code{bcachefs-tools}) (type: file-like)
+The bcachefs-tools package to use, used for resizing the bcachefs file system.
+
+@end table
+
 @cindex Backup
 @subsubheading Backup Services
 
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index 24ff659a01..a92b3b0ecc 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -3,6 +3,8 @@
 ;;; Copyright © 2016-2023 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2020 Brice Waegeneire <brice <at> waegenei.re>
 ;;; Copyright © 2023 Giacomo Leidi <goodoldpaul <at> autistici.org>
+;;; Copyright © 2024 Gabriel Wicki <gabriel <at> erlikon.ch>
+;;; Copyright © 2024 Richard Sent <richard <at> freakingpenguin.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -20,11 +22,15 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu services admin)
+  #:use-module (gnu system file-systems)
   #:use-module (gnu packages admin)
   #:use-module ((gnu packages base)
                 #:select (canonical-package findutils coreutils sed))
+  #:use-module (gnu packages file-systems)
   #:use-module (gnu packages certs)
+  #:use-module (gnu packages disk)
   #:use-module (gnu packages package-management)
+  #:use-module (gnu packages linux)
   #:use-module (gnu services)
   #:use-module (gnu services configuration)
   #:use-module (gnu services mcron)
@@ -93,7 +99,16 @@ (define-module (gnu services admin)
             unattended-upgrade-configuration-services-to-restart
             unattended-upgrade-configuration-system-expiration
             unattended-upgrade-configuration-maximum-duration
-            unattended-upgrade-configuration-log-file))
+            unattended-upgrade-configuration-log-file
+
+            resize-fs-service-type
+            resize-fs-configuration
+            resize-fs-configuration?
+            resize-fs-configuration-file-system
+            resize-fs-configuration-cloud-utils
+            resize-fs-configuration-e2fsprogs
+            resize-fs-configuration-btrfs-progs
+            resize-fs-configuration-bcachefs-tools))
 
 ;;; Commentary:
 ;;;
@@ -550,4 +565,120 @@ (define unattended-upgrade-service-type
     "Periodically upgrade the system from the current configuration.")
    (default-value (unattended-upgrade-configuration))))
 
+;;;
+;;; Resize file system.
+;;;
+
+(define-record-type* <resize-fs-configuration>
+  resize-fs-configuration make-resize-fs-configuration
+  resize-fs-configuration?
+  (file-system    resize-fs-file-system
+                  (default #f))
+  (cloud-utils    resize-fs-cloud-utils
+                  (default cloud-utils))
+  (e2fsprogs      resize-fs-e2fsprogs
+                  (default e2fsprogs))
+  (btrfs-progs    resize-fs-btrfs-progs
+                  (default btrfs-progs))
+  (bcachefs-tools resize-fs-bcachefs-tools
+                  (default bcachefs-tools)))
+
+(define (resize-fs-shepherd-service config)
+  "Returns a <shepherd-service> for resize-fs-service for CONFIG."
+  (match-record config <resize-fs-configuration>
+                (file-system cloud-utils e2fsprogs btrfs-progs
+                             bcachefs-tools)
+    (let ((fs-spec (file-system->spec file-system)))
+      (shepherd-service
+       (documentation "Resize a file system. Intended for Guix Systems that
+are booted from a system image flashed onto a larger medium.")
+       ;; XXX: This could be extended with file-system info.
+       (provision '(resize-fs))
+       (requirement '(user-processes))
+       (one-shot? #t)
+       (respawn? #f)
+       (modules '((guix build utils)
+                  (gnu build file-systems)
+                  (gnu system file-systems)
+                  (ice-9 control)
+                  (ice-9 match)
+                  (ice-9 ftw)
+                  (ice-9 rdelim)
+                  (srfi srfi-34)))
+       (start (with-imported-modules (source-module-closure
+                                      '((guix build utils)
+                                        (gnu build file-systems)
+                                        (gnu system file-systems)))
+                #~(lambda _
+                    (use-modules (guix build utils)
+                                 (gnu build file-systems)
+                                 (gnu system file-systems)
+                                 (ice-9 control)
+                                 (ice-9 match)
+                                 (ice-9 ftw)
+                                 (ice-9 rdelim)
+                                 (srfi srfi-34))
+
+                    (define file-system
+                      (spec->file-system '#$fs-spec))
+
+                    ;; Shepherd recommends the start constructor takes <1
+                    ;; minute, canonicalize-device-spec will hang for up to
+                    ;; max-trials seconds (20 seconds) if an invalid device is
+                    ;; connected. Revisit this if max-trials increases.
+                    (define device (canonicalize-device-spec
+                                    (file-system-device file-system)))
+
+                    (define grow-partition-command
+                      (let* ((sysfs-device
+                              (string-append "/sys/class/block/"
+                                             (basename device)))
+                             (partition-number
+                              (with-input-from-file
+                                  (string-append sysfs-device
+                                                 "/partition")
+                                read-line))
+                             (parent (string-append
+                                      "/dev/"
+                                      (basename (dirname (readlink sysfs-device))))))
+                        (list #$(file-append cloud-utils "/bin/growpart")
+                              parent partition-number)))
+
+                    (define grow-filesystem-command
+                      (match (file-system-type file-system)
+                        ((or "ext2" "ext3" "ext4")
+                         (list #$(file-append e2fsprogs "/sbin/resize2fs") device))
+                        ("btrfs"
+                         (list #$(file-append btrfs-progs "/bin/btrfs")
+                               "filesystem" "resize" device))
+                        ("bcachefs"
+                         (list #$(file-append bcachefs-tools "/sbin/bcachefs")
+                               "device" "resize" device))
+                        (e (error "Unsupported filesystem type" e))))
+
+                    (let/ec return
+                      (guard (c ((and (invoke-error? c)
+                                      ;; growpart NOCHANGE exits with 1. It is
+                                      ;; unlikely the partition was resized
+                                      ;; while the file system was not. Just
+                                      ;; exit.
+                                      (equal? (invoke-error-exit-status c) 1))
+                                 (format (current-error-port)
+                                         "The device ~a is already resized.~%" device)
+                                 ;; Must return something or Shepherd considers
+                                 ;; the service perpetually starting.
+                                 (return 0)))
+                        (apply invoke grow-partition-command))
+                      (apply invoke grow-filesystem-command)))))))))
+
+(define resize-fs-service-type
+  (service-type
+   (name 'resize-fs)
+   (description "Resize a partition during boot.")
+   (extensions
+    (list
+     (service-extension shepherd-root-service-type
+                        (compose list resize-fs-shepherd-service))))
+   (default-value (resize-fs-configuration))))
+
 ;;; admin.scm ends here

base-commit: a9003b8e6b40b59c9545ae87bb441d3549630db7
-- 
2.46.0

Message #17 received at 74837 <at> debbugs.gnu.org (full text, mbox):

From: Richard Sent <richard <at> freakingpenguin.com>
To: 74837 <at> debbugs.gnu.org
Cc: Richard Sent <richard <at> freakingpenguin.com>
Subject: [PATCH v2 2/2] packages: cloud-utils: Add missing growpart programs
 to path.
Date: Thu, 12 Dec 2024 16:15:30 -0500

This fixes a bug in resize-fs-service.

* gnu/packages/admin.scm (cloud-utils): Augment growpart PATH with necessary
programs.

Change-Id: Idf0ca9e14e4c9d82208fbf715dd0b235cd5a371b
---
 gnu/packages/admin.scm | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/admin.scm b/gnu/packages/admin.scm
index b1889ceea0..d39bc5adc5 100644
--- a/gnu/packages/admin.scm
+++ b/gnu/packages/admin.scm
@@ -581,12 +581,21 @@ (define-public cloud-utils
                    ,(map dirname
                          (map (cut search-input-file inputs <>)
                               (list "bin/readlink"
-                                    "sbin/sfdisk")))))))))))
+                                    "sbin/sfdisk"
+                                    "bin/sed"
+                                    "bin/grep"
+                                    "bin/awk"
+                                    "bin/udevadm"
+                                    "bin/flock")))))))))))
     (inputs
      (list bash-minimal                 ;for wrap-program
-           coreutils                    ;for readlink
+           coreutils                    ;for readlink and cat
+           sed                          ;growpart
+           grep                         ;growpart
+           gawk                         ;awk for growpart
+           eudev                        ;udevadm for growpart
            python
-           util-linux))                 ;sfdisk for growpart
+           util-linux))                 ;sfdisk, flock, partx, blkid for growpart
     (home-page "https://launchpad.net/cloud-utils")
     (synopsis "Set of utilities for cloud computing environments")
     (description
-- 
2.46.0

Message #20 received at 74837 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Richard Sent <richard <at> freakingpenguin.com>
Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, 74837 <at> debbugs.gnu.org
Subject: Re: [bug#74837] [PATCH v2 1/2] gnu: services: Add resize-fs-service.
Date: Sat, 14 Dec 2024 16:23:27 +0100

Hello,

Richard Sent <richard <at> freakingpenguin.com> skribis:

> * gnu/services/admin.scm (resize-fs-configuration): New configuration
> type.
> (resize-fs-shepherd-service): New procedure.
> (resize-fs-service-type): New variable.
> * doc/guix.texi (Miscallaneous Services): Document it.
>
> Change-Id: Icae2fefc9a8d936d4c3add47520258b341f689a4

Nice!  Overall LGTM.  Minor comments below.

> +@subsubheading Resize File System service
> +
> +This service type lets you resize a live file system during boot, which
> +can be convenient if a Guix image is flashed on an SD Card (e.g. for an
> +embedded device) or uploaded to a VPS.  In both cases the medium the
> +image will reside upon may be larger than the image you want to produce.
> +
> +For an embedded device booting from an SD card you may use something like:
> +@lisp
> +(service resize-fs-service-type
> +  (resize-fs-configuration
> +    (file-system
> +     (device (file-system-label "root"))
> +     (type "ext4"))))
> +@end lisp

I would avoid abbreviations as usual and go for
‘file-system-resizing-service-type’.  WDYT?

> +Be extra cautious to use the correct device and type.  The service has
> +little error handling of its own and relies on the underlying tools.
> +Wrong use could end in loss of data or the corruption of the operating
> +system.

Maybe wrap this paragraph in “@quotation Warning”.

> +@item @code{file-system} (default: @code{#f}) (type: file-system)
> +The file-system object to resize.  This object must have the device and
                                   ^
Maybe add “(@pxref{File Systems})”.

> +type fields set.  The others are ignored.

“the @code{device} and @code{type} fields set.  Other fields are
ignored.”

> +@item @code{cloud-utils} (default: @code{cloud-utils}) (type: file-like)
> +The cloud-utils package to use.

Maybe add a sentence explaining that ‘cloud-utils’ is used for its
‘growpart’ command.

I wonder if Guile-Parted could be used instead of ‘growpart’ (shouldn’t
be a blocker though).

> +                    (let/ec return
> +                      (guard (c ((and (invoke-error? c)
> +                                      ;; growpart NOCHANGE exits with 1. It is
> +                                      ;; unlikely the partition was resized
> +                                      ;; while the file system was not. Just
> +                                      ;; exit.
> +                                      (equal? (invoke-error-exit-status c) 1))
> +                                 (format (current-error-port)
> +                                         "The device ~a is already resized.~%" device)
> +                                 ;; Must return something or Shepherd considers
> +                                 ;; the service perpetually starting.
> +                                 (return 0)))
> +                        (apply invoke grow-partition-command))
> +                      (apply invoke grow-filesystem-command)))))))))

No need for ‘let/ec’ here, you can just return from the ‘guard’ handler.

The second patch LGTM, though perhaps it should come before this patch
since it fixes something that the resize service needs.

Could you send updated patches?

Thanks!

Ludo’.

Message #23 received at 74837 <at> debbugs.gnu.org (full text, mbox):

From: Richard Sent <richard <at> freakingpenguin.com>
To: 74837 <at> debbugs.gnu.org
Cc: Richard Sent <richard <at> freakingpenguin.com>
Subject: [PATCH v3 0/2] resize-file-system-service
Date: Sat, 14 Dec 2024 16:18:21 -0500

> I would avoid abbreviations as usual and go for
> ‘file-system-resizing-service-type’.  WDYT?

Sounds good to me.

> I wonder if Guile-Parted could be used instead of ‘growpart’ (shouldn’t
> be a blocker though).

This is an interesting idea. parted can tell us the file system type
at runtime, reducing the amount of information the user needs to
provide (at the cost of not catching invalid file-system-type values
at build-time.)

> No need for ‘let/ec’ here, you can just return from the ‘guard’ handler.

What would that look like? We don't want to capture an error in
grow-filesystem-command with an exit-status of 1, so it lives outside
the guard, and we don't want to raise an exception if the partition
was already resized.

(Side note, should Shepherd consider a service perpetually starting if
the start constructor exits without returning a value?)

V3 should have all other feedback implemented.

Richard Sent (2):
  packages: cloud-utils: Add missing growpart programs to path.
  gnu: services: Add resize-file-system-service.

 doc/guix.texi          |  54 +++++++++++++++++
 gnu/packages/admin.scm |  15 ++++-
 gnu/services/admin.scm | 133 ++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 198 insertions(+), 4 deletions(-)


base-commit: 0787a180b3ca64d3958438920ca1b58a2a27fba4
-- 
2.46.0

Message #26 received at 74837 <at> debbugs.gnu.org (full text, mbox):

From: Richard Sent <richard <at> freakingpenguin.com>
To: 74837 <at> debbugs.gnu.org
Cc: Richard Sent <richard <at> freakingpenguin.com>
Subject: [PATCH v3 1/2] packages: cloud-utils: Add missing growpart programs
 to path.
Date: Sat, 14 Dec 2024 16:18:22 -0500

This ensures growpart can run in environments that may not have a PATH
variable set.

* gnu/packages/admin.scm (cloud-utils): Augment growpart PATH with necessary
programs.

Change-Id: Idf0ca9e14e4c9d82208fbf715dd0b235cd5a371b
---
 gnu/packages/admin.scm | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/admin.scm b/gnu/packages/admin.scm
index b1889ceea0..d39bc5adc5 100644
--- a/gnu/packages/admin.scm
+++ b/gnu/packages/admin.scm
@@ -581,12 +581,21 @@ (define-public cloud-utils
                    ,(map dirname
                          (map (cut search-input-file inputs <>)
                               (list "bin/readlink"
-                                    "sbin/sfdisk")))))))))))
+                                    "sbin/sfdisk"
+                                    "bin/sed"
+                                    "bin/grep"
+                                    "bin/awk"
+                                    "bin/udevadm"
+                                    "bin/flock")))))))))))
     (inputs
      (list bash-minimal                 ;for wrap-program
-           coreutils                    ;for readlink
+           coreutils                    ;for readlink and cat
+           sed                          ;growpart
+           grep                         ;growpart
+           gawk                         ;awk for growpart
+           eudev                        ;udevadm for growpart
            python
-           util-linux))                 ;sfdisk for growpart
+           util-linux))                 ;sfdisk, flock, partx, blkid for growpart
     (home-page "https://launchpad.net/cloud-utils")
     (synopsis "Set of utilities for cloud computing environments")
     (description
-- 
2.46.0

Message #29 received at 74837 <at> debbugs.gnu.org (full text, mbox):

From: Richard Sent <richard <at> freakingpenguin.com>
To: 74837 <at> debbugs.gnu.org
Cc: Richard Sent <richard <at> freakingpenguin.com>
Subject: [PATCH v3 2/2] gnu: services: Add resize-file-system-service.
Date: Sat, 14 Dec 2024 16:18:23 -0500

* gnu/services/admin.scm (resize-file-system-configuration): New configuration
type.
(resize-file-system-shepherd-service): New procedure.
(resize-file-system-service-type): New variable.
* doc/guix.texi (Miscallaneous Services): Document it.

Change-Id: Icae2fefc9a8d936d4c3add47520258b341f689a4
---
 doc/guix.texi          |  54 +++++++++++++++++
 gnu/services/admin.scm | 133 ++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 186 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index a2915de954..cc19fb6cff 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41891,6 +41891,60 @@ Miscellaneous Services
 
 @c End of auto-generated fail2ban documentation.
 
+@cindex resize-file-system
+@subsubheading Resize File System Service
+
+This service type lets you resize a live file system during boot, which
+can be convenient if a Guix image is flashed on an SD Card (e.g. for an
+embedded device) or uploaded to a VPS.  In both cases the medium the
+image will reside upon may be larger than the image you want to produce.
+
+For an embedded device booting from an SD card you may use something like:
+@lisp
+(service resize-file-system-service-type
+         (resize-file-system-configuration
+          (file-system
+           (file-system (device (file-system-label "root"))
+                        (type "ext4")))))
+@end lisp
+
+@quotation Warning
+Be extra cautious to use the correct device and type.  The service has
+little error handling of its own and relies on the underlying tools.
+Wrong use could end in loss of data or the corruption of the operating
+system.
+@end quotation
+
+Partitions and file systems are grown to the maximum size available.
+File systems can only grow when they are on the last partition on a
+device and have empty space available.
+
+This service supports the ext2, ext3, ext4, btrfs, and bcachefs file
+systems.
+
+@table @asis
+
+@item @code{file-system} (default: @code{#f}) (type: file-system)
+The file-system object to resize (@pxref{File Systems}).  This object
+must have the @code{device} and @code{type} fields set.  Other fields
+are ignored.
+
+@item @code{cloud-utils} (default: @code{cloud-utils}) (type: file-like)
+The cloud-utils package to use.  This package is used for the
+@code{growpart} command.
+
+@item @code{e2fsprogs} (default: @code{e2fsprogs}) (type: file-like)
+The e2fsprogs package to use, used for resizing ext2, ext3, and ext4
+file systems.
+
+@item @code{btrfs-progs} (default: @code{btrfs-progs}) (type: file-like)
+The btrfs-progs package to use, used for resizing the btrfs file system.
+
+@item @code{bcachefs-tools} (default: @code{bcachefs-tools}) (type: file-like)
+The bcachefs-tools package to use, used for resizing the bcachefs file system.
+
+@end table
+
 @cindex Backup
 @subsubheading Backup Services
 
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index 24ff659a01..4a2f5cb12d 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -3,6 +3,8 @@
 ;;; Copyright © 2016-2023 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2020 Brice Waegeneire <brice <at> waegenei.re>
 ;;; Copyright © 2023 Giacomo Leidi <goodoldpaul <at> autistici.org>
+;;; Copyright © 2024 Gabriel Wicki <gabriel <at> erlikon.ch>
+;;; Copyright © 2024 Richard Sent <richard <at> freakingpenguin.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -20,11 +22,15 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu services admin)
+  #:use-module (gnu system file-systems)
   #:use-module (gnu packages admin)
   #:use-module ((gnu packages base)
                 #:select (canonical-package findutils coreutils sed))
+  #:use-module (gnu packages file-systems)
   #:use-module (gnu packages certs)
+  #:use-module (gnu packages disk)
   #:use-module (gnu packages package-management)
+  #:use-module (gnu packages linux)
   #:use-module (gnu services)
   #:use-module (gnu services configuration)
   #:use-module (gnu services mcron)
@@ -93,7 +99,16 @@ (define-module (gnu services admin)
             unattended-upgrade-configuration-services-to-restart
             unattended-upgrade-configuration-system-expiration
             unattended-upgrade-configuration-maximum-duration
-            unattended-upgrade-configuration-log-file))
+            unattended-upgrade-configuration-log-file
+
+            resize-file-system-service-type
+            resize-file-system-configuration
+            resize-file-system-configuration?
+            resize-file-system-configuration-file-system
+            resize-file-system-configuration-cloud-utils
+            resize-file-system-configuration-e2fsprogs
+            resize-file-system-configuration-btrfs-progs
+            resize-file-system-configuration-bcachefs-tools))
 
 ;;; Commentary:
 ;;;
@@ -550,4 +565,120 @@ (define unattended-upgrade-service-type
     "Periodically upgrade the system from the current configuration.")
    (default-value (unattended-upgrade-configuration))))
 
+;;;
+;;; Resize file system.
+;;;
+
+(define-record-type* <resize-file-system-configuration>
+  resize-file-system-configuration make-resize-file-system-configuration
+  resize-file-system-configuration?
+  (file-system    resize-file-system-file-system
+                  (default #f))
+  (cloud-utils    resize-file-system-cloud-utils
+                  (default cloud-utils))
+  (e2fsprogs      resize-file-system-e2fsprogs
+                  (default e2fsprogs))
+  (btrfs-progs    resize-file-system-btrfs-progs
+                  (default btrfs-progs))
+  (bcachefs-tools resize-file-system-bcachefs-tools
+                  (default bcachefs-tools)))
+
+(define (resize-file-system-shepherd-service config)
+  "Returns a <shepherd-service> for resize-file-system-service for CONFIG."
+  (match-record config <resize-file-system-configuration>
+                (file-system cloud-utils e2fsprogs btrfs-progs
+                             bcachefs-tools)
+    (let ((fs-spec (file-system->spec file-system)))
+      (shepherd-service
+       (documentation "Resize a file system. Intended for Guix Systems that
+are booted from a system image flashed onto a larger medium.")
+       ;; XXX: This could be extended with file-system info.
+       (provision '(resize-file-system))
+       (requirement '(user-processes))
+       (one-shot? #t)
+       (respawn? #f)
+       (modules '((guix build utils)
+                  (gnu build file-systems)
+                  (gnu system file-systems)
+                  (ice-9 control)
+                  (ice-9 match)
+                  (ice-9 ftw)
+                  (ice-9 rdelim)
+                  (srfi srfi-34)))
+       (start (with-imported-modules (source-module-closure
+                                      '((guix build utils)
+                                        (gnu build file-systems)
+                                        (gnu system file-systems)))
+                #~(lambda _
+                    (use-modules (guix build utils)
+                                 (gnu build file-systems)
+                                 (gnu system file-systems)
+                                 (ice-9 control)
+                                 (ice-9 match)
+                                 (ice-9 ftw)
+                                 (ice-9 rdelim)
+                                 (srfi srfi-34))
+
+                    (define file-system
+                      (spec->file-system '#$fs-spec))
+
+                    ;; Shepherd recommends the start constructor takes <1
+                    ;; minute, canonicalize-device-spec will hang for up to
+                    ;; max-trials seconds (20 seconds) if an invalid device is
+                    ;; connected. Revisit this if max-trials increases.
+                    (define device (canonicalize-device-spec
+                                    (file-system-device file-system)))
+
+                    (define grow-partition-command
+                      (let* ((sysfs-device
+                              (string-append "/sys/class/block/"
+                                             (basename device)))
+                             (partition-number
+                              (with-input-from-file
+                                  (string-append sysfs-device
+                                                 "/partition")
+                                read-line))
+                             (parent (string-append
+                                      "/dev/"
+                                      (basename (dirname (readlink sysfs-device))))))
+                        (list #$(file-append cloud-utils "/bin/growpart")
+                              parent partition-number)))
+
+                    (define grow-filesystem-command
+                      (match (file-system-type file-system)
+                        ((or "ext2" "ext3" "ext4")
+                         (list #$(file-append e2fsprogs "/sbin/resize2fs") device))
+                        ("btrfs"
+                         (list #$(file-append btrfs-progs "/bin/btrfs")
+                               "filesystem" "resize" device))
+                        ("bcachefs"
+                         (list #$(file-append bcachefs-tools "/sbin/bcachefs")
+                               "device" "resize" device))
+                        (e (error "Unsupported filesystem type" e))))
+
+                    (let/ec return
+                      (guard (c ((and (invoke-error? c)
+                                      ;; growpart NOCHANGE exits with 1. It is
+                                      ;; unlikely the partition was resized
+                                      ;; while the file system was not. Just
+                                      ;; exit.
+                                      (equal? (invoke-error-exit-status c) 1))
+                                 (format (current-error-port)
+                                         "The device ~a is already resized.~%" device)
+                                 ;; Must return something or Shepherd considers
+                                 ;; the service perpetually starting.
+                                 (return 0)))
+                        (apply invoke grow-partition-command))
+                      (apply invoke grow-filesystem-command)))))))))
+
+(define resize-file-system-service-type
+  (service-type
+   (name 'resize-file-system)
+   (description "Resize a partition and the underlying file system during boot.")
+   (extensions
+    (list
+     (service-extension shepherd-root-service-type
+                        (compose list resize-file-system-shepherd-service))))
+   (default-value (resize-file-system-configuration))))
+
 ;;; admin.scm ends here
-- 
2.46.0

Message #34 received at 74837-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Richard Sent <richard <at> freakingpenguin.com>
Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, 74837-done <at> debbugs.gnu.org
Subject: Re: [bug#74837] [PATCH v3 0/2] resize-file-system-service
Date: Mon, 16 Dec 2024 00:02:33 +0100

Hi,

Richard Sent <richard <at> freakingpenguin.com> skribis:

> What would that look like? We don't want to capture an error in
> grow-filesystem-command with an exit-status of 1, so it lives outside
> the guard, and we don't want to raise an exception if the partition
> was already resized.

Ah yes, that makes sense, sorry for misunderstanding.

> (Side note, should Shepherd consider a service perpetually starting if
> the start constructor exits without returning a value?)

A constructor must return exactly one value.  (Or it can throw an
exception, which is interpreted as failure to start.)

> V3 should have all other feedback implemented.
>
> Richard Sent (2):
>   packages: cloud-utils: Add missing growpart programs to path.
>   gnu: services: Add resize-file-system-service.

Perfect.  Applied, thanks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.