GNU bug report logs - #75045
[PATCH] services: restic-backup: Implement as a Shepherd timer.

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#75045: closed ([PATCH] services: restic-backup: Implement as
 a Shepherd timer.)
Date: Fri, 24 Jan 2025 23:02:02 +0000

[Message part 1 (text/plain, inline)]

Your message dated Sat, 25 Jan 2025 00:01:45 +0100
with message-id <875xm3ddeu.fsf <at> gnu.org>
and subject line Re: [bug#75045] [PATCH v2] services: restic-backup: Implement as a Shepherd timer.
has caused the debbugs.gnu.org bug report #75045,
regarding [PATCH] services: restic-backup: Implement as a Shepherd timer.
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
75045: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=75045
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Giacomo Leidi <goodoldpaul <at> autistici.org>
To: guix-patches <at> gnu.org
Cc: Giacomo Leidi <goodoldpaul <at> autistici.org>
Subject: [PATCH] services: restic-backup: Implement as a Shepherd timer.
Date: Mon, 23 Dec 2024 11:46:05 +0100

This patch implements restic backup with Shepherd services.  It is
supposed not to break any existing setup.

* gnu/services/backup.scm (restic-backup-job): Add Shepherd
configuration options;
(restic-backup-job->mcron-job): Replace with...;
(restic-job-log-file): New procedure;
(restic-backup-job->shepherd-service): New procedure;
(restic-backup-activation): New procedure;
(restic-backup-service-type): Replace mcron with Shepherd extension and add
activation extension hook.
* doc/guix.texi: Document it.

Change-Id: I66de3b6a1cb6177f9e4ee0c2acf3013ecbcdd338
---
 doc/guix.texi           |  36 +++++++++----
 gnu/services/backup.scm | 114 ++++++++++++++++++++++++++++++++++------
 2 files changed, 123 insertions(+), 27 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 57030102ca..f77b765933 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41987,19 +41987,16 @@ Miscellaneous Services
                                     "/etc/guix/signing-key.sec"))))))))))
 @end lisp
 
-Each @code{restic-backup-job} translates to an mcron job which sets the
+Each @code{restic-backup-job} translates to a Shepherd timer which sets the
 @env{RESTIC_PASSWORD} environment variable by reading the first line of
 @code{password-file} and runs @command{restic backup}, creating backups
 using rclone of all the files listed in the @code{files} field.
 
-The @code{restic-backup-service-type} installs as well @code{restic-guix}
-to the system profile, a @code{restic} utility wrapper that allows for easier
-interaction with the Guix configured backup jobs.  For example the following
-could be used to instantaneusly trigger a backup for the above shown
-configuration, without waiting for the scheduled job:
+The @code{restic-backup-service-type} provides the ability to instantaneously
+trigger a backup with the @code{trigger} Shepherd action:
 
 @example
-restic-guix backup remote-ftp
+sudo herd trigger remote-ftp-job
 @end example
 
 @c %start of fragment
@@ -42030,6 +42027,22 @@ Miscellaneous Services
 @item @code{user} (default: @code{"root"}) (type: string)
 The user used for running the current job.
 
+@item @code{group} (default: @code{"root"}) (type: string)
+The group used for running the current job.
+
+@item @code{log-file} (type: maybe-string)
+The file system path to the log file for this job.  By default the file will
+have the name of the job and be under @code{/var/log/restic-backup}.
+
+@item @code{max-duration} (type: maybe-number)
+The maximum duration in seconds that a job may last.  Past
+@code{max-duration} seconds, the job will forcefully terminated.
+
+@item @code{wait-for-termination?} (default: @code{#f}) (type: boolean)
+Wait until the job has finished before considering executing it again;
+otherwise, perform it strictly on every occurrence of event, at the risk of
+having multiple instances running concurrently.
+
 @item @code{repository} (type: string)
 The restic repository target of this job.
 
@@ -42042,9 +42055,12 @@ Miscellaneous Services
 for the current job.
 
 @item @code{schedule} (type: gexp-or-string)
-A string or a gexp that will be passed as time specification in the
-mcron job specification (@pxref{Syntax, mcron job specifications,,
-mcron,GNU <at> tie{}mcron}).
+A string or a gexp representing the frequency of the backup.  Gexp must
+evaluate to @code{calendar-event} records or to strings.  Strings must contain
+Vixie cron date lines.
+
+@item @code{requirement} (default: @code{'()}) (type: list-of-symbols)
+The list of Shepherd services that this backup job depends upon.
 
 @item @code{files} (default: @code{'()}) (type: list-of-lowerables)
 The list of files or directories to be backed up.  It must be a list of
diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm
index 555e9fc959..fc8934873b 100644
--- a/gnu/services/backup.scm
+++ b/gnu/services/backup.scm
@@ -18,9 +18,10 @@
 
 (define-module (gnu services backup)
   #:use-module (gnu packages backup)
+  #:use-module (gnu packages bash)
   #:use-module (gnu services)
   #:use-module (gnu services configuration)
-  #:use-module (gnu services mcron)
+  #:use-module (gnu services shepherd)
   #:use-module (guix build-system copy)
   #:use-module (guix gexp)
   #:use-module ((guix licenses)
@@ -33,11 +34,16 @@ (define-module (gnu services backup)
             restic-backup-job-fields
             restic-backup-job-restic
             restic-backup-job-user
+            restic-backup-job-group
+            restic-backup-job-log-file
+            restic-backup-job-max-duration
+            restic-backup-job-wait-for-termination?
             restic-backup-job-name
             restic-backup-job-repository
             restic-backup-job-password-file
             restic-backup-job-schedule
             restic-backup-job-files
+            restic-backup-job-requirement
             restic-backup-job-verbose?
             restic-backup-job-extra-flags
 
@@ -64,6 +70,12 @@ (define (lowerable? value)
 (define list-of-lowerables?
   (list-of lowerable?))
 
+(define list-of-symbols?
+  (list-of symbol?))
+
+(define-maybe string)
+(define-maybe number)
+
 (define-configuration/no-serialization restic-backup-job
   (restic
    (package restic)
@@ -71,6 +83,22 @@ (define-configuration/no-serialization restic-backup-job
   (user
    (string "root")
    "The user used for running the current job.")
+  (group
+   (string "root")
+   "The group used for running the current job.")
+  (log-file
+   (maybe-string)
+   "The file system path to the log file for this job.  By default the file will
+have the name of the job and be under @code{/var/log/restic-backup}.")
+  (max-duration
+   (maybe-number)
+   "The maximum duration in seconds that a job may last.  Past
+@code{max-duration} seconds, the job will forcefully terminated.")
+  (wait-for-termination?
+   (boolean #f)
+   "Wait until the job has finished before considering executing it again;
+otherwise, perform it strictly on every occurrence of event, at the risk of
+having multiple instances running concurrently.")
   (name
    (string)
    "A string denoting a name for this job.")
@@ -84,9 +112,12 @@ (define-configuration/no-serialization restic-backup-job
 current job.")
   (schedule
    (gexp-or-string)
-   "A string or a gexp that will be passed as time specification in the mcron
-job specification (@pxref{Syntax, mcron job specifications,, mcron,
-GNU <at> tie{}mcron}).")
+   "A string or a gexp representing the frequency of the backup.  Gexp must
+evaluate to @code{calendar-event} records or to strings.  Strings must contain
+Vixie cron date lines.")
+  (requirement
+   (list-of-symbols '())
+   "The list of Shepherd services that this backup job depends upon.")
   (files
    (list-of-lowerables '())
    "The list of files or directories to be backed up.  It must be a list of
@@ -175,16 +206,56 @@ (define (restic-guix jobs)
 
        (main (command-line)))))
 
-(define (restic-backup-job->mcron-job config)
-  (let ((user
-         (restic-backup-job-user config))
-        (schedule
-         (restic-backup-job-schedule config))
-        (name
-         (restic-backup-job-name config)))
-    #~(job #$schedule
-           #$(string-append "restic-guix backup " name)
-           #:user #$user)))
+(define (restic-job-log-file job)
+  (let ((name (restic-backup-job-name job))
+        (log-file (restic-backup-job-log-file job)))
+    (if (maybe-value-set? log-file)
+        log-file
+        (string-append "/var/log/restic-backup/" name ".log"))))
+
+(define (restic-backup-job->shepherd-service config)
+  (let ((schedule (restic-backup-job-schedule config))
+        (name (restic-backup-job-name config))
+        (user (restic-backup-job-user config))
+        (group (restic-backup-job-group config))
+        (max-duration (restic-backup-job-max-duration config))
+        (wait-for-termination? (restic-backup-job-wait-for-termination? config))
+        (log-file (restic-job-log-file config))
+        (requirement (restic-backup-job-requirement config)))
+    (shepherd-service (provision `(,(string->symbol
+                                     (string-append name "-job"))))
+                      (requirement
+                       `(user-processes file-systems ,@requirement))
+                      (documentation
+                       "Run @code{restic} backed backups on a regular basis.")
+                      (modules '((shepherd service timer)))
+                      (start
+                       #~(make-timer-constructor
+                          (if (string? #$schedule)
+                              (cron-string->calendar-event #$schedule)
+                              #$schedule)
+                          (command
+                           (list
+                            (string-append #+bash-minimal "/bin/bash")
+                            "-l" "-c"
+                            (string-append "restic-guix backup " #$name))
+                           #:user #$user
+                           #:group #$group
+                           #:environment-variables
+                           (list
+                            (string-append
+                             "HOME=" (passwd:dir (getpwnam #$user)))))
+                          #:log-file #$log-file
+                          #:wait-for-termination? #$wait-for-termination?
+                          #:max-duration #$(and (maybe-value-set? max-duration)
+                                                max-duration)))
+                      (stop
+                       #~(make-timer-destructor))
+                      (actions (list (shepherd-action
+                                      (name 'trigger)
+                                      (documentation "Manually trigger a backup,
+without waiting for the scheduled time.")
+                                      (procedure #~trigger-timer)))))))
 
 (define (restic-guix-wrapper-package jobs)
   (package
@@ -212,15 +283,24 @@ (define restic-backup-service-profile
          (restic-guix-wrapper-package jobs))
         '())))
 
+(define (restic-backup-activation config)
+  #~(for-each
+     (lambda (log-file)
+       (mkdir-p (dirname log-file)))
+     (list #$@(map restic-job-log-file
+                   (restic-backup-configuration-jobs config)))))
+
 (define restic-backup-service-type
   (service-type (name 'restic-backup)
                 (extensions
                  (list
+                  (service-extension activation-service-type
+                                     restic-backup-activation)
                   (service-extension profile-service-type
                                      restic-backup-service-profile)
-                  (service-extension mcron-service-type
+                  (service-extension shepherd-root-service-type
                                      (lambda (config)
-                                       (map restic-backup-job->mcron-job
+                                       (map restic-backup-job->shepherd-service
                                             (restic-backup-configuration-jobs
                                              config))))))
                 (compose concatenate)
@@ -232,5 +312,5 @@ (define restic-backup-service-type
                                   jobs)))))
                 (default-value (restic-backup-configuration))
                 (description
-                 "This service configures @code{mcron} jobs for running backups
+                 "This service configures @code{Shepherd} timers for running backups
 with @code{restic}.")))

base-commit: 2743faebb2893f65fb29a5cfd55c72a66a2b98a9
-- 
2.46.0

[Message part 3 (message/rfc822, inline)]

From: Ludovic Courtès <ludo <at> gnu.org>
To: Giacomo Leidi <goodoldpaul <at> autistici.org>
Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, 75045-done <at> debbugs.gnu.org
Subject: Re: [bug#75045] [PATCH v2] services: restic-backup: Implement as a
 Shepherd timer.
Date: Sat, 25 Jan 2025 00:01:45 +0100

[Message part 4 (text/plain, inline)]

Giacomo Leidi <goodoldpaul <at> autistici.org> skribis:

> This patch implements restic backup with Shepherd services.  It is
> supposed not to break any existing setup.
>
> * gnu/services/backup.scm (restic-backup-job): Add Shepherd
> configuration options;
> (restic-backup-job->mcron-job): Replace with...;
> (restic-job-log-file): New procedure;
> (restic-backup-job->shepherd-service): New procedure;
> (restic-backup-activation): New procedure;
> (restic-backup-service-type): Replace mcron with Shepherd extension and add
> activation extension hook.
> * doc/guix.texi: Document it.
>
> Change-Id: I66de3b6a1cb6177f9e4ee0c2acf3013ecbcdd338

Applied with the cosmetic change below, thanks!

I just realized there’s no test for this service, so let’s everything
still works well.  Would be nice to add a test.

> +                            ;; We go through bash, instead of executing
> +                            ;; restic-guix directly, because the login shell
> +                            ;; gives us the correct user environment that some
> +                            ;; backends require, such as rclone.
> +                            (string-append #+bash-minimal "/bin/bash")
> +                            "-l" "-c"
> +                            (string-append "restic-guix backup " #$name))

Thanks for adding this comment.

Ludo’.

[Message part 5 (text/x-patch, inline)]

diff --git a/doc/guix.texi b/doc/guix.texi
index d18209f288..9a53bdcd37 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -42755,7 +42755,7 @@ Miscellaneous Services
 
 @item @code{log-file} (type: maybe-string)
 The file system path to the log file for this job.  By default the file will
-have be @file{/var/log/restic-backup/JOB-NAME.log}, where @code{JOB-NAME} is the
+have be @file{/var/log/restic-backup/@var{job-name}.log}, where @var{job-name} is the
 name defined in the @code{name} field.
 
 @item @code{max-duration} (type: maybe-number)
diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm
index 3dda6ca370..6cada58b10 100644
--- a/gnu/services/backup.scm
+++ b/gnu/services/backup.scm
@@ -89,7 +89,7 @@ (define-configuration/no-serialization restic-backup-job
   (log-file
    (maybe-string)
    "The file system path to the log file for this job.  By default the file will
-have be @file{/var/log/restic-backup/JOB-NAME.log}, where @code{JOB-NAME} is the
+have be @file{/var/log/restic-backup/@var{job-name}.log}, where @var{job-name} is the
 name defined in the @code{name} field.")
   (max-duration
    (maybe-number)
@@ -316,5 +316,5 @@ (define restic-backup-service-type
                                   jobs)))))
                 (default-value (restic-backup-configuration))
                 (description
-                 "This service configures @code{Shepherd} timers for running backups
+                 "This service configures Shepherd timers for running backups
 with restic.")))

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.