From unknown Tue Jun 17 01:48:49 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#75045] [PATCH] services: restic-backup: Implement as a Shepherd timer. Resent-From: Giacomo Leidi Original-Sender: "Debbugs-submit" Resent-CC: ludo@gnu.org, maxim.cournoyer@gmail.com, guix-patches@gnu.org Resent-Date: Mon, 23 Dec 2024 10:47:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 75045 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 75045@debbugs.gnu.org Cc: Giacomo Leidi , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Maxim Cournoyer X-Debbugs-Original-To: guix-patches@gnu.org X-Debbugs-Original-Xcc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Maxim Cournoyer Received: via spool by submit@debbugs.gnu.org id=B.173495080623966 (code B ref -1); Mon, 23 Dec 2024 10:47:02 +0000 Received: (at submit) by debbugs.gnu.org; 23 Dec 2024 10:46:46 +0000 Received: from localhost ([127.0.0.1]:53469 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tPfxV-0006EU-KK for submit@debbugs.gnu.org; Mon, 23 Dec 2024 05:46:46 -0500 Received: from lists.gnu.org ([209.51.188.17]:40722) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tPfxS-0006EK-V4 for submit@debbugs.gnu.org; Mon, 23 Dec 2024 05:46:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tPfxS-0006tn-PF for guix-patches@gnu.org; Mon, 23 Dec 2024 05:46:42 -0500 Received: from confino.investici.org ([93.190.126.19]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tPfxP-0003gD-NF for guix-patches@gnu.org; Mon, 23 Dec 2024 05:46:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=autistici.org; s=stigmate; t=1734950794; bh=rSJVA8TCLbx9MDrPZX5JM4ksnkPmSsxTJSgSPHPdU70=; h=From:To:Cc:Subject:Date:From; b=VxYTW+YUFQxlKnOcEpMYG/o8/WTqizDDm2HYIybcfuYoS214q+8ayxeN6LOoR7ETW UhocFXvXz/2JmunYjPC3Qx0SluDRkOAoFwBZDR9MJxrxyobDYlXiOUXXYCO33/ZOHf /qTBRTSL4f1Wlv9nYbeqseMFN55P5J0iE7MWU0J0= Received: from mx1.investici.org (unknown [127.0.0.1]) by confino.investici.org (Postfix) with ESMTP id 4YGvpL679Kz10wk; Mon, 23 Dec 2024 10:46:34 +0000 (UTC) Received: from [93.190.126.19] (mx1.investici.org [93.190.126.19]) (Authenticated sender: goodoldpaul@autistici.org) by localhost (Postfix) with ESMTPSA id 4YGvpL53vBz10wf; Mon, 23 Dec 2024 10:46:34 +0000 (UTC) From: Giacomo Leidi Date: Mon, 23 Dec 2024 11:46:05 +0100 Message-ID: X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=93.190.126.19; envelope-from=goodoldpaul@autistici.org; helo=confino.investici.org X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) This patch implements restic backup with Shepherd services. It is supposed not to break any existing setup. * gnu/services/backup.scm (restic-backup-job): Add Shepherd configuration options; (restic-backup-job->mcron-job): Replace with...; (restic-job-log-file): New procedure; (restic-backup-job->shepherd-service): New procedure; (restic-backup-activation): New procedure; (restic-backup-service-type): Replace mcron with Shepherd extension and add activation extension hook. * doc/guix.texi: Document it. Change-Id: I66de3b6a1cb6177f9e4ee0c2acf3013ecbcdd338 --- doc/guix.texi | 36 +++++++++---- gnu/services/backup.scm | 114 ++++++++++++++++++++++++++++++++++------ 2 files changed, 123 insertions(+), 27 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 57030102ca..f77b765933 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -41987,19 +41987,16 @@ Miscellaneous Services "/etc/guix/signing-key.sec")))))))))) @end lisp -Each @code{restic-backup-job} translates to an mcron job which sets the +Each @code{restic-backup-job} translates to a Shepherd timer which sets the @env{RESTIC_PASSWORD} environment variable by reading the first line of @code{password-file} and runs @command{restic backup}, creating backups using rclone of all the files listed in the @code{files} field. -The @code{restic-backup-service-type} installs as well @code{restic-guix} -to the system profile, a @code{restic} utility wrapper that allows for easier -interaction with the Guix configured backup jobs. For example the following -could be used to instantaneusly trigger a backup for the above shown -configuration, without waiting for the scheduled job: +The @code{restic-backup-service-type} provides the ability to instantaneously +trigger a backup with the @code{trigger} Shepherd action: @example -restic-guix backup remote-ftp +sudo herd trigger remote-ftp-job @end example @c %start of fragment @@ -42030,6 +42027,22 @@ Miscellaneous Services @item @code{user} (default: @code{"root"}) (type: string) The user used for running the current job. +@item @code{group} (default: @code{"root"}) (type: string) +The group used for running the current job. + +@item @code{log-file} (type: maybe-string) +The file system path to the log file for this job. By default the file will +have the name of the job and be under @code{/var/log/restic-backup}. + +@item @code{max-duration} (type: maybe-number) +The maximum duration in seconds that a job may last. Past +@code{max-duration} seconds, the job will forcefully terminated. + +@item @code{wait-for-termination?} (default: @code{#f}) (type: boolean) +Wait until the job has finished before considering executing it again; +otherwise, perform it strictly on every occurrence of event, at the risk of +having multiple instances running concurrently. + @item @code{repository} (type: string) The restic repository target of this job. @@ -42042,9 +42055,12 @@ Miscellaneous Services for the current job. @item @code{schedule} (type: gexp-or-string) -A string or a gexp that will be passed as time specification in the -mcron job specification (@pxref{Syntax, mcron job specifications,, -mcron,GNU@tie{}mcron}). +A string or a gexp representing the frequency of the backup. Gexp must +evaluate to @code{calendar-event} records or to strings. Strings must contain +Vixie cron date lines. + +@item @code{requirement} (default: @code{'()}) (type: list-of-symbols) +The list of Shepherd services that this backup job depends upon. @item @code{files} (default: @code{'()}) (type: list-of-lowerables) The list of files or directories to be backed up. It must be a list of diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm index 555e9fc959..fc8934873b 100644 --- a/gnu/services/backup.scm +++ b/gnu/services/backup.scm @@ -18,9 +18,10 @@ (define-module (gnu services backup) #:use-module (gnu packages backup) + #:use-module (gnu packages bash) #:use-module (gnu services) #:use-module (gnu services configuration) - #:use-module (gnu services mcron) + #:use-module (gnu services shepherd) #:use-module (guix build-system copy) #:use-module (guix gexp) #:use-module ((guix licenses) @@ -33,11 +34,16 @@ (define-module (gnu services backup) restic-backup-job-fields restic-backup-job-restic restic-backup-job-user + restic-backup-job-group + restic-backup-job-log-file + restic-backup-job-max-duration + restic-backup-job-wait-for-termination? restic-backup-job-name restic-backup-job-repository restic-backup-job-password-file restic-backup-job-schedule restic-backup-job-files + restic-backup-job-requirement restic-backup-job-verbose? restic-backup-job-extra-flags @@ -64,6 +70,12 @@ (define (lowerable? value) (define list-of-lowerables? (list-of lowerable?)) +(define list-of-symbols? + (list-of symbol?)) + +(define-maybe string) +(define-maybe number) + (define-configuration/no-serialization restic-backup-job (restic (package restic) @@ -71,6 +83,22 @@ (define-configuration/no-serialization restic-backup-job (user (string "root") "The user used for running the current job.") + (group + (string "root") + "The group used for running the current job.") + (log-file + (maybe-string) + "The file system path to the log file for this job. By default the file will +have the name of the job and be under @code{/var/log/restic-backup}.") + (max-duration + (maybe-number) + "The maximum duration in seconds that a job may last. Past +@code{max-duration} seconds, the job will forcefully terminated.") + (wait-for-termination? + (boolean #f) + "Wait until the job has finished before considering executing it again; +otherwise, perform it strictly on every occurrence of event, at the risk of +having multiple instances running concurrently.") (name (string) "A string denoting a name for this job.") @@ -84,9 +112,12 @@ (define-configuration/no-serialization restic-backup-job current job.") (schedule (gexp-or-string) - "A string or a gexp that will be passed as time specification in the mcron -job specification (@pxref{Syntax, mcron job specifications,, mcron, -GNU@tie{}mcron}).") + "A string or a gexp representing the frequency of the backup. Gexp must +evaluate to @code{calendar-event} records or to strings. Strings must contain +Vixie cron date lines.") + (requirement + (list-of-symbols '()) + "The list of Shepherd services that this backup job depends upon.") (files (list-of-lowerables '()) "The list of files or directories to be backed up. It must be a list of @@ -175,16 +206,56 @@ (define (restic-guix jobs) (main (command-line))))) -(define (restic-backup-job->mcron-job config) - (let ((user - (restic-backup-job-user config)) - (schedule - (restic-backup-job-schedule config)) - (name - (restic-backup-job-name config))) - #~(job #$schedule - #$(string-append "restic-guix backup " name) - #:user #$user))) +(define (restic-job-log-file job) + (let ((name (restic-backup-job-name job)) + (log-file (restic-backup-job-log-file job))) + (if (maybe-value-set? log-file) + log-file + (string-append "/var/log/restic-backup/" name ".log")))) + +(define (restic-backup-job->shepherd-service config) + (let ((schedule (restic-backup-job-schedule config)) + (name (restic-backup-job-name config)) + (user (restic-backup-job-user config)) + (group (restic-backup-job-group config)) + (max-duration (restic-backup-job-max-duration config)) + (wait-for-termination? (restic-backup-job-wait-for-termination? config)) + (log-file (restic-job-log-file config)) + (requirement (restic-backup-job-requirement config))) + (shepherd-service (provision `(,(string->symbol + (string-append name "-job")))) + (requirement + `(user-processes file-systems ,@requirement)) + (documentation + "Run @code{restic} backed backups on a regular basis.") + (modules '((shepherd service timer))) + (start + #~(make-timer-constructor + (if (string? #$schedule) + (cron-string->calendar-event #$schedule) + #$schedule) + (command + (list + (string-append #+bash-minimal "/bin/bash") + "-l" "-c" + (string-append "restic-guix backup " #$name)) + #:user #$user + #:group #$group + #:environment-variables + (list + (string-append + "HOME=" (passwd:dir (getpwnam #$user))))) + #:log-file #$log-file + #:wait-for-termination? #$wait-for-termination? + #:max-duration #$(and (maybe-value-set? max-duration) + max-duration))) + (stop + #~(make-timer-destructor)) + (actions (list (shepherd-action + (name 'trigger) + (documentation "Manually trigger a backup, +without waiting for the scheduled time.") + (procedure #~trigger-timer))))))) (define (restic-guix-wrapper-package jobs) (package @@ -212,15 +283,24 @@ (define restic-backup-service-profile (restic-guix-wrapper-package jobs)) '()))) +(define (restic-backup-activation config) + #~(for-each + (lambda (log-file) + (mkdir-p (dirname log-file))) + (list #$@(map restic-job-log-file + (restic-backup-configuration-jobs config))))) + (define restic-backup-service-type (service-type (name 'restic-backup) (extensions (list + (service-extension activation-service-type + restic-backup-activation) (service-extension profile-service-type restic-backup-service-profile) - (service-extension mcron-service-type + (service-extension shepherd-root-service-type (lambda (config) - (map restic-backup-job->mcron-job + (map restic-backup-job->shepherd-service (restic-backup-configuration-jobs config)))))) (compose concatenate) @@ -232,5 +312,5 @@ (define restic-backup-service-type jobs))))) (default-value (restic-backup-configuration)) (description - "This service configures @code{mcron} jobs for running backups + "This service configures @code{Shepherd} timers for running backups with @code{restic}."))) base-commit: 2743faebb2893f65fb29a5cfd55c72a66a2b98a9 -- 2.46.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 23 06:26:30 2024 Received: (at control) by debbugs.gnu.org; 23 Dec 2024 11:26:30 +0000 Received: from localhost ([127.0.0.1]:53550 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tPgZy-00089J-HE for submit@debbugs.gnu.org; Mon, 23 Dec 2024 06:26:30 -0500 Received: from confino.investici.org ([93.190.126.19]:27483) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tPgZx-00089C-8M for control@debbugs.gnu.org; Mon, 23 Dec 2024 06:26:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=autistici.org; s=stigmate; t=1734953188; bh=kcYPzq10ZS7hPgfmaCaLRTsSuizGr65MwpArJHalXrw=; h=Date:To:From:Subject:From; b=C/uIGDgTg+I6qtanXBhRXrvSXXSFNXYpKNvCNSPjrEql55elogWd/Id5Mcn76xsZW VSL2ygaOFYG9yukpniDHWfAapvh80GVp9EsDDxtJTDsqGwAbRI2R/vYHyjJ3kzqwO5 byYjjEyjATaTnIL4F7n6PCkG9qV39xG+FgWQdkgo= Received: from mx1.investici.org (unknown [127.0.0.1]) by confino.investici.org (Postfix) with ESMTP id 4YGwhN5k4Rz10w5 for ; Mon, 23 Dec 2024 11:26:28 +0000 (UTC) Received: from [93.190.126.19] (mx1.investici.org [93.190.126.19]) (Authenticated sender: goodoldpaul@autistici.org) by localhost (Postfix) with ESMTPSA id 4YGwhN5Czbz10vx for ; Mon, 23 Dec 2024 11:26:28 +0000 (UTC) Message-ID: <581fb166-b3b6-4eab-a349-ff7a75798873@autistici.org> Date: Mon, 23 Dec 2024 12:26:28 +0100 MIME-Version: 1.0 User-Agent: Icedove Daily Content-Language: en-US To: control@debbugs.gnu.org From: paul Subject: block 72803 by 75045 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) block 72803 by 75045 From unknown Tue Jun 17 01:48:49 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#75045] [PATCH] services: restic-backup: Implement as a Shepherd timer. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 24 Dec 2024 10:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75045 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Giacomo Leidi Cc: Maxim Cournoyer , 75045@debbugs.gnu.org Received: via spool by 75045-submit@debbugs.gnu.org id=B75045.1735036964620 (code B ref 75045); Tue, 24 Dec 2024 10:43:02 +0000 Received: (at 75045) by debbugs.gnu.org; 24 Dec 2024 10:42:44 +0000 Received: from localhost ([127.0.0.1]:59638 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQ2NA-00009w-1j for submit@debbugs.gnu.org; Tue, 24 Dec 2024 05:42:44 -0500 Received: from eggs.gnu.org ([209.51.188.92]:36120) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tQ2N6-00009d-9M for 75045@debbugs.gnu.org; Tue, 24 Dec 2024 05:42:42 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tQ2Ks-0000S1-Sz; Tue, 24 Dec 2024 05:40:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=55p6zUpTXy0yyvL3ZLv7MqnbwCIJL6VEk3fWg7KSDTA=; b=GhlDPQCiI1VfedkZeXa7 Vi6m14cRRxXje2wt0bSRDphzI2x2jOr0qkkT1JrOUV58YXsrOpNo0ulDExkQEVelD9O92SXtCmF0A Vha3/K3XmCeBVDpsHh5pAlCbcJ+FqVdxPG5ENNRJWWJY4awm9LGjBX4u86VDRofrgPfhZqItqVVRm n6uk4nep9uOg/NNc6RdxQY7dgZtcwD17eeSEnWva+Yo7o8gTvsd+P7yoWRRI+WMDjtBLFWJ71HH7s ZPHzsBkB/ErMQWwq8aRugIFpGM/4YnTT7944gi3hEbI0om9diqZCid5lV6IaokSUL1uJ35tgFJSNs bjB+QAB0Wu997A==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= In-Reply-To: (Giacomo Leidi's message of "Mon, 23 Dec 2024 11:46:05 +0100") References: Date: Tue, 24 Dec 2024 11:40:19 +0100 Message-ID: <87ikr95ppo.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Giacomo, Giacomo Leidi skribis: > This patch implements restic backup with Shepherd services. It is > supposed not to break any existing setup. > > * gnu/services/backup.scm (restic-backup-job): Add Shepherd > configuration options; > (restic-backup-job->mcron-job): Replace with...; > (restic-job-log-file): New procedure; > (restic-backup-job->shepherd-service): New procedure; > (restic-backup-activation): New procedure; > (restic-backup-service-type): Replace mcron with Shepherd extension and a= dd > activation extension hook. > * doc/guix.texi: Document it. > > Change-Id: I66de3b6a1cb6177f9e4ee0c2acf3013ecbcdd338 Woo, nice! As mentioned in , I think we should postpone a little bit since these new features won=E2=80=99t work for peopl= e who haven=E2=80=99t rebooted into Shepherd 1.0, and they=E2=80=99ll get possibl= y confusing messages when reconfiguring. How we should postpone, I=E2=80=99m not sure. The conservative approach wo= uld be to wait until after the next Guix release, but that doesn=E2=80=99t sound reasonable=E2=80=A6 One month after the Shepherd upgrade, which would be Jan. 9th? It=E2=80=99s common for servers to have much longer uptimes thou= gh, plus there=E2=80=99s end-of-year vacation here. > + (log-file > + (maybe-string) > + "The file system path to the log file for this job. By default the f= ile will > +have the name of the job and be under @code{/var/log/restic-backup}.") Rather @file, with the =E2=80=9C.log=E2=80=9D suffix too, if I=E2=80=99m no= t mistaken. > + (max-duration > + (maybe-number) > + "The maximum duration in seconds that a job may last. Past > +@code{max-duration} seconds, the job will forcefully terminated.") s/will/is/ > + (shepherd-service (provision `(,(string->symbol > + (string-append name "-job")))) I would tend to not add the =E2=80=9C-job=E2=80=9D suffix, but that=E2=80= =99s a matter of taste! > + (command > + (list > + (string-append #+bash-minimal "/bin/bash") > + "-l" "-c" > + (string-append "restic-guix backup " #$name)) Why go through bash? Would it be possible to execute =E2=80=98restic-guix= =E2=80=99 directly? > (description > - "This service configures @code{mcron} jobs for running = backups > + "This service configures @code{Shepherd} timers for run= ning backups > with @code{restic}."))) You can drop @code here. From unknown Tue Jun 17 01:48:49 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#75045] [PATCH] services: restic-backup: Implement as a Shepherd timer. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 17 Jan 2025 20:29:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75045 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Giacomo Leidi Cc: 75045@debbugs.gnu.org, Maxim Cournoyer Received: via spool by 75045-submit@debbugs.gnu.org id=B75045.173714571620224 (code B ref 75045); Fri, 17 Jan 2025 20:29:02 +0000 Received: (at 75045) by debbugs.gnu.org; 17 Jan 2025 20:28:36 +0000 Received: from localhost ([127.0.0.1]:38949 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tYsxH-0005G5-CQ for submit@debbugs.gnu.org; Fri, 17 Jan 2025 15:28:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41874) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tYsxE-0005Fn-VW for 75045@debbugs.gnu.org; Fri, 17 Jan 2025 15:28:33 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tYsx9-0000QA-FH; Fri, 17 Jan 2025 15:28:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=X1A9pp3vZp+DWWAs4UqDVFOy+WeCKdNJX7BWQRSM6bU=; b=rNnPVug/mdUI5jL4ib/M otmAs8YESCb2EpJhbYW4Dz5F4yszFawdsrIHEiYW4baYJGTR2fCi9bbjZkKpPQFv2IccOMPShGHqW aYPD5K7/5/0/qQFl5ol+34q7608TsfVIo9jx/6EJFQmyEgLIiMlaWoOXVjH3RQ6FvhP9BBN2nCIBv voBnd1KXD35WtHE0cXoBew94/fiux4XNF3ZPoBtLRMvGDq+ddBFmGUk+3vcbipLwu7CxjOemtFlxI jYUxp4MHqRQuCXtsJlq9TonCSBzSrgUK9Arz7FtzVyRObIrYYp6kChz5X/15QyWymuWSEclABEfzR PDww78l5rvlipw==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= In-Reply-To: <87ikr95ppo.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Tue, 24 Dec 2024 11:40:19 +0100") References: <87ikr95ppo.fsf@gnu.org> Date: Fri, 17 Jan 2025 21:28:22 +0100 Message-ID: <87sephuqvt.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, Ludovic Court=C3=A8s skribis: > As mentioned in , I think we should > postpone a little bit since these new features won=E2=80=99t work for peo= ple who > haven=E2=80=99t rebooted into Shepherd 1.0, and they=E2=80=99ll get possi= bly confusing > messages when reconfiguring. > > How we should postpone, I=E2=80=99m not sure. The conservative approach = would > be to wait until after the next Guix release, but that doesn=E2=80=99t so= und > reasonable=E2=80=A6 One month after the Shepherd upgrade, which would be > Jan. 9th? It=E2=80=99s common for servers to have much longer uptimes th= ough, > plus there=E2=80=99s end-of-year vacation here. Now that is merged, I think we can gradually start migrating services such as this one. Feel free to send v2! Ludo=E2=80=99. From unknown Tue Jun 17 01:48:49 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#75045] [PATCH] services: restic-backup: Implement as a Shepherd timer. Resent-From: paul Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 19 Jan 2025 22:03:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75045 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Maxim Cournoyer , 75045@debbugs.gnu.org Received: via spool by 75045-submit@debbugs.gnu.org id=B75045.173732415225680 (code B ref 75045); Sun, 19 Jan 2025 22:03:01 +0000 Received: (at 75045) by debbugs.gnu.org; 19 Jan 2025 22:02:32 +0000 Received: from localhost ([127.0.0.1]:47673 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tZdNI-0006g7-EC for submit@debbugs.gnu.org; Sun, 19 Jan 2025 17:02:32 -0500 Received: from confino.investici.org ([2a11:7980:1::2:0]:48453) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tZdNG-0006fw-6B for 75045@debbugs.gnu.org; Sun, 19 Jan 2025 17:02:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=autistici.org; s=stigmate; t=1737324147; bh=lJretlzHnnn3oFhhVEQ0mkDAAdVsKRAzyAhlGW+DKkE=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=lB9m+t2m2Vex5C72b0dMQmojxXJGpiomPvMXMXu+ALbtwWeTeSmlxE+F3VK3+2/yU e8bvurFk5q4oU/e9mwwVVhY6hYYDO95QY8DU7P8FkzOOKyxP3u7QQX+eKcazkrx+tl mt2kLF11ycRZJbFThyd3sz44nK6tIKk35/WvGosE= Received: from mx1.investici.org (unknown [127.0.0.1]) by confino.investici.org (Postfix) with ESMTP id 4YbnWl1zPbz10yH; Sun, 19 Jan 2025 22:02:27 +0000 (UTC) Received: from [93.190.126.19] (mx1.investici.org [93.190.126.19]) (Authenticated sender: goodoldpaul@autistici.org) by localhost (Postfix) with ESMTPSA id 4YbnWl0dyPz10nQ; Sun, 19 Jan 2025 22:02:27 +0000 (UTC) Content-Type: multipart/alternative; boundary="------------CVWh6DSSzKMK8tDvG6ncZvCa" Message-ID: Date: Sun, 19 Jan 2025 23:02:26 +0100 MIME-Version: 1.0 User-Agent: Icedove Daily References: <87ikr95ppo.fsf@gnu.org> Content-Language: en-US From: paul In-Reply-To: <87ikr95ppo.fsf@gnu.org> X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) This is a multi-part message in MIME format. --------------CVWh6DSSzKMK8tDvG6ncZvCa Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi Ludo', I should have addressed your comments, thank you for your help! I'm about to send a v2. On 12/24/24 11:40, Ludovic Courtès wrote: >> + (command >> + (list >> + (string-append #+bash-minimal "/bin/bash") >> + "-l" "-c" >> + (string-append "restic-guix backup " #$name)) > Why go through bash? Would it be possible to execute ‘restic-guix’ > directly? There may be some better way but we go through bash, instead of executing restic-guix directly, because the login shell gives us the correct user environment that some backends require, such as rclone. I hope this clarifies and I'm definitely open to change this in case we find a better solution. cheers, giacomo --------------CVWh6DSSzKMK8tDvG6ncZvCa Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

Hi Ludo',

I should have addressed your comments, thank you for your help! I'm about to send a v2.

On 12/24/24 11:40, Ludovic Courtès wrote:
+                          (command
+                           (list
+                            (string-append #+bash-minimal "/bin/bash")
+                            "-l" "-c"
+                            (string-append "restic-guix backup " #$name))

Why go through bash?  Would it be possible to execute ‘restic-guix’
directly?
There may be some better way but we go through bash, instead of executing restic-guix directly, because the login shell gives us the correct user environment that some backends require, such as rclone.

I hope this clarifies and I'm definitely open to change this in case we find a better solution.

cheers,

giacomo
--------------CVWh6DSSzKMK8tDvG6ncZvCa-- From unknown Tue Jun 17 01:48:49 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#75045] [PATCH v2] services: restic-backup: Implement as a Shepherd timer. References: In-Reply-To: Resent-From: Giacomo Leidi Original-Sender: "Debbugs-submit" Resent-CC: ludo@gnu.org, maxim.cournoyer@gmail.com, guix-patches@gnu.org Resent-Date: Sun, 19 Jan 2025 22:05:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75045 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 75045@debbugs.gnu.org Cc: Giacomo Leidi , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Maxim Cournoyer X-Debbugs-Original-Xcc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Maxim Cournoyer Received: via spool by 75045-submit@debbugs.gnu.org id=B75045.173732428625895 (code B ref 75045); Sun, 19 Jan 2025 22:05:02 +0000 Received: (at 75045) by debbugs.gnu.org; 19 Jan 2025 22:04:46 +0000 Received: from localhost ([127.0.0.1]:47677 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tZdPR-0006jb-8O for submit@debbugs.gnu.org; Sun, 19 Jan 2025 17:04:45 -0500 Received: from confino.investici.org ([93.190.126.19]:47119) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tZdPN-0006jQ-MF for 75045@debbugs.gnu.org; Sun, 19 Jan 2025 17:04:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=autistici.org; s=stigmate; t=1737324280; bh=Z/4vWsWRY35iW7yItPwCX4x+WW7Mg4MHy4+T5WA0Rrs=; h=From:To:Cc:Subject:Date:From; b=VQAQNoM9lnLPsGPIA9/DjgHzma9dlxb3yxkQ9499GKJdXvRoz9HoOIZSPuYryQulg RSPCwkul/2kEbDISE7gsLWk4JpFTzshf0Jtmj6qiyIZiytyvdCfJXMPJtAiTjBCnUo W6kIz7FQJ35f7Av7vo1Ok3k2k3EkLXYIMv7agejc= Received: from mx1.investici.org (unknown [127.0.0.1]) by confino.investici.org (Postfix) with ESMTP id 4YbnZJ5S0sz10yH; Sun, 19 Jan 2025 22:04:40 +0000 (UTC) Received: from [93.190.126.19] (mx1.investici.org [93.190.126.19]) (Authenticated sender: goodoldpaul@autistici.org) by localhost (Postfix) with ESMTPSA id 4YbnZJ4MJ3z10xb; Sun, 19 Jan 2025 22:04:40 +0000 (UTC) From: Giacomo Leidi Date: Sun, 19 Jan 2025 23:04:00 +0100 Message-ID: <9b62d9a0cd9f00b074a5871729286925788a0eab.1737324240.git.goodoldpaul@autistici.org> X-Mailer: git-send-email 2.47.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) This patch implements restic backup with Shepherd services. It is supposed not to break any existing setup. * gnu/services/backup.scm (restic-backup-job): Add Shepherd configuration options; (restic-backup-job->mcron-job): Replace with...; (restic-job-log-file): New procedure; (restic-backup-job->shepherd-service): New procedure; (restic-backup-activation): New procedure; (restic-backup-service-type): Replace mcron with Shepherd extension and add activation extension hook. * doc/guix.texi: Document it. Change-Id: I66de3b6a1cb6177f9e4ee0c2acf3013ecbcdd338 --- doc/guix.texi | 39 +++++++++---- gnu/services/backup.scm | 122 +++++++++++++++++++++++++++++++++------- 2 files changed, 131 insertions(+), 30 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 0015d739bb6..33552065fe3 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -111,7 +111,7 @@ Copyright @copyright{} 2022 John Kehayias@* Copyright @copyright{} 2022⁠–⁠2023 Bruno Victal@* Copyright @copyright{} 2022 Ivan Vilata-i-Balaguer@* -Copyright @copyright{} 2023-2024 Giacomo Leidi@* +Copyright @copyright{} 2023-2025 Giacomo Leidi@* Copyright @copyright{} 2022 Antero Mejr@* Copyright @copyright{} 2023 Karl Hallsby@* Copyright @copyright{} 2023 Nathaniel Nicandro@* @@ -42709,19 +42709,16 @@ Miscellaneous Services "/etc/guix/signing-key.sec")))))))))) @end lisp -Each @code{restic-backup-job} translates to an mcron job which sets the +Each @code{restic-backup-job} translates to a Shepherd timer which sets the @env{RESTIC_PASSWORD} environment variable by reading the first line of @code{password-file} and runs @command{restic backup}, creating backups using rclone of all the files listed in the @code{files} field. -The @code{restic-backup-service-type} installs as well @code{restic-guix} -to the system profile, a @code{restic} utility wrapper that allows for easier -interaction with the Guix configured backup jobs. For example the following -could be used to instantaneusly trigger a backup for the above shown -configuration, without waiting for the scheduled job: +The @code{restic-backup-service-type} provides the ability to instantaneously +trigger a backup with the @code{trigger} Shepherd action: @example -restic-guix backup remote-ftp +sudo herd trigger remote-ftp @end example @c %start of fragment @@ -42752,6 +42749,23 @@ Miscellaneous Services @item @code{user} (default: @code{"root"}) (type: string) The user used for running the current job. +@item @code{group} (default: @code{"root"}) (type: string) +The group used for running the current job. + +@item @code{log-file} (type: maybe-string) +The file system path to the log file for this job. By default the file will +have be @file{/var/log/restic-backup/JOB-NAME.log}, where @code{JOB-NAME} is the +name defined in the @code{name} field. + +@item @code{max-duration} (type: maybe-number) +The maximum duration in seconds that a job may last. Past +@code{max-duration} seconds, the job is forcefully terminated. + +@item @code{wait-for-termination?} (default: @code{#f}) (type: boolean) +Wait until the job has finished before considering executing it again; +otherwise, perform it strictly on every occurrence of event, at the risk of +having multiple instances running concurrently. + @item @code{repository} (type: string) The restic repository target of this job. @@ -42764,9 +42778,12 @@ Miscellaneous Services for the current job. @item @code{schedule} (type: gexp-or-string) -A string or a gexp that will be passed as time specification in the -mcron job specification (@pxref{Syntax, mcron job specifications,, -mcron,GNU@tie{}mcron}). +A string or a gexp representing the frequency of the backup. Gexp must +evaluate to @code{calendar-event} records or to strings. Strings must contain +Vixie cron date lines. + +@item @code{requirement} (default: @code{'()}) (type: list-of-symbols) +The list of Shepherd services that this backup job depends upon. @item @code{files} (default: @code{'()}) (type: list-of-lowerables) The list of files or directories to be backed up. It must be a list of diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm index 555e9fc9590..3dda6ca370c 100644 --- a/gnu/services/backup.scm +++ b/gnu/services/backup.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2024 Giacomo Leidi +;;; Copyright © 2024, 2025 Giacomo Leidi ;;; ;;; This file is part of GNU Guix. ;;; @@ -18,9 +18,10 @@ (define-module (gnu services backup) #:use-module (gnu packages backup) + #:use-module (gnu packages bash) #:use-module (gnu services) #:use-module (gnu services configuration) - #:use-module (gnu services mcron) + #:use-module (gnu services shepherd) #:use-module (guix build-system copy) #:use-module (guix gexp) #:use-module ((guix licenses) @@ -33,11 +34,16 @@ (define-module (gnu services backup) restic-backup-job-fields restic-backup-job-restic restic-backup-job-user + restic-backup-job-group + restic-backup-job-log-file + restic-backup-job-max-duration + restic-backup-job-wait-for-termination? restic-backup-job-name restic-backup-job-repository restic-backup-job-password-file restic-backup-job-schedule restic-backup-job-files + restic-backup-job-requirement restic-backup-job-verbose? restic-backup-job-extra-flags @@ -64,6 +70,12 @@ (define (lowerable? value) (define list-of-lowerables? (list-of lowerable?)) +(define list-of-symbols? + (list-of symbol?)) + +(define-maybe string) +(define-maybe number) + (define-configuration/no-serialization restic-backup-job (restic (package restic) @@ -71,6 +83,23 @@ (define-configuration/no-serialization restic-backup-job (user (string "root") "The user used for running the current job.") + (group + (string "root") + "The group used for running the current job.") + (log-file + (maybe-string) + "The file system path to the log file for this job. By default the file will +have be @file{/var/log/restic-backup/JOB-NAME.log}, where @code{JOB-NAME} is the +name defined in the @code{name} field.") + (max-duration + (maybe-number) + "The maximum duration in seconds that a job may last. Past +@code{max-duration} seconds, the job is forcefully terminated.") + (wait-for-termination? + (boolean #f) + "Wait until the job has finished before considering executing it again; +otherwise, perform it strictly on every occurrence of event, at the risk of +having multiple instances running concurrently.") (name (string) "A string denoting a name for this job.") @@ -84,9 +113,12 @@ (define-configuration/no-serialization restic-backup-job current job.") (schedule (gexp-or-string) - "A string or a gexp that will be passed as time specification in the mcron -job specification (@pxref{Syntax, mcron job specifications,, mcron, -GNU@tie{}mcron}).") + "A string or a gexp representing the frequency of the backup. Gexp must +evaluate to @code{calendar-event} records or to strings. Strings must contain +Vixie cron date lines.") + (requirement + (list-of-symbols '()) + "The list of Shepherd services that this backup job depends upon.") (files (list-of-lowerables '()) "The list of files or directories to be backed up. It must be a list of @@ -175,16 +207,59 @@ (define (restic-guix jobs) (main (command-line))))) -(define (restic-backup-job->mcron-job config) - (let ((user - (restic-backup-job-user config)) - (schedule - (restic-backup-job-schedule config)) - (name - (restic-backup-job-name config))) - #~(job #$schedule - #$(string-append "restic-guix backup " name) - #:user #$user))) +(define (restic-job-log-file job) + (let ((name (restic-backup-job-name job)) + (log-file (restic-backup-job-log-file job))) + (if (maybe-value-set? log-file) + log-file + (string-append "/var/log/restic-backup/" name ".log")))) + +(define (restic-backup-job->shepherd-service config) + (let ((schedule (restic-backup-job-schedule config)) + (name (restic-backup-job-name config)) + (user (restic-backup-job-user config)) + (group (restic-backup-job-group config)) + (max-duration (restic-backup-job-max-duration config)) + (wait-for-termination? (restic-backup-job-wait-for-termination? config)) + (log-file (restic-job-log-file config)) + (requirement (restic-backup-job-requirement config))) + (shepherd-service (provision `(,(string->symbol name))) + (requirement + `(user-processes file-systems ,@requirement)) + (documentation + "Run @code{restic} backed backups on a regular basis.") + (modules '((shepherd service timer))) + (start + #~(make-timer-constructor + (if (string? #$schedule) + (cron-string->calendar-event #$schedule) + #$schedule) + (command + (list + ;; We go through bash, instead of executing + ;; restic-guix directly, because the login shell + ;; gives us the correct user environment that some + ;; backends require, such as rclone. + (string-append #+bash-minimal "/bin/bash") + "-l" "-c" + (string-append "restic-guix backup " #$name)) + #:user #$user + #:group #$group + #:environment-variables + (list + (string-append + "HOME=" (passwd:dir (getpwnam #$user))))) + #:log-file #$log-file + #:wait-for-termination? #$wait-for-termination? + #:max-duration #$(and (maybe-value-set? max-duration) + max-duration))) + (stop + #~(make-timer-destructor)) + (actions (list (shepherd-action + (name 'trigger) + (documentation "Manually trigger a backup, +without waiting for the scheduled time.") + (procedure #~trigger-timer))))))) (define (restic-guix-wrapper-package jobs) (package @@ -212,15 +287,24 @@ (define restic-backup-service-profile (restic-guix-wrapper-package jobs)) '()))) +(define (restic-backup-activation config) + #~(for-each + (lambda (log-file) + (mkdir-p (dirname log-file))) + (list #$@(map restic-job-log-file + (restic-backup-configuration-jobs config))))) + (define restic-backup-service-type (service-type (name 'restic-backup) (extensions (list + (service-extension activation-service-type + restic-backup-activation) (service-extension profile-service-type restic-backup-service-profile) - (service-extension mcron-service-type + (service-extension shepherd-root-service-type (lambda (config) - (map restic-backup-job->mcron-job + (map restic-backup-job->shepherd-service (restic-backup-configuration-jobs config)))))) (compose concatenate) @@ -232,5 +316,5 @@ (define restic-backup-service-type jobs))))) (default-value (restic-backup-configuration)) (description - "This service configures @code{mcron} jobs for running backups -with @code{restic}."))) + "This service configures @code{Shepherd} timers for running backups +with restic."))) base-commit: 5e834c220e81fddb77a26e23cf0cd5055b866844 -- 2.47.1 From unknown Tue Jun 17 01:48:49 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Giacomo Leidi Subject: bug#75045: closed (Re: [bug#75045] [PATCH v2] services: restic-backup: Implement as a Shepherd timer.) Message-ID: References: <875xm3ddeu.fsf@gnu.org> X-Gnu-PR-Message: they-closed 75045 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 75045@debbugs.gnu.org Date: Fri, 24 Jan 2025 23:02:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1737759722-1992-1" This is a multi-part message in MIME format... ------------=_1737759722-1992-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #75045: [PATCH] services: restic-backup: Implement as a Shepherd timer. which was filed against the guix-patches package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 75045@debbugs.gnu.org. --=20 75045: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D75045 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1737759722-1992-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 75045-done) by debbugs.gnu.org; 24 Jan 2025 23:01:57 +0000 Received: from localhost ([127.0.0.1]:47417 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbSgX-0000Vr-AM for submit@debbugs.gnu.org; Fri, 24 Jan 2025 18:01:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:37420) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tbSgV-0000VZ-6G for 75045-done@debbugs.gnu.org; Fri, 24 Jan 2025 18:01:55 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tbSgO-0006qo-KQ; Fri, 24 Jan 2025 18:01:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=i8IboycKb6LtY1vQFDNXj8n/E4FDco2Rg17/aneZSZ4=; b=gSw8wdaXl6WPYPN0lXUX OfJH3xp/Dj4nNCOQoRpdBfLX5Tbc4xsPwvMfxeR0ZSMy1YN9R4cU+X/okgpbnhiDFgOLxiFOU0A/A rvJsv0HX9Pk0J2xwZoLlT5YpuHL0njWMo9ljUPfGmTyAPmqM4ms77ub4ArO91xBb8XVhQbLvKXDF0 XK17ZKKvs2ZwoYUXC1xMs8nLshKESyMhDiIwAn/ZkWKFUeNEOu9iLUvc44RzQC2IlHe2CCkepn9L5 TOcUGlZJhpYvyCVV44xqYWXCJ69UB6dRl3hQfxHK+dLDc4TaHfphoXDSO6AXj6TsAnZwqPs75kuFZ AZSTJWVsGQKhjg==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Giacomo Leidi Subject: Re: [bug#75045] [PATCH v2] services: restic-backup: Implement as a Shepherd timer. In-Reply-To: <9b62d9a0cd9f00b074a5871729286925788a0eab.1737324240.git.goodoldpaul@autistici.org> (Giacomo Leidi's message of "Sun, 19 Jan 2025 23:04:00 +0100") References: <9b62d9a0cd9f00b074a5871729286925788a0eab.1737324240.git.goodoldpaul@autistici.org> Date: Sat, 25 Jan 2025 00:01:45 +0100 Message-ID: <875xm3ddeu.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 75045-done Cc: Maxim Cournoyer , 75045-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Giacomo Leidi skribis: > This patch implements restic backup with Shepherd services. It is > supposed not to break any existing setup. > > * gnu/services/backup.scm (restic-backup-job): Add Shepherd > configuration options; > (restic-backup-job->mcron-job): Replace with...; > (restic-job-log-file): New procedure; > (restic-backup-job->shepherd-service): New procedure; > (restic-backup-activation): New procedure; > (restic-backup-service-type): Replace mcron with Shepherd extension and a= dd > activation extension hook. > * doc/guix.texi: Document it. > > Change-Id: I66de3b6a1cb6177f9e4ee0c2acf3013ecbcdd338 Applied with the cosmetic change below, thanks! I just realized there=E2=80=99s no test for this service, so let=E2=80=99s = everything still works well. Would be nice to add a test. > + ;; We go through bash, instead of executing > + ;; restic-guix directly, because the login s= hell > + ;; gives us the correct user environment tha= t some > + ;; backends require, such as rclone. > + (string-append #+bash-minimal "/bin/bash") > + "-l" "-c" > + (string-append "restic-guix backup " #$name)) Thanks for adding this comment. Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/doc/guix.texi b/doc/guix.texi index d18209f288..9a53bdcd37 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -42755,7 +42755,7 @@ Miscellaneous Services @item @code{log-file} (type: maybe-string) The file system path to the log file for this job. By default the file will -have be @file{/var/log/restic-backup/JOB-NAME.log}, where @code{JOB-NAME} is the +have be @file{/var/log/restic-backup/@var{job-name}.log}, where @var{job-name} is the name defined in the @code{name} field. @item @code{max-duration} (type: maybe-number) diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm index 3dda6ca370..6cada58b10 100644 --- a/gnu/services/backup.scm +++ b/gnu/services/backup.scm @@ -89,7 +89,7 @@ (define-configuration/no-serialization restic-backup-job (log-file (maybe-string) "The file system path to the log file for this job. By default the file will -have be @file{/var/log/restic-backup/JOB-NAME.log}, where @code{JOB-NAME} is the +have be @file{/var/log/restic-backup/@var{job-name}.log}, where @var{job-name} is the name defined in the @code{name} field.") (max-duration (maybe-number) @@ -316,5 +316,5 @@ (define restic-backup-service-type jobs))))) (default-value (restic-backup-configuration)) (description - "This service configures @code{Shepherd} timers for running backups + "This service configures Shepherd timers for running backups with restic."))) --=-=-=-- ------------=_1737759722-1992-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 23 Dec 2024 10:46:46 +0000 Received: from localhost ([127.0.0.1]:53469 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tPfxV-0006EU-KK for submit@debbugs.gnu.org; Mon, 23 Dec 2024 05:46:46 -0500 Received: from lists.gnu.org ([209.51.188.17]:40722) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tPfxS-0006EK-V4 for submit@debbugs.gnu.org; Mon, 23 Dec 2024 05:46:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tPfxS-0006tn-PF for guix-patches@gnu.org; Mon, 23 Dec 2024 05:46:42 -0500 Received: from confino.investici.org ([93.190.126.19]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tPfxP-0003gD-NF for guix-patches@gnu.org; Mon, 23 Dec 2024 05:46:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=autistici.org; s=stigmate; t=1734950794; bh=rSJVA8TCLbx9MDrPZX5JM4ksnkPmSsxTJSgSPHPdU70=; h=From:To:Cc:Subject:Date:From; b=VxYTW+YUFQxlKnOcEpMYG/o8/WTqizDDm2HYIybcfuYoS214q+8ayxeN6LOoR7ETW UhocFXvXz/2JmunYjPC3Qx0SluDRkOAoFwBZDR9MJxrxyobDYlXiOUXXYCO33/ZOHf /qTBRTSL4f1Wlv9nYbeqseMFN55P5J0iE7MWU0J0= Received: from mx1.investici.org (unknown [127.0.0.1]) by confino.investici.org (Postfix) with ESMTP id 4YGvpL679Kz10wk; Mon, 23 Dec 2024 10:46:34 +0000 (UTC) Received: from [93.190.126.19] (mx1.investici.org [93.190.126.19]) (Authenticated sender: goodoldpaul@autistici.org) by localhost (Postfix) with ESMTPSA id 4YGvpL53vBz10wf; Mon, 23 Dec 2024 10:46:34 +0000 (UTC) From: Giacomo Leidi To: guix-patches@gnu.org Subject: [PATCH] services: restic-backup: Implement as a Shepherd timer. Date: Mon, 23 Dec 2024 11:46:05 +0100 Message-ID: X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 X-Debbugs-Cc: Ludovic Courtès , Maxim Cournoyer Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=93.190.126.19; envelope-from=goodoldpaul@autistici.org; helo=confino.investici.org X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit Cc: Giacomo Leidi X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) This patch implements restic backup with Shepherd services. It is supposed not to break any existing setup. * gnu/services/backup.scm (restic-backup-job): Add Shepherd configuration options; (restic-backup-job->mcron-job): Replace with...; (restic-job-log-file): New procedure; (restic-backup-job->shepherd-service): New procedure; (restic-backup-activation): New procedure; (restic-backup-service-type): Replace mcron with Shepherd extension and add activation extension hook. * doc/guix.texi: Document it. Change-Id: I66de3b6a1cb6177f9e4ee0c2acf3013ecbcdd338 --- doc/guix.texi | 36 +++++++++---- gnu/services/backup.scm | 114 ++++++++++++++++++++++++++++++++++------ 2 files changed, 123 insertions(+), 27 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 57030102ca..f77b765933 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -41987,19 +41987,16 @@ Miscellaneous Services "/etc/guix/signing-key.sec")))))))))) @end lisp -Each @code{restic-backup-job} translates to an mcron job which sets the +Each @code{restic-backup-job} translates to a Shepherd timer which sets the @env{RESTIC_PASSWORD} environment variable by reading the first line of @code{password-file} and runs @command{restic backup}, creating backups using rclone of all the files listed in the @code{files} field. -The @code{restic-backup-service-type} installs as well @code{restic-guix} -to the system profile, a @code{restic} utility wrapper that allows for easier -interaction with the Guix configured backup jobs. For example the following -could be used to instantaneusly trigger a backup for the above shown -configuration, without waiting for the scheduled job: +The @code{restic-backup-service-type} provides the ability to instantaneously +trigger a backup with the @code{trigger} Shepherd action: @example -restic-guix backup remote-ftp +sudo herd trigger remote-ftp-job @end example @c %start of fragment @@ -42030,6 +42027,22 @@ Miscellaneous Services @item @code{user} (default: @code{"root"}) (type: string) The user used for running the current job. +@item @code{group} (default: @code{"root"}) (type: string) +The group used for running the current job. + +@item @code{log-file} (type: maybe-string) +The file system path to the log file for this job. By default the file will +have the name of the job and be under @code{/var/log/restic-backup}. + +@item @code{max-duration} (type: maybe-number) +The maximum duration in seconds that a job may last. Past +@code{max-duration} seconds, the job will forcefully terminated. + +@item @code{wait-for-termination?} (default: @code{#f}) (type: boolean) +Wait until the job has finished before considering executing it again; +otherwise, perform it strictly on every occurrence of event, at the risk of +having multiple instances running concurrently. + @item @code{repository} (type: string) The restic repository target of this job. @@ -42042,9 +42055,12 @@ Miscellaneous Services for the current job. @item @code{schedule} (type: gexp-or-string) -A string or a gexp that will be passed as time specification in the -mcron job specification (@pxref{Syntax, mcron job specifications,, -mcron,GNU@tie{}mcron}). +A string or a gexp representing the frequency of the backup. Gexp must +evaluate to @code{calendar-event} records or to strings. Strings must contain +Vixie cron date lines. + +@item @code{requirement} (default: @code{'()}) (type: list-of-symbols) +The list of Shepherd services that this backup job depends upon. @item @code{files} (default: @code{'()}) (type: list-of-lowerables) The list of files or directories to be backed up. It must be a list of diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm index 555e9fc959..fc8934873b 100644 --- a/gnu/services/backup.scm +++ b/gnu/services/backup.scm @@ -18,9 +18,10 @@ (define-module (gnu services backup) #:use-module (gnu packages backup) + #:use-module (gnu packages bash) #:use-module (gnu services) #:use-module (gnu services configuration) - #:use-module (gnu services mcron) + #:use-module (gnu services shepherd) #:use-module (guix build-system copy) #:use-module (guix gexp) #:use-module ((guix licenses) @@ -33,11 +34,16 @@ (define-module (gnu services backup) restic-backup-job-fields restic-backup-job-restic restic-backup-job-user + restic-backup-job-group + restic-backup-job-log-file + restic-backup-job-max-duration + restic-backup-job-wait-for-termination? restic-backup-job-name restic-backup-job-repository restic-backup-job-password-file restic-backup-job-schedule restic-backup-job-files + restic-backup-job-requirement restic-backup-job-verbose? restic-backup-job-extra-flags @@ -64,6 +70,12 @@ (define (lowerable? value) (define list-of-lowerables? (list-of lowerable?)) +(define list-of-symbols? + (list-of symbol?)) + +(define-maybe string) +(define-maybe number) + (define-configuration/no-serialization restic-backup-job (restic (package restic) @@ -71,6 +83,22 @@ (define-configuration/no-serialization restic-backup-job (user (string "root") "The user used for running the current job.") + (group + (string "root") + "The group used for running the current job.") + (log-file + (maybe-string) + "The file system path to the log file for this job. By default the file will +have the name of the job and be under @code{/var/log/restic-backup}.") + (max-duration + (maybe-number) + "The maximum duration in seconds that a job may last. Past +@code{max-duration} seconds, the job will forcefully terminated.") + (wait-for-termination? + (boolean #f) + "Wait until the job has finished before considering executing it again; +otherwise, perform it strictly on every occurrence of event, at the risk of +having multiple instances running concurrently.") (name (string) "A string denoting a name for this job.") @@ -84,9 +112,12 @@ (define-configuration/no-serialization restic-backup-job current job.") (schedule (gexp-or-string) - "A string or a gexp that will be passed as time specification in the mcron -job specification (@pxref{Syntax, mcron job specifications,, mcron, -GNU@tie{}mcron}).") + "A string or a gexp representing the frequency of the backup. Gexp must +evaluate to @code{calendar-event} records or to strings. Strings must contain +Vixie cron date lines.") + (requirement + (list-of-symbols '()) + "The list of Shepherd services that this backup job depends upon.") (files (list-of-lowerables '()) "The list of files or directories to be backed up. It must be a list of @@ -175,16 +206,56 @@ (define (restic-guix jobs) (main (command-line))))) -(define (restic-backup-job->mcron-job config) - (let ((user - (restic-backup-job-user config)) - (schedule - (restic-backup-job-schedule config)) - (name - (restic-backup-job-name config))) - #~(job #$schedule - #$(string-append "restic-guix backup " name) - #:user #$user))) +(define (restic-job-log-file job) + (let ((name (restic-backup-job-name job)) + (log-file (restic-backup-job-log-file job))) + (if (maybe-value-set? log-file) + log-file + (string-append "/var/log/restic-backup/" name ".log")))) + +(define (restic-backup-job->shepherd-service config) + (let ((schedule (restic-backup-job-schedule config)) + (name (restic-backup-job-name config)) + (user (restic-backup-job-user config)) + (group (restic-backup-job-group config)) + (max-duration (restic-backup-job-max-duration config)) + (wait-for-termination? (restic-backup-job-wait-for-termination? config)) + (log-file (restic-job-log-file config)) + (requirement (restic-backup-job-requirement config))) + (shepherd-service (provision `(,(string->symbol + (string-append name "-job")))) + (requirement + `(user-processes file-systems ,@requirement)) + (documentation + "Run @code{restic} backed backups on a regular basis.") + (modules '((shepherd service timer))) + (start + #~(make-timer-constructor + (if (string? #$schedule) + (cron-string->calendar-event #$schedule) + #$schedule) + (command + (list + (string-append #+bash-minimal "/bin/bash") + "-l" "-c" + (string-append "restic-guix backup " #$name)) + #:user #$user + #:group #$group + #:environment-variables + (list + (string-append + "HOME=" (passwd:dir (getpwnam #$user))))) + #:log-file #$log-file + #:wait-for-termination? #$wait-for-termination? + #:max-duration #$(and (maybe-value-set? max-duration) + max-duration))) + (stop + #~(make-timer-destructor)) + (actions (list (shepherd-action + (name 'trigger) + (documentation "Manually trigger a backup, +without waiting for the scheduled time.") + (procedure #~trigger-timer))))))) (define (restic-guix-wrapper-package jobs) (package @@ -212,15 +283,24 @@ (define restic-backup-service-profile (restic-guix-wrapper-package jobs)) '()))) +(define (restic-backup-activation config) + #~(for-each + (lambda (log-file) + (mkdir-p (dirname log-file))) + (list #$@(map restic-job-log-file + (restic-backup-configuration-jobs config))))) + (define restic-backup-service-type (service-type (name 'restic-backup) (extensions (list + (service-extension activation-service-type + restic-backup-activation) (service-extension profile-service-type restic-backup-service-profile) - (service-extension mcron-service-type + (service-extension shepherd-root-service-type (lambda (config) - (map restic-backup-job->mcron-job + (map restic-backup-job->shepherd-service (restic-backup-configuration-jobs config)))))) (compose concatenate) @@ -232,5 +312,5 @@ (define restic-backup-service-type jobs))))) (default-value (restic-backup-configuration)) (description - "This service configures @code{mcron} jobs for running backups + "This service configures @code{Shepherd} timers for running backups with @code{restic}."))) base-commit: 2743faebb2893f65fb29a5cfd55c72a66a2b98a9 -- 2.46.0 ------------=_1737759722-1992-1-- From unknown Tue Jun 17 01:48:49 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#75045] [PATCH v2] services: restic-backup: Implement as a Shepherd timer. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 24 Jan 2025 23:05:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75045 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Giacomo Leidi Cc: Maxim Cournoyer , 75045@debbugs.gnu.org Received: via spool by 75045-submit@debbugs.gnu.org id=B75045.17377598562310 (code B ref 75045); Fri, 24 Jan 2025 23:05:02 +0000 Received: (at 75045) by debbugs.gnu.org; 24 Jan 2025 23:04:16 +0000 Received: from localhost ([127.0.0.1]:47429 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbSil-0000bA-Ig for submit@debbugs.gnu.org; Fri, 24 Jan 2025 18:04:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:37458) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tbSij-0000at-Li for 75045@debbugs.gnu.org; Fri, 24 Jan 2025 18:04:14 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tbSid-00070e-Iu; Fri, 24 Jan 2025 18:04:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=0Dxm046Bb/ytc/o/qK+LwyFZqQl2tLPJuI9bt/fcEZg=; b=Ec/8Ip8sEbxJ75V3ZpE8 ISP5LaXTsHV7yr8CWjQvvk56ch3CdrbJ0qtRu/PH2hgQbyLK7n4++JrtPzdXsYqWaqgVHuu8pbO0z f0dscA54JYQYkgNaQuY8+UOKsekNILxooCj6ZmeKXHv5HwpKqzr/kyVXrdm3rg9SRFq25c4N2Gul5 axpGUIqeL5g0dTKdq76NiW4hVTpoWhCAMtJp24o1DX1et9pbFdM6jLSTnSuA0vxvMXabke0juf/pD +YqdG1Q34745iqHLggcmCEmWDbQMAahSjAcedKQkY1T6z4Qw/jzi3xjYpGbKiFyVK2taRyP0EJySM ucInALmuWku2pw==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= In-Reply-To: <9b62d9a0cd9f00b074a5871729286925788a0eab.1737324240.git.goodoldpaul@autistici.org> (Giacomo Leidi's message of "Sun, 19 Jan 2025 23:04:00 +0100") References: <9b62d9a0cd9f00b074a5871729286925788a0eab.1737324240.git.goodoldpaul@autistici.org> Date: Sat, 25 Jan 2025 00:04:03 +0100 Message-ID: <871pwrddb0.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain And also: --=-=-= Content-Type: text/x-patch Content-Disposition: inline -(define-maybe string) -(define-maybe number) +(define-maybe/no-serialization string) +(define-maybe/no-serialization number) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable =E2=80=A6 to silence a warning about =E2=80=98serialize-string=E2=80=99 etc= . being unbound. --=-=-=--