GNU bug report logs - #54659
[PATCH 0/2] Periodically delete build logs

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 54659 in the body.
You can then email your comments to 54659 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 0/2] Periodically delete build logs
Date: Thu, 31 Mar 2022 23:20:38 +0200

Hi!

We’ve been accumulating cruft in /var/guix/log/drvs for too long,
time to do something about it!

Thoughts?

Ludo’.

Ludovic Courtès (2):
  services: Add 'log-cleanup-service-type'.
  services: Add 'log-cleanup' service to '%base-services' for build
    logs.

 doc/guix.texi          | 39 +++++++++++++++++++++++++++++++
 gnu/services/admin.scm | 53 +++++++++++++++++++++++++++++++++++++++++-
 gnu/services/base.scm  |  6 +++++
 3 files changed, 97 insertions(+), 1 deletion(-)


base-commit: 53b04339fe521f486d3017930a419d5ca8a6cffd
-- 
2.34.0

Message #8 received at 54659 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 54659 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 1/2] services: Add 'log-cleanup-service-type'.
Date: Thu, 31 Mar 2022 23:22:07 +0200

* gnu/services/admin.scm (<log-cleanup-configuration>): New record
type.
(log-cleanup-program, log-cleanup-mcron-jobs): New procedures.
(log-cleanup-service-type): New variable.
* doc/guix.texi (Log Rotation): Document it.
---
 doc/guix.texi          | 28 ++++++++++++++++++++++
 gnu/services/admin.scm | 53 +++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 80 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index e8ef4286be..ad2763ec8a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -17641,6 +17641,34 @@ The list of syslog-controlled files to be rotated.  By default it is:
 "/var/log/maillog")}.
 @end defvr
 
+Some log files just need to be deleted periodically once they are old,
+without any other criterion and without any archival step.  This is the
+case of build logs stored by @command{guix-daemon} under
+@file{/var/log/guix/drvs} (@pxref{Invoking guix-daemon}).  The
+@code{log-cleanup} service addresses this use case.
+
+@defvr {Scheme Variable} log-cleanup-service-type
+This is the type of the service to delete old logs.  Its value must be a
+@code{log-cleanup-configuration} record as described below.
+@end defvr
+
+@deftp {Data Type} log-cleanup-configuration
+Data type representing the log cleanup configuration
+
+@table @asis
+@item @code{directory}
+Name of the directory containing log files.
+
+@item @code{expiry} (default: @code{(* 6 30 24 3600)})
+Age in seconds after which a file is subject to deletion (six months by
+default).
+
+@item @code{schedule} (default: @code{"30 12 01,08,15,22 * *"})
+String or gexp denoting the corresponding mcron job schedule
+(@pxref{Scheduled Job Execution}).
+@end table
+@end deftp
+
 @node Networking Setup
 @subsection Networking Setup
 
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index 043517262f..3096acdf5a 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -1,6 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 Jan Nieuwenhuizen <janneke <at> gnu.org>
-;;; Copyright © 2016, 2017, 2018, 2019, 2020, 2021 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2016-2022 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2020 Brice Waegeneire <brice <at> waegenei.re>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -46,6 +46,13 @@ (define-module (gnu services admin)
             rottlog-service
             rottlog-service-type
 
+            log-cleanup-service-type
+            log-cleanup-configuration
+            log-cleanup-configuration?
+            log-cleanup-configuration-directory
+            log-cleanup-configuration-expiry
+            log-cleanup-configuration-schedule
+
             unattended-upgrade-service-type
             unattended-upgrade-configuration
             unattended-upgrade-configuration?
@@ -191,6 +198,50 @@ (define rottlog-service-type
                                  rotations)))))
    (default-value (rottlog-configuration))))
 
+
+;;;
+;;; Build log removal.
+;;;
+
+(define-record-type* <log-cleanup-configuration>
+  log-cleanup-configuration make-log-cleanup-configuration
+  log-cleanup-configuration?
+  (directory log-cleanup-configuration-directory) ;string
+  (expiry    log-cleanup-configuration-expiry     ;integer (seconds)
+             (default (* 6 30 24 3600)))
+  (schedule  log-cleanup-configuration-schedule   ;string or gexp
+             (default "30 12 01,08,15,22 * *")))
+
+(define (log-cleanup-program directory expiry)
+  (program-file "delete-old-logs"
+                (with-imported-modules '((guix build utils))
+                  #~(begin
+                      (use-modules (guix build utils))
+
+                      (let* ((now  (car (gettimeofday)))
+                             (logs (find-files #$directory
+					       (lambda (file stat)
+					         (> (- now (stat:mtime stat))
+						    #$expiry)))))
+                        (format #t "deleting ~a log files from '~a'...~%"
+                                (length logs) #$directory)
+                        (for-each delete-file logs))))))
+
+(define (log-cleanup-mcron-jobs configuration)
+  (match-record configuration <log-cleanup-configuration>
+    (directory expiry schedule)
+    (list #~(job #$schedule
+                 #$(log-cleanup-program directory expiry)))))
+
+(define log-cleanup-service-type
+  (service-type
+   (name 'log-cleanup)
+   (extensions
+    (list (service-extension mcron-service-type
+                             log-cleanup-mcron-jobs)))
+   (description
+    "Periodically delete old log files.")))
+
 
 ;;;
 ;;; Unattended upgrade.
-- 
2.34.0

Message #11 received at 54659 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 54659 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 2/2] services: Add 'log-cleanup' service to '%base-services'
 for build logs.
Date: Thu, 31 Mar 2022 23:22:08 +0200

* gnu/services/base.scm (%base-services): Add 'log-cleanup-service-type'
instance.
* doc/guix.texi (Log Rotation): Add example and mention '%base-services'.
---
 doc/guix.texi         | 13 ++++++++++++-
 gnu/services/base.scm |  6 ++++++
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index ad2763ec8a..eaaf829aa2 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -17645,7 +17645,18 @@ Some log files just need to be deleted periodically once they are old,
 without any other criterion and without any archival step.  This is the
 case of build logs stored by @command{guix-daemon} under
 @file{/var/log/guix/drvs} (@pxref{Invoking guix-daemon}).  The
-@code{log-cleanup} service addresses this use case.
+@code{log-cleanup} service addresses this use case.  For example,
+@code{%base-services} (@pxref{Base Services}) includes the following:
+
+@lisp
+;; Periodically delete build logs more than 4 months old.
+(service log-cleanup-service-type
+         (log-cleanup-configuration
+          (directory "/var/log/guix/drvs")
+          (expiry (* 4 30 24 3600))))
+@end lisp
+
+That ensures build logs do not accumulate endlessly.
 
 @defvr {Scheme Variable} log-cleanup-service-type
 This is the type of the service to delete old logs.  Its value must be a
diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index f278cb76de..ebaba524bc 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -2817,6 +2817,12 @@ (define %base-services
 
         (service rottlog-service-type)
 
+        ;; Periodically delete build logs more than 4 months old.
+        (service log-cleanup-service-type
+                 (log-cleanup-configuration
+                  (directory "/var/log/guix/drvs")
+                  (expiry (* 4 30 24 3600))))
+
         ;; The LVM2 rules are needed as soon as LVM2 or the device-mapper is
         ;; used, so enable them by default.  The FUSE and ALSA rules are
         ;; less critical, but handy.
-- 
2.34.0

Message #14 received at 54659 <at> debbugs.gnu.org (full text, mbox):

From: Liliana Marie Prikler <liliana.prikler <at> ist.tugraz.at>
To: Ludovic Courtès <ludo <at> gnu.org>, 54659 <at> debbugs.gnu.org
Subject: Re: [PATCH 2/2] services: Add 'log-cleanup' service to
 '%base-services' for build logs.
Date: Fri, 01 Apr 2022 08:05:30 +0200

Am Donnerstag, dem 31.03.2022 um 23:22 +0200 schrieb Ludovic Courtès:
> * gnu/services/base.scm (%base-services): Add 'log-cleanup-service-
> type'
> instance.
> * doc/guix.texi (Log Rotation): Add example and mention '%base-
> services'.
> ---
>  doc/guix.texi         | 13 ++++++++++++-
>  gnu/services/base.scm |  6 ++++++
>  2 files changed, 18 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/guix.texi b/doc/guix.texi
> index ad2763ec8a..eaaf829aa2 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -17645,7 +17645,18 @@ Some log files just need to be deleted
> periodically once they are old,
>  without any other criterion and without any archival step.  This is
> the
>  case of build logs stored by @command{guix-daemon} under
>  @file{/var/log/guix/drvs} (@pxref{Invoking guix-daemon}).  The
> -@code{log-cleanup} service addresses this use case.
> +@code{log-cleanup} service addresses this use case.  For example,
> +@code{%base-services} (@pxref{Base Services}) includes the
> following:
> +
> +@lisp
> +;; Periodically delete build logs more than 4 months old.
> +(service log-cleanup-service-type
> +         (log-cleanup-configuration
> +          (directory "/var/log/guix/drvs")
> +          (expiry (* 4 30 24 3600))))
> +@end lisp
> +
> +That ensures build logs do not accumulate endlessly.
>  
>  @defvr {Scheme Variable} log-cleanup-service-type
>  This is the type of the service to delete old logs.  Its value must
> be a
> diff --git a/gnu/services/base.scm b/gnu/services/base.scm
> index f278cb76de..ebaba524bc 100644
> --- a/gnu/services/base.scm
> +++ b/gnu/services/base.scm
> @@ -2817,6 +2817,12 @@ (define %base-services
>  
>          (service rottlog-service-type)
>  
> +        ;; Periodically delete build logs more than 4 months old.
> +        (service log-cleanup-service-type
> +                 (log-cleanup-configuration
> +                  (directory "/var/log/guix/drvs")
> +                  (expiry (* 4 30 24 3600))))
> +
This might be very nitpicky, but I think we should leave expiry
undefined so that the default of 6 months is applied.  This way, if
there's a future decision that all logs should probably kept e.g. for
only 3 months, there is just one place to edit rather than two.

Cheers

Message #17 received at 54659 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Liliana Marie Prikler <liliana.prikler <at> ist.tugraz.at>
Cc: 54659 <at> debbugs.gnu.org
Subject: Re: [PATCH 2/2] services: Add 'log-cleanup' service to
 '%base-services' for build logs.
Date: Fri, 01 Apr 2022 17:41:39 +0200

Hi!

Liliana Marie Prikler <liliana.prikler <at> ist.tugraz.at> skribis:

>> +        ;; Periodically delete build logs more than 4 months old.
>> +        (service log-cleanup-service-type
>> +                 (log-cleanup-configuration
>> +                  (directory "/var/log/guix/drvs")
>> +                  (expiry (* 4 30 24 3600))))
>> +
> This might be very nitpicky, but I think we should leave expiry
> undefined so that the default of 6 months is applied.  This way, if
> there's a future decision that all logs should probably kept e.g. for
> only 3 months, there is just one place to edit rather than two.

Yeah, dunno; I thought we’d rather be explicit, but maybe you’re right.

Ludo’.

Message #22 received at 54659-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Liliana Marie Prikler <liliana.prikler <at> ist.tugraz.at>
Cc: 54659-done <at> debbugs.gnu.org
Subject: Re: bug#54659: [PATCH 0/2] Periodically delete build logs
Date: Mon, 04 Apr 2022 23:17:04 +0200

Hi,

Liliana Marie Prikler <liliana.prikler <at> ist.tugraz.at> skribis:

> This might be very nitpicky, but I think we should leave expiry
> undefined so that the default of 6 months is applied.  This way, if
> there's a future decision that all logs should probably kept e.g. for
> only 3 months, there is just one place to edit rather than two.

In the end I followed your suggestion and pushed as
e692dc632cbb0e6d21ed6f09f4c7f52391802cfb.

Thanks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.