GNU bug report logs - #64471
[PATCH 0/2] File database update services

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 64471 in the body.
You can then email your comments to 64471 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 0/2] File database update services
Date: Wed,  5 Jul 2023 11:59:44 +0200

Hi!

These patches add two system services to periodically update file
databases: one for 'updatedb', and one for 'guix locate --update'.

I did not add them to '%base-services' mainly because: (1) 'updatedb'
builds a database of all the files available on the system, which
can be a problem on multi-user systems, and (2) the 'guix locate'
service is quite expensive ('time-machine' + 'store' method by
default) and eventually users may often download a pre-built database
from ci.guix or similar.

Thoughts?

Ludo'.

Ludovic Courtès (2):
  services: Add 'file-database' service.
  services: Add 'package-database' service.

 doc/guix.texi          | 111 ++++++++++++++++++++++++++++++++++++++-
 gnu/services/admin.scm | 116 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 226 insertions(+), 1 deletion(-)


base-commit: c4a836f65d178786a5dd1f7c2d9491bb2c7482b3
-- 
2.40.1

Message #8 received at 64471 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 64471 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 1/2] services: Add 'file-database' service.
Date: Wed,  5 Jul 2023 12:16:35 +0200

* gnu/services/admin.scm (%default-file-database-update-schedule)
(%default-file-database-excluded-directories): New variables.
(<file-database-configuration>): New record type.
(file-database-mcron-jobs): New procedure.
(file-database-service-type): New variable.
* doc/guix.texi (File Search Services): New node.
---
 doc/guix.texi          | 62 ++++++++++++++++++++++++++++++++++++++
 gnu/services/admin.scm | 67 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 129 insertions(+)

diff --git a/doc/guix.texi b/doc/guix.texi
index 853396f776..21ff15ccbc 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -388,6 +388,7 @@ Top
 * Printing Services::           Local and remote printer support.
 * Desktop Services::            D-Bus and desktop services.
 * Sound Services::              ALSA and Pulseaudio services.
+* File Search Services::        Tools to search for files.
 * Database Services::           SQL databases, key-value stores, etc.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
@@ -18414,6 +18415,7 @@ Services
 * Printing Services::           Local and remote printer support.
 * Desktop Services::            D-Bus and desktop services.
 * Sound Services::              ALSA and Pulseaudio services.
+* File Search Services::        Tools to search for files.
 * Database Services::           SQL databases, key-value stores, etc.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
@@ -24924,6 +24926,66 @@ Sound Services
 
 @end defvar
 
+@node File Search Services
+@subsection File Search Services
+
+@cindex file search
+@cindex searching for a file
+The services in this section populate @dfn{file databases} that let you
+search for files on your machine.  These services are provided by the
+@code{(gnu services admin)} module.
+
+The first one, @code{file-database-service-type}, periodically runs the
+venerable @command{updatedb} command (@pxref{Invoking updatedb,,, find,
+GNU Findutils}).  That command populates a database of file names that
+you can then search with the @command{locate} command (@pxref{Invoing
+locate,,, find, GNU Findutils}), as in this example:
+
+@example
+locate important-notes.txt
+@end example
+
+You can enable this service with its default settings by adding this
+snippet to your operating system services:
+
+@lisp
+(service file-database-service-type)
+@end lisp
+
+This updates the database once a week, excluding files from
+@file{/gnu/store}---these are more usefully handled by @command{guix
+locate} (@pxref{Invoking guix locate}).  You can of course provide a
+custom configuration, as described below.
+
+@defvar file-database-service-type
+This is the type of the file database service, which runs
+@command{updatedb} periodically.  Its associated value must be a
+@code{file-database-configuration} record, as described below.
+@end defvar
+
+@deftp {Data Type} file-database-configuration
+Record type for the @code{file-database-service-type} configuration,
+with the following fields:
+
+@table @asis
+@item @code{package} (default: @code{findutils})
+The GNU <at> tie{}Findutils package from which the @command{updatedb} command
+is taken.
+
+@item @code{schedule} (default: @code{%default-file-database-update-schedule})
+String or G-exp denoting an mcron schedule for the periodic
+@command{updatedb} job (@pxref{Guile Syntax,,, mcron, GNU <at> tie{}mcron}).
+
+@item @code{excluded-directories} (default @code{%default-file-database-excluded-directories})
+List of directories to ignore when building the file database.  By
+default, this includes @file{/tmp} and @file{/gnu/store}, which should
+instead be indexed by @command{guix locate} (@pxref{Invoking guix
+locate}).  This list is passed to the @option{--prunepaths} option of
+@command{updatedb} (@pxref{Invoking updatedb,,, find,
+GNU <at> tie{}Findutils}).
+@end table
+@end deftp
+
 @node Database Services
 @subsection Database Services
 
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index 1c10cfb1f6..57fdfc35c0 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -21,11 +21,14 @@
 
 (define-module (gnu services admin)
   #:use-module (gnu packages admin)
+  #:use-module ((gnu packages base)
+                #:select (canonical-package findutils))
   #:use-module (gnu packages certs)
   #:use-module (gnu packages package-management)
   #:use-module (gnu services)
   #:use-module (gnu services mcron)
   #:use-module (gnu services shepherd)
+  #:use-module ((guix store) #:select (%store-prefix))
   #:use-module (guix gexp)
   #:use-module (guix modules)
   #:use-module (guix packages)
@@ -55,6 +58,15 @@ (define-module (gnu services admin)
             log-cleanup-configuration-expiry
             log-cleanup-configuration-schedule
 
+            file-database-service-type
+            file-database-configuration
+            file-database-configuration?
+            file-database-configuration-package
+            file-database-configuration-schedule
+            file-database-configuration-excluded-directories
+            %default-file-database-update-schedule
+            %default-file-database-excluded-directories
+
             unattended-upgrade-service-type
             unattended-upgrade-configuration
             unattended-upgrade-configuration?
@@ -255,6 +267,61 @@ (define log-cleanup-service-type
    (description
     "Periodically delete old log files.")))
 
+
+;;;
+;;; File databases.
+;;;
+
+(define %default-file-database-update-schedule
+  ;; Default mcron schedule for the periodic 'updatedb' job: once every
+  ;; Sunday.
+  "10 23 * * 0")
+
+(define %default-file-database-excluded-directories
+  ;; Directories excluded from the 'locate' database.
+  (list (%store-prefix)
+        "/tmp" "/var/tmp" "/var/cache" ".*/\\.cache"
+        "/run/udev"))
+
+(define-record-type* <file-database-configuration>
+  file-database-configuration make-file-database-configuration
+  file-database-configuration?
+  (package              file-database-configuration-package
+                        (default
+                          (let-system (system target)
+                            ;; Unless we're cross-compiling, avoid pulling a
+                            ;; second copy of findutils.
+                            (if target
+                                findutils
+                                (canonical-package findutils)))))
+  (schedule             file-database-configuration-schedule
+                        (default %default-file-database-update-schedule))
+  (excluded-directories file-database-configuration-excluded-directories
+                        (default %default-file-database-excluded-directories)))
+
+(define (file-database-mcron-jobs configuration)
+  (match-record configuration <file-database-configuration>
+    (package schedule excluded-directories)
+    (let ((updatedb (program-file
+                     "updatedb"
+                     #~(execl #$(file-append package "/bin/updatedb")
+                              "updatedb"
+                              #$(string-append "--prunepaths="
+                                               (string-join
+                                                excluded-directories))))))
+      (list #~(job #$schedule #$updatedb)))))
+
+(define file-database-service-type
+  (service-type
+   (name 'file-database)
+   (extensions (list (service-extension mcron-service-type
+                                        file-database-mcron-jobs)))
+   (description
+    "Periodically update the file database used by the @command{locate} command,
+which lets you search for files by name.  The database is created by running
+the @command{updatedb} command.")
+   (default-value (file-database-configuration))))
+
 
 ;;;
 ;;; Unattended upgrade.
-- 
2.40.1

Message #11 received at 64471 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 64471 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 2/2] services: Add 'package-database' service.
Date: Wed,  5 Jul 2023 12:16:36 +0200

* gnu/services/admin.scm (%default-package-database-update-schedule):
New variable.
(<package-database-configuration>): New record type.
(package-database-mcron-jobs): New procedure.
(package-database-service-type): New variable.
* doc/guix.texi (File Search Services): Document it.
---
 doc/guix.texi          | 49 +++++++++++++++++++++++++++++++++++++++++-
 gnu/services/admin.scm | 49 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 97 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 21ff15ccbc..05cfdb9bc7 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -4481,7 +4481,9 @@ Invoking guix locate
 exist or is too old, it falls back to the per-user database, by default
 under @file{~/.cache/guix/locate}.  On a multi-user system,
 administrators may want to periodically update the system-wide database
-so that all users can benefit from it.
+so that all users can benefit from it, for instance by setting up
+@code{package-database-service-type} (@pxref{File Search Services,
+@code{package-database-service-type}}).
 
 The general syntax is:
 
@@ -24986,6 +24988,51 @@ File Search Services
 @end table
 @end deftp
 
+The second service, @code{package-database-service-type}, builds the
+database used by @command{guix locate}, which lets you search for
+packages that contain a given file (@pxref{Invoking guix locate}).  The
+service periodically updates a system-wide database, which will be
+readily available to anyone running @command{guix locate} on the system.
+To use this service with its default settings, add this snippet to your
+service list:
+
+@lisp
+(service package-database-service-type)
+@end lisp
+
+This will run @command{guix locate --update} once a week.
+
+@defvar package-database-service-type
+This is the service type for periodic @command{guix locate} updates
+(@pxref{Invoking guix locate}).  Its value must be a
+@code{package-database-configuration} record, as shown below.
+@end defvar
+
+@deftp {Data Type} package-database-configuration
+Data type to configure periodic package database updates.  It has the
+following fields:
+
+@table @asis
+@item @code{package} (default: @code{guix})
+The Guix package to use.
+
+@item @code{schedule} (default: @code{%default-package-database-update-schedule})
+String or G-exp denoting an mcron schedule for the periodic
+@command{guix locate --update} job (@pxref{Guile Syntax,,, mcron,
+GNU <at> tie{}mcron}).
+
+@item @code{method} (default: @code{'store})
+Indexing method for @command{guix locate}.  The default value,
+@code{'store}, yields a more complete database but is relatively
+expensive in terms of CPU and input/output.
+
+@item @code{channels} (default: @code{#~%default-channels})
+G-exp denoting the channels to use when updating the database
+(@pxref{Channels}).
+@end table
+@end deftp
+
+
 @node Database Services
 @subsection Database Services
 
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index 57fdfc35c0..f16880029f 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -28,6 +28,8 @@ (define-module (gnu services admin)
   #:use-module (gnu services)
   #:use-module (gnu services mcron)
   #:use-module (gnu services shepherd)
+  #:use-module (gnu system accounts)
+  #:use-module ((gnu system shadow) #:select (account-service-type))
   #:use-module ((guix store) #:select (%store-prefix))
   #:use-module (guix gexp)
   #:use-module (guix modules)
@@ -67,6 +69,14 @@ (define-module (gnu services admin)
             %default-file-database-update-schedule
             %default-file-database-excluded-directories
 
+            package-database-service-type
+            package-database-configuration
+            package-database-configuration?
+            package-database-configuration-package
+            package-database-configuration-schedule
+            package-database-configuration-method
+            package-database-configuration-channels
+
             unattended-upgrade-service-type
             unattended-upgrade-configuration
             unattended-upgrade-configuration?
@@ -322,6 +332,45 @@ (define file-database-service-type
 the @command{updatedb} command.")
    (default-value (file-database-configuration))))
 
+(define %default-package-database-update-schedule
+  ;; Default mcron schedule for the periodic 'guix locate --update' job: once
+  ;; every Monday.
+  "10 23 * * 1")
+
+(define-record-type* <package-database-configuration>
+  package-database-configuration make-package-database-configuration
+  package-database-configuration?
+  (package   package-database-configuration-package
+             (default guix))
+  (schedule  package-database-configuration-schedule
+             (default %default-package-database-update-schedule))
+  (method    package-database-configuration-method
+             (default 'store))
+  (channels  package-database-configuration-channels
+             (default #~%default-channels)))
+
+(define (package-database-mcron-jobs configuration)
+  (match-record configuration <package-database-configuration>
+    (package schedule method channels)
+    (let ((channels (scheme-file "channels.scm" channels)))
+      (list #~(job #$schedule
+                   ;; XXX: The whole thing's running as "root" just because it
+                   ;; needs write access to /var/cache/guix/locate.
+                   (string-append #$(file-append package "/bin/guix")
+                                  " time-machine -C " #$channels
+                                  " -- locate --update --method="
+                                  #$(symbol->string method)))))))
+
+(define package-database-service-type
+  (service-type
+   (name 'package-database)
+   (extensions (list (service-extension mcron-service-type
+                                        package-database-mcron-jobs)))
+   (description
+    "Periodically update the package database used by the @code{guix locate} command,
+which lets you search for packages that provide a given file.")
+   (default-value (package-database-configuration))))
+
 
 ;;;
 ;;; Unattended upgrade.
-- 
2.40.1

Message #14 received at 64471 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Victal <mirai <at> makinata.eu>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 64471 <at> debbugs.gnu.org
Subject: Re: [bug#64471] [PATCH 1/2] services: Add 'file-database' service.
Date: Thu, 13 Jul 2023 19:00:46 +0100

Hi Ludo’,

On 2023-07-05 11:16, Ludovic Courtès wrote:
> +(define-record-type* <file-database-configuration>
> +  file-database-configuration make-file-database-configuration
> +  file-database-configuration?
> +  (package              file-database-configuration-package
> +                        (default
> +                          (let-system (system target)
> +                            ;; Unless we're cross-compiling, avoid pulling a
> +                            ;; second copy of findutils.
> +                            (if target
> +                                findutils
> +                                (canonical-package findutils)))))
> +  (schedule             file-database-configuration-schedule
> +                        (default %default-file-database-update-schedule))
> +  (excluded-directories file-database-configuration-excluded-directories
> +                        (default %default-file-database-excluded-directories)))

How about using define-configuration instead to have the documentation neatly
in sync with the fields?

> +
> +(define (file-database-mcron-jobs configuration)
> +  (match-record configuration <file-database-configuration>
> +    (package schedule excluded-directories)
> +    (let ((updatedb (program-file
> +                     "updatedb"
> +                     #~(execl #$(file-append package "/bin/updatedb")
> +                              "updatedb"
> +                              #$(string-append "--prunepaths="
> +                                               (string-join
> +                                                excluded-directories))))))
> +      (list #~(job #$schedule #$updatedb)))))

I'm afraid #$schedule might be insufficient if this is a _mcron_ time-spec.
There's an elaborate dance done by fstrim-service-type to handle the more exotic
mcron time expressions, perhaps you can reuse it here?


-- 
Furthermore, I consider that nonfree software must be eradicated.

Cheers,
Bruno.

Message #17 received at 64471 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Bruno Victal <mirai <at> makinata.eu>
Cc: 64471 <at> debbugs.gnu.org
Subject: Re: bug#64471: [PATCH 0/2] File database update services
Date: Mon, 17 Jul 2023 22:22:14 +0200

Hello,

Bruno Victal <mirai <at> makinata.eu> skribis:

> On 2023-07-05 11:16, Ludovic Courtès wrote:
>> +(define-record-type* <file-database-configuration>
>> +  file-database-configuration make-file-database-configuration
>> +  file-database-configuration?
>> +  (package              file-database-configuration-package
>> +                        (default
>> +                          (let-system (system target)
>> +                            ;; Unless we're cross-compiling, avoid pulling a
>> +                            ;; second copy of findutils.
>> +                            (if target
>> +                                findutils
>> +                                (canonical-package findutils)))))
>> +  (schedule             file-database-configuration-schedule
>> +                        (default %default-file-database-update-schedule))
>> +  (excluded-directories file-database-configuration-excluded-directories
>> +                        (default %default-file-database-excluded-directories)))
>
> How about using define-configuration instead to have the documentation neatly
> in sync with the fields?

Hmm, good point, I’ll take a look.

>> +(define (file-database-mcron-jobs configuration)
>> +  (match-record configuration <file-database-configuration>
>> +    (package schedule excluded-directories)
>> +    (let ((updatedb (program-file
>> +                     "updatedb"
>> +                     #~(execl #$(file-append package "/bin/updatedb")
>> +                              "updatedb"
>> +                              #$(string-append "--prunepaths="
>> +                                               (string-join
>> +                                                excluded-directories))))))
>> +      (list #~(job #$schedule #$updatedb)))))
>
> I'm afraid #$schedule might be insufficient if this is a _mcron_ time-spec.
> There's an elaborate dance done by fstrim-service-type to handle the more exotic
> mcron time expressions, perhaps you can reuse it here?

I’m not sure what fstrim-service-type is trying to achieve with this:

    #~(job
       ;; Note: The “if” below is to ensure that
       ;; lists are ungexp'd correctly since @var{schedule}
       ;; can be either a procedure, a string or a list.
       #$(if (list? schedule)
             #~'(#$@schedule)
             schedule)
       …")

If we simply have:

  #~(job #$schedule …)

then ‘schedule’ can be anything you might expect, like:

  • "0 * * * *"         ;string

  • #~(next-hour …)     ;gexp

  • #~(lambda (x) …)    ;another gexp

What’s the problem? :-)

Ludo’.

Message #20 received at 64471 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 64471 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Bruno Victal <mirai <at> makinata.eu>
Subject: [PATCH v2 1/2] services: Add 'file-database' service.
Date: Thu, 20 Jul 2023 23:22:27 +0200

* gnu/services/admin.scm (%default-file-database-update-schedule)
(%default-file-database-excluded-directories): New variables.
(<file-database-configuration>): New record type.
(file-database-mcron-jobs): New procedure.
(file-database-service-type): New variable.
* doc/guix.texi (File Search Services): New node.
---
 doc/guix.texi          | 62 +++++++++++++++++++++++++++++++
 gnu/services/admin.scm | 83 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 145 insertions(+)

Change since v1: use of 'define-configuration' instead of
'define-record-type*'.

However, I am not using the generated documentation due
to <https://issues.guix.gnu.org/64754>.  (Overall I'm not
fully satisfied with 'define-configuration'.)

Thoughts?

Ludo'.

diff --git a/doc/guix.texi b/doc/guix.texi
index 1d8ebcd72f..1d5f7c6b47 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -388,6 +388,7 @@ Top
 * Printing Services::           Local and remote printer support.
 * Desktop Services::            D-Bus and desktop services.
 * Sound Services::              ALSA and Pulseaudio services.
+* File Search Services::        Tools to search for files.
 * Database Services::           SQL databases, key-value stores, etc.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
@@ -18428,6 +18429,7 @@ Services
 * Printing Services::           Local and remote printer support.
 * Desktop Services::            D-Bus and desktop services.
 * Sound Services::              ALSA and Pulseaudio services.
+* File Search Services::        Tools to search for files.
 * Database Services::           SQL databases, key-value stores, etc.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
@@ -24938,6 +24940,66 @@ Sound Services
 
 @end defvar
 
+@node File Search Services
+@subsection File Search Services
+
+@cindex file search
+@cindex searching for a file
+The services in this section populate @dfn{file databases} that let you
+search for files on your machine.  These services are provided by the
+@code{(gnu services admin)} module.
+
+The first one, @code{file-database-service-type}, periodically runs the
+venerable @command{updatedb} command (@pxref{Invoking updatedb,,, find,
+GNU Findutils}).  That command populates a database of file names that
+you can then search with the @command{locate} command (@pxref{Invoing
+locate,,, find, GNU Findutils}), as in this example:
+
+@example
+locate important-notes.txt
+@end example
+
+You can enable this service with its default settings by adding this
+snippet to your operating system services:
+
+@lisp
+(service file-database-service-type)
+@end lisp
+
+This updates the database once a week, excluding files from
+@file{/gnu/store}---these are more usefully handled by @command{guix
+locate} (@pxref{Invoking guix locate}).  You can of course provide a
+custom configuration, as described below.
+
+@defvar file-database-service-type
+This is the type of the file database service, which runs
+@command{updatedb} periodically.  Its associated value must be a
+@code{file-database-configuration} record, as described below.
+@end defvar
+
+@deftp {Data Type} file-database-configuration
+Record type for the @code{file-database-service-type} configuration,
+with the following fields:
+
+@table @asis
+@item @code{package} (default: @code{findutils})
+The GNU <at> tie{}Findutils package from which the @command{updatedb} command
+is taken.
+
+@item @code{schedule} (default: @code{%default-file-database-update-schedule})
+String or G-exp denoting an mcron schedule for the periodic
+@command{updatedb} job (@pxref{Guile Syntax,,, mcron, GNU <at> tie{}mcron}).
+
+@item @code{excluded-directories} (default @code{%default-file-database-excluded-directories})
+List of directories to ignore when building the file database.  By
+default, this includes @file{/tmp} and @file{/gnu/store}, which should
+instead be indexed by @command{guix locate} (@pxref{Invoking guix
+locate}).  This list is passed to the @option{--prunepaths} option of
+@command{updatedb} (@pxref{Invoking updatedb,,, find,
+GNU <at> tie{}Findutils}).
+@end table
+@end deftp
+
 @node Database Services
 @subsection Database Services
 
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index 1c10cfb1f6..636367a6f3 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -21,16 +21,21 @@
 
 (define-module (gnu services admin)
   #:use-module (gnu packages admin)
+  #:use-module ((gnu packages base)
+                #:select (canonical-package findutils))
   #:use-module (gnu packages certs)
   #:use-module (gnu packages package-management)
   #:use-module (gnu services)
+  #:use-module (gnu services configuration)
   #:use-module (gnu services mcron)
   #:use-module (gnu services shepherd)
+  #:use-module ((guix store) #:select (%store-prefix))
   #:use-module (guix gexp)
   #:use-module (guix modules)
   #:use-module (guix packages)
   #:use-module (guix records)
   #:use-module (srfi srfi-1)
+  #:use-module (ice-9 match)
   #:use-module (ice-9 vlist)
   #:export (%default-rotations
             %rotated-files
@@ -55,6 +60,15 @@ (define-module (gnu services admin)
             log-cleanup-configuration-expiry
             log-cleanup-configuration-schedule
 
+            file-database-service-type
+            file-database-configuration
+            file-database-configuration?
+            file-database-configuration-package
+            file-database-configuration-schedule
+            file-database-configuration-excluded-directories
+            %default-file-database-update-schedule
+            %default-file-database-excluded-directories
+
             unattended-upgrade-service-type
             unattended-upgrade-configuration
             unattended-upgrade-configuration?
@@ -255,6 +269,75 @@ (define log-cleanup-service-type
    (description
     "Periodically delete old log files.")))
 
+
+;;;
+;;; File databases.
+;;;
+
+(define %default-file-database-update-schedule
+  ;; Default mcron schedule for the periodic 'updatedb' job: once every
+  ;; Sunday.
+  "10 23 * * 0")
+
+(define %default-file-database-excluded-directories
+  ;; Directories excluded from the 'locate' database.
+  (list (%store-prefix)
+        "/tmp" "/var/tmp" "/var/cache" ".*/\\.cache"
+        "/run/udev"))
+
+(define (string-or-gexp? obj)
+  (or (string? obj) (gexp? obj)))
+
+(define string-list?
+  (match-lambda
+    (((? string) ...) #t)
+    (_ #f)))
+
+(define-configuration/no-serialization file-database-configuration
+  (package
+    (file-like (let-system (system target)
+                 ;; Unless we're cross-compiling, avoid pulling a second copy
+                 ;; of findutils.
+                 (if target
+                     findutils
+                     (canonical-package findutils))))
+    "The GNU <at> tie{}Findutils package from which the @command{updatedb} command
+is taken.")
+  (schedule
+   (string-or-gexp %default-file-database-update-schedule)
+   "String or G-exp denoting an mcron schedule for the periodic
+@command{updatedb} job (@pxref{Guile Syntax,,, mcron, GNU <at> tie{}mcron}).")
+  (excluded-directories
+   (string-list %default-file-database-excluded-directories)
+   "List of directories to ignore when building the file database.  By
+default, this includes @file{/tmp} and @file{/gnu/store}, which should instead
+be indexed by @command{guix locate} (@pxref{Invoking guix locate}).  This list
+is passed to the @option{--prunepaths} option of
+@command{updatedb} (@pxref{Invoking updatedb,,, find, GNU <at> tie{}Findutils})."))
+
+(define (file-database-mcron-jobs configuration)
+  (match-record configuration <file-database-configuration>
+    (package schedule excluded-directories)
+    (let ((updatedb (program-file
+                     "updatedb"
+                     #~(execl #$(file-append package "/bin/updatedb")
+                              "updatedb"
+                              #$(string-append "--prunepaths="
+                                               (string-join
+                                                excluded-directories))))))
+      (list #~(job #$schedule #$updatedb)))))
+
+(define file-database-service-type
+  (service-type
+   (name 'file-database)
+   (extensions (list (service-extension mcron-service-type
+                                        file-database-mcron-jobs)))
+   (description
+    "Periodically update the file database used by the @command{locate} command,
+which lets you search for files by name.  The database is created by running
+the @command{updatedb} command.")
+   (default-value (file-database-configuration))))
+
 
 ;;;
 ;;; Unattended upgrade.

base-commit: 283969d0c527aa41e65bb4f5c2a7fa3baf86c49a
-- 
2.41.0

Message #23 received at 64471 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 64471 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Bruno Victal <mirai <at> makinata.eu>
Subject: [PATCH v2 2/2] services: Add 'package-database' service.
Date: Thu, 20 Jul 2023 23:22:28 +0200

* gnu/services/admin.scm (%default-package-database-update-schedule):
New variable.
(<package-database-configuration>): New record type.
(package-database-mcron-jobs): New procedure.
(package-database-service-type): New variable.
* doc/guix.texi (File Search Services): Document it.
---
 doc/guix.texi          | 49 +++++++++++++++++++++++++++++++++++++-
 gnu/services/admin.scm | 53 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 101 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 1d5f7c6b47..833865f009 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -4481,7 +4481,9 @@ Invoking guix locate
 exist or is too old, it falls back to the per-user database, by default
 under @file{~/.cache/guix/locate}.  On a multi-user system,
 administrators may want to periodically update the system-wide database
-so that all users can benefit from it.
+so that all users can benefit from it, for instance by setting up
+@code{package-database-service-type} (@pxref{File Search Services,
+@code{package-database-service-type}}).
 
 The general syntax is:
 
@@ -25000,6 +25002,51 @@ File Search Services
 @end table
 @end deftp
 
+The second service, @code{package-database-service-type}, builds the
+database used by @command{guix locate}, which lets you search for
+packages that contain a given file (@pxref{Invoking guix locate}).  The
+service periodically updates a system-wide database, which will be
+readily available to anyone running @command{guix locate} on the system.
+To use this service with its default settings, add this snippet to your
+service list:
+
+@lisp
+(service package-database-service-type)
+@end lisp
+
+This will run @command{guix locate --update} once a week.
+
+@defvar package-database-service-type
+This is the service type for periodic @command{guix locate} updates
+(@pxref{Invoking guix locate}).  Its value must be a
+@code{package-database-configuration} record, as shown below.
+@end defvar
+
+@deftp {Data Type} package-database-configuration
+Data type to configure periodic package database updates.  It has the
+following fields:
+
+@table @asis
+@item @code{package} (default: @code{guix})
+The Guix package to use.
+
+@item @code{schedule} (default: @code{%default-package-database-update-schedule})
+String or G-exp denoting an mcron schedule for the periodic
+@command{guix locate --update} job (@pxref{Guile Syntax,,, mcron,
+GNU <at> tie{}mcron}).
+
+@item @code{method} (default: @code{'store})
+Indexing method for @command{guix locate}.  The default value,
+@code{'store}, yields a more complete database but is relatively
+expensive in terms of CPU and input/output.
+
+@item @code{channels} (default: @code{#~%default-channels})
+G-exp denoting the channels to use when updating the database
+(@pxref{Channels}).
+@end table
+@end deftp
+
+
 @node Database Services
 @subsection Database Services
 
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index 636367a6f3..0c24cedd43 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -29,6 +29,8 @@ (define-module (gnu services admin)
   #:use-module (gnu services configuration)
   #:use-module (gnu services mcron)
   #:use-module (gnu services shepherd)
+  #:use-module (gnu system accounts)
+  #:use-module ((gnu system shadow) #:select (account-service-type))
   #:use-module ((guix store) #:select (%store-prefix))
   #:use-module (guix gexp)
   #:use-module (guix modules)
@@ -69,6 +71,14 @@ (define-module (gnu services admin)
             %default-file-database-update-schedule
             %default-file-database-excluded-directories
 
+            package-database-service-type
+            package-database-configuration
+            package-database-configuration?
+            package-database-configuration-package
+            package-database-configuration-schedule
+            package-database-configuration-method
+            package-database-configuration-channels
+
             unattended-upgrade-service-type
             unattended-upgrade-configuration
             unattended-upgrade-configuration?
@@ -338,6 +348,49 @@ (define file-database-service-type
 the @command{updatedb} command.")
    (default-value (file-database-configuration))))
 
+(define %default-package-database-update-schedule
+  ;; Default mcron schedule for the periodic 'guix locate --update' job: once
+  ;; every Monday.
+  "10 23 * * 1")
+
+(define-configuration/no-serialization package-database-configuration
+  (package (file-like guix)
+           "The Guix package to use.")
+  (schedule (string-or-gexp
+             %default-package-database-update-schedule)
+            "String or G-exp denoting an mcron schedule for the periodic
+@command{guix locate --update} job (@pxref{Guile Syntax,,, mcron,
+GNU <at> tie{}mcron}).")
+  (method    (symbol 'store)
+             "Indexing method for @command{guix locate}.  The default value,
+@code{'store}, yields a more complete database but is relatively expensive in
+terms of CPU and input/output.")
+  (channels (gexp #~%default-channels)
+            "G-exp denoting the channels to use when updating the database
+(@pxref{Channels})."))
+
+(define (package-database-mcron-jobs configuration)
+  (match-record configuration <package-database-configuration>
+    (package schedule method channels)
+    (let ((channels (scheme-file "channels.scm" channels)))
+      (list #~(job #$schedule
+                   ;; XXX: The whole thing's running as "root" just because it
+                   ;; needs write access to /var/cache/guix/locate.
+                   (string-append #$(file-append package "/bin/guix")
+                                  " time-machine -C " #$channels
+                                  " -- locate --update --method="
+                                  #$(symbol->string method)))))))
+
+(define package-database-service-type
+  (service-type
+   (name 'package-database)
+   (extensions (list (service-extension mcron-service-type
+                                        package-database-mcron-jobs)))
+   (description
+    "Periodically update the package database used by the @code{guix locate} command,
+which lets you search for packages that provide a given file.")
+   (default-value (package-database-configuration))))
+
 
 ;;;
 ;;; Unattended upgrade.
-- 
2.41.0

Message #26 received at 64471 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Victal <mirai <at> makinata.eu>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 64471 <at> debbugs.gnu.org
Subject: Re: bug#64471: [PATCH 0/2] File database update services
Date: Wed, 26 Jul 2023 14:22:27 +0100

Hi Ludo’,

On 2023-07-17 21:22, Ludovic Courtès wrote:
> Bruno Victal <mirai <at> makinata.eu> skribis:
> 
>>
>> I'm afraid #$schedule might be insufficient if this is a _mcron_ time-spec.
>> There's an elaborate dance done by fstrim-service-type to handle the more exotic
>> mcron time expressions, perhaps you can reuse it here?
> 
> I’m not sure what fstrim-service-type is trying to achieve with this:
> 
>     #~(job
>        ;; Note: The “if” below is to ensure that
>        ;; lists are ungexp'd correctly since @var{schedule}
>        ;; can be either a procedure, a string or a list.
>        #$(if (list? schedule)
>              #~'(#$@schedule)
>              schedule)
>        …")
> 
> If we simply have:
> 
>   #~(job #$schedule …)
> 
> then ‘schedule’ can be anything you might expect, like:
> 
>   • "0 * * * *"         ;string
> 
>   • #~(next-hour …)     ;gexp
> 
>   • #~(lambda (x) …)    ;another gexp
> 
> What’s the problem? :-)

There's a third choice for that argument: a staged expression/list, hence
the gexp/ungexp dance.

[1]: <https://www.gnu.org/software/mcron/manual/mcron.html#Guile-Syntax>
[2]: <https://www.gnu.org/software/mcron/manual/mcron.html#Extended-Guile-examples>


-- 
Furthermore, I consider that nonfree software must be eradicated.

Cheers,
Bruno.

Message #29 received at 64471 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Victal <mirai <at> makinata.eu>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 64471 <at> debbugs.gnu.org
Subject: Re: [PATCH v2 1/2] services: Add 'file-database' service.
Date: Wed, 26 Jul 2023 14:28:40 +0100

On 2023-07-20 22:22, Ludovic Courtès wrote:

> +(define string-list?
> +  (match-lambda
> +    (((? string) ...) #t)
> +    (_ #f)))

There's a procedure 'list-of' from (gnu services configuration) that you can use
to define these kinds of procedures. In fact, there's already a list-of-strings?
predicate as a starting point within said module that you can use instead.

-- 
Furthermore, I consider that nonfree software must be eradicated.

Cheers,
Bruno.

Message #34 received at 64471-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 64471-done <at> debbugs.gnu.org
Cc: Bruno Victal <mirai <at> makinata.eu>
Subject: Re: bug#64471: [PATCH 0/2] File database update services
Date: Mon, 07 Aug 2023 16:39:18 +0200

Ludovic Courtès <ludo <at> gnu.org> skribis:

>   services: Add 'file-database' service.
>   services: Add 'package-database' service.

Pushed as b3a2b3e7238161ebd86c7609f68e8f1e9c1dd6b7!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.