Subject: [bug#61740] [PATCH v5] services: Add rspamd-service-type (was [PATCH v4] services: Add rspamd-service-type.)

From: Saku Laesvuori <[email-masked]>
Date: Tue, 12 Dec 2023 08:58:41 +0200

> >>> +     (service-extension profile-service-type
> >>> +                        (compose list rspamd-configuration-package))
> >>
> >> What's the motivation for adding the rspamd package to the profile?
> > 
> > That was also there when I picked up this patch. I assume it is added to
> > the profile so that the `rspamadm` and `rspamc` programs are available
> > and compatible with the daemon. I don't have strong feelings about this
> > in either direction.
> 
> I think it's better to omit this, users who are interested in the tools
> can use 'guix shell rspamd'.

Yes, except if the rspamd package in the configuration is incompatible
with the one in the user's Guix. Anyway, this is now removed in v5.

> > +(define (list-of-symbols? x)
> > +  (and (list? x)
> > +       (every symbol? x)))
> 
> list-of-symbols? is already defined in (gnu services configuration),
> you can omit this.

Done after a rebase in v5. Apparently it also defines some other useful
functions that I didn't know about.

> > +        (shepherd-action
> > +         (name 'reopenlog)
> > +         (documentation "Reopen log files.")
> 
> Missed this in my previous reply, I'd prefer naming this action as
> 'reopen instead.

Done in v5.

> > +(define %rspamd-os
> > +  (simple-operating-system
> > +   (service dhcp-client-service-type)
> 
> […]
> 
> > +   (service rspamd-service-type
> > +            (rspamd-configuration
> > +              (shepherd-requirements '(networking))
> > +              (local.d-files `(("worker-controller.inc"
> > +                                ,(plain-file
> > +                                   "rspamd-public-web-controller.conf"
> > +                                   "bind_socket = \"0.0.0.0:11334\";"))))))))
> 
> I wonder if you could remove dhcp-client-service-type and use the
> loopback device for this test instead, by binding to '[::1]' or '127.0.0.1'.
> (You don't need to add %loopback-static-networking here since it is already
> included in %base-services.)

I tried that but it didn't work because the vm does not have an IP route
configured to the host...

> 
> > +(define (run-rspamd-test)
> > +  "Return a test of an OS running Rspamd service."
> > +
> > +  (define rspamd-ports
> > +    '((22668 . 11334)))    ;; web controller
> 
> […]
> 
> > +
> > +  (define vm
> > +    (virtual-machine
> > +     (operating-system (marionette-operating-system
> > +                        %rspamd-os
> > +                        #:imported-modules '((gnu services herd))))
> > +     (port-forwardings rspamd-ports)))
> 
> […]
> 
> > +          ;; Check that we can access the web ui
> > +
> > +          (test-equal "http-get"
> > +            200
> > +            (response-code (http-get "http://localhost:22668/"))) ; HEAD is unsupported
> 
> Actually I've realized that these port-forwards are unnecessary
> and it would be better to instead do:
> 
> --8<---------------cut here---------------start------------->8---
> ;; Note: remove (web client) and (web response) in the imports above
> ;; i.e. after the #~(begin (use-modules …
> 
> (test-equal "http-get"
>   200
>   (marionette-eval
>    '(begin
>       (use-modules (web client)
>                    (web response))
>       (response-code (http-head "http://localhost:11334/")))
>    marionette))
> --8<---------------cut here---------------end--------------->8---

... However with changes like these it does work in v5.
From 7aee03ff1a7ada82436ff424fe7fdbac117fbf29 Mon Sep 17 00:00:00 2001
Message-ID: <[email-masked]>
From: Thomas Ieong <[email-masked]>
Date: Thu, 23 Feb 2023 21:16:14 +0100
Subject: [PATCH v5] services: Add rspamd-service-type.

* gnu/services/mail.scm (rspamd-service-type): New variable.
* gnu/tests/mail.scm (%test-rspamd): New variable.
* doc/guix.texi: Document it.

Co-authored-by: Saku Laesvuori <[email-masked]>
Change-Id: I7196643f087ffe9fc91aab231b69d5ed8dc9d198
---
 doc/guix.texi         |  62 +++++++++++++
 gnu/services/mail.scm | 200 +++++++++++++++++++++++++++++++++++++++++-
 gnu/tests/mail.scm    |  67 +++++++++++++-
 3 files changed, 327 insertions(+), 2 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 3ad8508a32..9dabd51f87 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -120,6 +120,8 @@
 Copyright @copyright{} 2023 Brian Cully@*
 Copyright @copyright{} 2023 Felix Lechner@*
 Copyright @copyright{} 2023 Foundation Devices, Inc.@*
+Copyright @copyright{} 2023 Thomas Ieong@*
+Copyright @copyright{} 2023 Saku Laesvuori@*
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -27617,6 +27619,66 @@ Mail Services
 @end table
 @end deftp
 
+@subsubheading Rspamd Service
+@cindex email
+@cindex spam
+
+@defvar rspamd-service-type
+This is the type of the @uref{https://rspamd.com/, Rspamd} filtering
+system whose value should be a @code{rspamd-configuration}.
+@end defvar
+
+@c %start of fragment
+
+@deftp {Data Type} rspamd-configuration
+Available @code{rspamd-configuration} fields are:
+
+@table @asis
+@item @code{package} (default: @code{rspamd}) (type: file-like)
+The package that provides rspamd.
+
+@item @code{config-file} (default: @code{%default-rspamd-config-file}) (type: file-like)
+File-like object of the configuration file to use.  By default all
+workers are enabled except fuzzy and they are binded to their usual
+ports, e.g localhost:11334, localhost:11333 and so on
+
+@item @code{local.d-files} (default: @code{()}) (type: directory-tree)
+Configuration files in local.d, provided as a list of two element lists
+where the first element is the filename and the second one is a
+file-like object.  Settings in these files will be merged with the
+defaults.
+
+@item @code{override.d-files} (default: @code{()}) (type: directory-tree)
+Configuration files in override.d, provided as a list of two element
+lists where the first element is the filename and the second one is a
+file-like object.  Settings in these files will override the defaults.
+
+@item @code{user} (default: @code{%default-rspamd-account}) (type: user-account)
+The user to run rspamd as.
+
+@item @code{group} (default: @code{%default-rspamd-group}) (type: user-group)
+The group to run rspamd as.
+
+@item @code{debug?} (default: @code{#f}) (type: boolean)
+Force debug output.
+
+@item @code{insecure?} (default: @code{#f}) (type: boolean)
+Ignore running workers as privileged users.
+
+@item @code{skip-template?} (default: @code{#f}) (type: boolean)
+Do not apply Jinja templates.
+
+@item @code{shepherd-requirements} (default: @code{(loopback)}) (type: list-of-symbols)
+This is a list of symbols naming Shepherd services that this service
+will depend on.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
 @node Messaging Services
 @subsection Messaging Services
 
diff --git a/gnu/services/mail.scm b/gnu/services/mail.scm
index 12dcc8e71d..afe1bb6016 100644
--- a/gnu/services/mail.scm
+++ b/gnu/services/mail.scm
@@ -5,6 +5,8 @@
 ;;; Copyright © 2017, 2020 Tobias Geerinckx-Rice <[email-masked]>
 ;;; Copyright © 2019 Kristofer Buffington <[email-masked]>
 ;;; Copyright © 2020 Jonathan Brielmaier <[email-masked]>
+;;; Copyright © 2023 Thomas Ieong <[email-masked]>
+;;; Copyright © 2023 Saku Laesvuori <[email-masked]>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -80,7 +82,13 @@ (define-module (gnu services mail)
             radicale-configuration
             radicale-configuration?
             radicale-service-type
-            %default-radicale-config-file))
+            %default-radicale-config-file
+
+            rspamd-configuration
+            rspamd-service-type
+            %default-rspamd-account
+            %default-rspamd-config-file
+            %default-rspamd-group))
 
 ;;; Commentary:
 ;;;
@@ -1987,3 +1995,193 @@ (define radicale-service-type
           (service-extension account-service-type (const %radicale-accounts))
           (service-extension activation-service-type radicale-activation)))
    (default-value (radicale-configuration))))
+
+;;;
+;;; Rspamd.
+;;;
+
+(define (directory-tree? xs)
+  (match xs
+    ((((? string?) (? file-like?)) ...) #t)
+    (_ #f)))
+
+(define-configuration/no-serialization rspamd-configuration
+  (package
+   (file-like rspamd)
+   "The package that provides rspamd.")
+  (config-file
+   (file-like %default-rspamd-config-file)
+   "File-like object of the configuration file to use.  By default
+all workers are enabled except fuzzy and they are binded
+to their usual ports, e.g localhost:11334, localhost:11333 and so on")
+  (local.d-files
+   (directory-tree '())
+   "Configuration files in local.d, provided as a list of two element lists where
+the first element is the filename and the second one is a file-like object.  Settings
+in these files will be merged with the defaults.")
+  (override.d-files
+   (directory-tree '())
+   "Configuration files in override.d, provided as a list of two element lists where
+the first element is the filename and the second one is a file-like object.  Settings
+in these files will override the defaults.")
+  (user
+   (user-account %default-rspamd-account)
+   "The user to run rspamd as.")
+  (group
+   (user-group %default-rspamd-group)
+   "The group to run rspamd as.")
+  (debug?
+   (boolean #f)
+   "Force debug output.")
+  (insecure?
+   (boolean #f)
+   "Ignore running workers as privileged users.")
+  (skip-template?
+   (boolean #f)
+   "Do not apply Jinja templates.")
+  (shepherd-requirements
+   (list-of-symbols '(loopback))
+   "This is a list of symbols naming Shepherd services that this service
+will depend on."))
+
+(define %default-rspamd-account
+  (user-account
+      (name "rspamd")
+      (group "rspamd")
+      (system? #t)
+      (comment "Rspamd daemon")
+      (home-directory "/var/empty")
+      (shell (file-append shadow "/sbin/nologin"))))
+
+(define %default-rspamd-group
+  (user-group
+    (name "rspamd")
+    (system? #t)))
+
+(define %default-rspamd-config-file
+  (plain-file "rspamd.conf" "
+.include \"$CONFDIR/common.conf\"
+
+options {
+    pidfile = \"$RUNDIR/rspamd.pid\";
+    .include \"$CONFDIR/options.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/options.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/options.inc\"
+}
+
+logging {
+    type = \"file\";
+    filename = \"$LOGDIR/rspamd.log\";
+    .include \"$CONFDIR/logging.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/logging.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/logging.inc\"
+}
+
+worker \"normal\" {
+    bind_socket = \"localhost:11333\";
+    .include \"$CONFDIR/worker-normal.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-normal.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-normal.inc\"
+}
+
+worker \"controller\" {
+    bind_socket = \"localhost:11334\";
+    .include \"$CONFDIR/worker-controller.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-controller.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-controller.inc\"
+}
+
+worker \"rspamd_proxy\" {
+    bind_socket = \"localhost:11332\";
+    .include \"$CONFDIR/worker-proxy.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-proxy.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-proxy.inc\"
+}
+
+# Local fuzzy storage is disabled by default
+
+worker \"fuzzy\" {
+    bind_socket = \"localhost:11335\";
+    count = -1; # Disable by default
+    .include \"$CONFDIR/worker-fuzzy.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-fuzzy.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-fuzzy.inc\"
+}
+"))
+
+(define (rspamd-accounts config)
+  (match-record config <rspamd-configuration>
+    (user group)
+    (list group user)))
+
+(define (rspamd-shepherd-service config)
+  (match-record config <rspamd-configuration>
+    (package config-file user group debug? insecure? skip-template?
+     local.d-files override.d-files shepherd-requirements)
+    (list
+     (shepherd-service
+      (provision '(rspamd))
+      (documentation "Run the rspamd daemon.")
+      (requirement shepherd-requirements)
+      (start (let ((rspamd (file-append package "/bin/rspamd"))
+                   (local-confdir
+                     (file-union
+                      "rspamd-local-confdir"
+                      `(("local.d" ,(file-union "local.d" local.d-files))
+                        ("override.d" ,(file-union "override.d" override.d-files))))))
+               (with-imported-modules (source-module-closure '((gnu build activation)))
+                 #~(begin
+                     (use-modules (gnu build activation)) ; for mkdir-p/perms
+                     (let ((user (getpwnam #$(user-account-name user))))
+                       (mkdir-p/perms "/var/run/rspamd" user #o755)
+                       (mkdir-p/perms "/var/log/rspamd" user #o755)
+                       (mkdir-p/perms "/var/lib/rspamd" user #o755))
+                     (make-forkexec-constructor
+                      (list #$rspamd "--config" #$config-file
+                            "--var" (string-append "LOCAL_CONFDIR=" #$local-confdir)
+                            "--no-fork"
+                            #$@(if debug?
+                                 '("--debug")
+                                 '())
+                            #$@(if insecure?
+                                 '("--insecure")
+                                 '())
+                            #$@(if skip-template?
+                                 '("--skip-template")
+                                 '()))
+                      #:user #$(user-account-name user)
+                      #:group #$(user-group-name group))))))
+      (stop #~(make-kill-destructor))
+      (actions
+       (list
+        (shepherd-configuration-action config-file)
+        (shepherd-action
+         (name 'reload)
+         (documentation "Reload rspamd.")
+         (procedure
+          #~(lambda (pid)
+              (if pid
+                (begin
+                  (kill pid SIGHUP)
+                  (display "Service rspamd has been reloaded"))
+                (format #t "Service rspamd is not running.")))))
+        (shepherd-action
+         (name 'reopen)
+         (documentation "Reopen log files.")
+         (procedure
+          #~(lambda (pid)
+              (if pid
+                (begin
+                  (kill pid SIGUSR1)
+                  (display "Reopening the logs for rspamd"))
+                (format #t "Service rspamd is not running.")))))))))))
+
+(define rspamd-service-type
+  (service-type
+   (name 'rspamd)
+   (description "Run the rapid spam filtering system.")
+   (extensions
+    (list
+     (service-extension shepherd-root-service-type rspamd-shepherd-service)
+     (service-extension account-service-type rspamd-accounts)))
+   (default-value (rspamd-configuration))))
diff --git a/gnu/tests/mail.scm b/gnu/tests/mail.scm
index dcb8f08ea8..176e7c1d07 100644
--- a/gnu/tests/mail.scm
+++ b/gnu/tests/mail.scm
@@ -6,6 +6,7 @@
 ;;; Copyright © 2018 Clément Lassieur <[email-masked]>
 ;;; Copyright © 2019 Christopher Baines <[email-masked]>
 ;;; Copyright © 2019, 2020 Tobias Geerinckx-Rice <[email-masked]>
+;;; Copyright © 2023 Thomas Ieong <[email-masked]>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -40,7 +41,8 @@ (define-module (gnu tests mail)
   #:export (%test-opensmtpd
             %test-exim
             %test-dovecot
-            %test-getmail))
+            %test-getmail
+            %test-rspamd))
 
 (define %opensmtpd-os
   (simple-operating-system
@@ -579,3 +581,66 @@ (define %test-getmail
    (name "getmail")
    (description "Connect to a running Getmail server.")
    (value (run-getmail-test))))
+
+(define %rspamd-os
+  (simple-operating-system
+   (service rspamd-service-type)))
+
+(define (run-rspamd-test)
+  "Return a test of an OS running Rspamd service."
+
+  (define vm
+    (virtual-machine
+      (marionette-operating-system
+        %rspamd-os
+        #:imported-modules '((gnu services herd)))))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-64)
+                       (gnu build marionette))
+
+          (define marionette
+            (make-marionette '(#$vm)))
+
+          (test-runner-current (system-test-runner #$output))
+          (test-begin "rspamd")
+
+          (test-assert "service is running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (start-service 'rspamd))
+             marionette))
+
+          (test-assert "rspamd socket ready"
+            (wait-for-unix-socket
+             "/var/lib/rspamd/rspamd.sock"
+             marionette))
+
+          (test-assert "rspamd log file"
+            (wait-for-file "/var/log/rspamd/rspamd.log" marionette))
+
+          ;; Check that we can access the web ui
+
+          (test-equal "http-get"
+            200
+            (marionette-eval
+              '(begin
+                 (use-modules (web client)
+                              (web response))
+                 ;; HEAD returns 500 internal server error, so use GET even though
+                 ;; only the headers are relevant
+                 (response-code (http-get "http://localhost:11334")))
+              marionette))
+
+          (test-end))))
+
+  (gexp->derivation "rspamd-test" test))
+
+(define %test-rspamd
+  (system-test
+   (name "rspamd")
+   (description "Basic rspamd service test.")
+   (value (run-rspamd-test))))

base-commit: 06587003b896755f876ecd57b848e1d663fafb87
-- 
2.41.0