GNU bug report logs - #52703
[PATCH 0/2] Improving rsync-service-type

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 52703 in the body.
You can then email your comments to 52703 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 0/2] Improving rsync-service-type
Date: Tue, 21 Dec 2021 15:18:58 +0100

Hello Guix!

While fiddling around for the infrastructure hackathon, I found that
the rsync service was a bit too limited: it would only let you export
a single directory.

These patches extend it so that users can specify several “modules”,
each of which exports one directory and has its own properties (read-only,
chroot, timeout, etc.).

Backwards compatibility is maintained.

Thoughts?

Ludo’.

Ludovic Courtès (2):
  services: rsync: Allow configuring several rsync "modules".
  tests: rsync: Define several modules.

 doc/guix.texi          |  71 ++++++++++++-----
 gnu/services/rsync.scm | 170 +++++++++++++++++++++++++++++------------
 gnu/tests/rsync.scm    |  40 +++++++++-
 3 files changed, 213 insertions(+), 68 deletions(-)

-- 
2.33.0

Message #8 received at 52703 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 52703 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 2/2] tests: rsync: Define several modules.
Date: Tue, 21 Dec 2021 15:23:37 +0100

* gnu/tests/rsync.scm (run-rsync-test)["Test file not copied to
read-only share", "Test file correctly received from read-only share"]:
New tests.
(%rsync-os): Define two modules.
---
 gnu/tests/rsync.scm | 40 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 39 insertions(+), 1 deletion(-)

diff --git a/gnu/tests/rsync.scm b/gnu/tests/rsync.scm
index 91f2b41cec..079a898cdc 100644
--- a/gnu/tests/rsync.scm
+++ b/gnu/tests/rsync.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017 Christopher Baines <mail <at> cbaines.net>
 ;;; Copyright © 2018 Clément Lassieur <clement <at> lassieur.org>
+;;; Copyright © 2021 Ludovic Courtès <ludo <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -104,6 +105,35 @@ (define marionette
                     (read-line port))))
              marionette))
 
+          (test-equal "Test file not copied to read-only share"
+            10                                  ;see "EXIT VALUES" in rsync(1)
+            (marionette-eval
+             '(status:exit-val
+               (system* "rsync" "/tmp/input"
+                        (string-append "rsync://localhost:"
+                                       (number->string #$rsync-port)
+                                       "/read-only/input")))
+             marionette))
+
+          (test-equal "Test file correctly received from read-only share"
+            "\"Hi!\" from the read-only share."
+            (marionette-eval
+             '(begin
+                (use-modules (ice-9 rdelim))
+
+                (call-with-output-file "/srv/read-only/the-file"
+                  (lambda (port)
+                    (display "\"Hi!\" from the read-only share." port)))
+
+                (zero?
+                 (system* "rsync"
+                          (string-append "rsync://localhost:"
+                                         (number->string #$rsync-port)
+                                         "/read-only/the-file")
+                          "/tmp/output"))
+                (call-with-input-file "/tmp/output" read-line))
+             marionette))
+
           (test-end))))
 
   (gexp->derivation "rsync-test" test))
@@ -113,7 +143,15 @@ (define* %rsync-os
   (let ((base-os
          (simple-operating-system
           (service dhcp-client-service-type)
-          (service rsync-service-type))))
+          (service rsync-service-type
+                   (rsync-configuration
+                    (modules (list (rsync-module
+                                    (name "read-only")
+                                    (file-name "/srv/read-only"))
+                                   (rsync-module
+                                    (name "files")
+                                    (file-name "/srv/read-write")
+                                    (read-only? #f)))))))))
     (operating-system
       (inherit base-os)
       (packages (cons* rsync
-- 
2.33.0

Message #11 received at 52703 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 52703 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH 1/2] services: rsync: Allow configuring several rsync
 "modules".
Date: Tue, 21 Dec 2021 15:23:36 +0100

Until now the rsync service would export a single module, named
"files".  This allows users to specify as many modules as they want, in
line with rsyncd.conf(5).

* gnu/services/rsync.scm (warn-share-field-deprecation): New procedure.
(<rsync-configuration>)[modules]: New field.
[share-path, share-comment, read-only?, timeout]: Mark as deprecated.
(<rsync-module>): New record type.
(%default-modules): New variable.
(rsync-configuration-modules): New procedure.
(rsync-activation): Create the directory of each module.
(rsync-config-file): Generate configuration for each module.
(rsync-service-type)[description]: New field.
* doc/guix.texi (Networking Services): Adjust documentation.  Augment
example.
---
 doc/guix.texi          |  71 ++++++++++++-----
 gnu/services/rsync.scm | 170 +++++++++++++++++++++++++++++------------
 2 files changed, 174 insertions(+), 67 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 333cb4117a..34e75156eb 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -18059,7 +18059,17 @@ The value for this service type is a
 @command{rsync-configuration} record as in this example:
 
 @lisp
-(service rsync-service-type)
+;; Export two directories over rsync.  By default rsync listens on
+;; all the network interfaces.
+(service rsync-service-type
+         (rsync-configuration
+           (modules (list (rsync-module
+                            (name "music")
+                            (file-name "/srv/zik")
+                            (read-only? #f))
+                          (rsync-module
+                            (name "movies")
+                            (file-name "/home/charlie/movies"))))))
 @end lisp
 
 See below for details about @code{rsync-configuration}.
@@ -18090,34 +18100,55 @@ Name of the file where @command{rsync} writes its lock file.
 @item @code{log-file} (default: @code{"/var/log/rsyncd.log"})
 Name of the file where @command{rsync} writes its log file.
 
-@item @code{use-chroot?} (default: @var{#t})
-Whether to use chroot for @command{rsync} shared directory.
-
-@item @code{share-path} (default: @file{/srv/rsync})
-Location of the @command{rsync} shared directory.
-
-@item @code{share-comment} (default: @code{"Rsync share"})
-Comment of the @command{rsync} shared directory.
-
-@item @code{read-only?} (default: @var{#f})
-Read-write permissions to shared directory.
-
-@item @code{timeout} (default: @code{300})
-I/O timeout in seconds.
-
-@item @code{user} (default: @var{"root"})
+@item @code{user} (default: @code{"root"})
 Owner of the @code{rsync} process.
 
-@item @code{group} (default: @var{"root"})
+@item @code{group} (default: @code{"root"})
 Group of the @code{rsync} process.
 
-@item @code{uid} (default: @var{"rsyncd"})
+@item @code{uid} (default: @code{"rsyncd"})
 User name or user ID that file transfers to and from that module should take
 place as when the daemon was run as @code{root}.
 
-@item @code{gid} (default: @var{"rsyncd"})
+@item @code{gid} (default: @code{"rsyncd"})
 Group name or group ID that will be used when accessing the module.
 
+@item @code{modules} (default: @code{%default-modules})
+List of ``modules''---i.e., directories exported over rsync.  Each
+element must be a @code{rsync-module} record, as described below.
+@end table
+@end deftp
+
+@deftp {Data Type} rsync-module
+This is the data type for rsync ``modules''.  A module is a directory
+exported over the rsync protocol.  The available fields are as follows:
+
+@table @asis
+@item @code{name}
+The module name.  This is the name that shows up in URLs.  For example,
+if the module is called @code{music}, the corresponding URL will be
+@code{rsync://host.example.org/music}.
+
+@item @code{file-name}
+Name of the directory being exported.
+
+@item @code{comment} (default: @code{""})
+Comment associated with the module.  Client user interfaces may display
+it when they obtain the list of available modules.
+
+@item @code{read-only?} (default: @code{#t})
+Whether or not client will be able to upload files.  If this is false,
+the uploads will be authorized if permissions on the daemon side permit
+it.
+
+@item @code{chroot?} (default: @code{#t})
+When this is true, the rsync daemon changes root to the module's
+directory before starting file transfers with the client.  This improves
+security, but requires rsync to run as root.
+
+@item @code{timeout} (default: @code{300})
+Idle time in seconds after which the daemon closes a connection with the
+client.
 @end table
 @end deftp
 
diff --git a/gnu/services/rsync.scm b/gnu/services/rsync.scm
index 6e27edde25..d456911563 100644
--- a/gnu/services/rsync.scm
+++ b/gnu/services/rsync.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017 Oleg Pykhalov <go.wigust <at> gmail.com>
+;;; Copyright © 2021 Ludovic Courtès <ludo <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -25,11 +26,23 @@ (define-module (gnu services rsync)
   #:use-module (gnu packages admin)
   #:use-module (guix records)
   #:use-module (guix gexp)
+  #:use-module (guix diagnostics)
+  #:use-module (guix i18n)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:use-module (ice-9 match)
   #:export (rsync-configuration
             rsync-configuration?
+            rsync-configuration-modules
+
+            rsync-module
+            rsync-module?
+            rsync-module-name
+            rsync-module-file-name
+            rsync-module-comment
+            rsync-module-read-only
+            rsync-module-timeout
+
             rsync-service-type))
 
 ;;;; Commentary:
@@ -39,6 +52,13 @@ (define-module (gnu services rsync)
 ;;;
 ;;;; Code:
 
+(define-with-syntax-properties (warn-share-field-deprecation (value properties))
+  (unless (unspecified? value)
+    (warning (source-properties->location properties)
+             (G_ "the 'share-path' and 'share-comment' fields is deprecated, \
+please use 'modules' instead~%")))
+  value)
+
 (define-record-type* <rsync-configuration>
   rsync-configuration
   make-rsync-configuration
@@ -56,15 +76,22 @@ (define-record-type* <rsync-configuration>
   (log-file      rsync-configuration-log-file             ; string
                  (default "/var/log/rsyncd.log"))
   (use-chroot?   rsync-configuration-use-chroot?          ; boolean
-                 (default #t))
+                 (sanitize warn-share-field-deprecation)
+                 (default *unspecified*))
+  (modules       rsync-configuration-actual-modules ;list of <rsync-module>
+                 (default %default-modules))  ;TODO: eventually remove default
   (share-path    rsync-configuration-share-path           ; string
-                 (default "/srv/rsyncd"))
+                 (sanitize warn-share-field-deprecation)
+                 (default *unspecified*))
   (share-comment rsync-configuration-share-comment        ; string
-                 (default "Rsync share"))
+                 (sanitize warn-share-field-deprecation)
+                 (default *unspecified*))
   (read-only?    rsync-configuration-read-only?           ; boolean
-                 (default #f))
+                 (sanitize warn-share-field-deprecation)
+                 (default *unspecified*))
   (timeout       rsync-configuration-timeout              ; integer
-                 (default 300))
+                 (sanitize warn-share-field-deprecation)
+                 (default *unspecified*))
   (user          rsync-configuration-user                 ; string
                  (default "root"))
   (group         rsync-configuration-group                ; string
@@ -74,6 +101,45 @@ (define-record-type* <rsync-configuration>
   (gid           rsync-configuration-gid                  ; string
                  (default "rsyncd")))
 
+;; Rsync "module": a directory exported the rsync protocol.
+(define-record-type* <rsync-module>
+  rsync-module make-rsync-module
+  rsync-module?
+  (name          rsync-module-name)               ;string
+  (file-name     rsync-module-file-name)          ;string
+  (comment       rsync-module-comment             ;string
+                 (default ""))
+  (read-only?    rsync-module-read-only?          ;boolean
+                 (default #t))
+  (chroot?       rsync-module-chroot?             ;boolean
+                 (default #t))
+  (timeout       rsync-module-timeout             ;integer
+                 (default 300)))
+
+(define %default-modules
+  ;; Default modules, provided for backward compatibility.
+  (list (rsync-module (name "files")
+                      (file-name "/srv/rsyncd")
+                      (comment "Rsync share")
+                      (read-only? #f))))          ;yes, that was the default
+
+(define (rsync-configuration-modules config)
+  (match-record config <rsync-configuration>
+    (modules
+     share-path share-comment use-chroot? read-only? timeout) ;deprecated
+    (if (unspecified? share-path)
+        (rsync-configuration-actual-modules config)
+        (list (rsync-module                       ;backward compatibility
+               (name "files")
+               (file-name share-path)
+               (comment "Rsync share")
+               (chroot?
+                (if (unspecified? use-chroot?) #t use-chroot?))
+               (read-only?
+                (if (unspecified? read-only?) #f read-only?))
+               (timeout
+                (if (unspecified? timeout) 300 timeout)))))))
+
 (define (rsync-account config)
   "Return the user accounts and user groups for CONFIG."
   (let ((rsync-user (if (rsync-configuration-uid config)
@@ -96,55 +162,62 @@ (define (rsync-activation config)
   "Return the activation GEXP for CONFIG."
   (with-imported-modules '((guix build utils))
     #~(begin
-        (let ((share-directory  #$(rsync-configuration-share-path config))
-              (user  (getpw (if #$(rsync-configuration-uid config)
+        (let ((user  (getpw (if #$(rsync-configuration-uid config)
                                 #$(rsync-configuration-uid config)
                                 #$(rsync-configuration-user config))))
               (group (getpw (if #$(rsync-configuration-gid config)
                                 #$(rsync-configuration-gid config)
                                 #$(rsync-configuration-group config)))))
           (mkdir-p (dirname #$(rsync-configuration-pid-file config)))
-          (and=> share-directory mkdir-p)
-          (chown share-directory
-                 (passwd:uid user)
-                 (group:gid group))))))
+          (for-each (lambda (directory)
+                      (mkdir-p directory)
+                      (chown directory (passwd:uid user) (group:gid group)))
+                    '#$(map rsync-module-file-name
+                            (rsync-configuration-modules config)))))))
 
-(define rsync-config-file
+(define (rsync-config-file config)
   ;; Return the rsync configuration file corresponding to CONFIG.
-  (match-lambda
-    (($ <rsync-configuration> package address port-number pid-file lock-file log-file
-                              use-chroot? share-path share-comment read-only?
-                              timeout user group uid gid)
-     (if (not (string=? user "root"))
-         (cond
-          ((<= port-number 1024)
-           (error (string-append "rsync-service: to run on port "
-                                 (number->string port-number)
-                                 ", user must be root.")))
-          (use-chroot?
-           (error (string-append "rsync-service: to run in a chroot"
-                                 ", user must be root.")))
-          (uid
-           (error "rsync-service: to use uid, user must be root."))
-          (gid
-           (error "rsync-service: to use gid, user must be root."))))
-     (mixed-text-file
-      "rsync.conf"
-      "# Generated by 'rsync-service'.\n\n"
-      "pid file = " pid-file "\n"
-      "lock file = " lock-file "\n"
-      "log file = " log-file "\n"
-      (if address (string-append "address = " address "\n") "")
-      "port = " (number->string port-number) "\n"
-      "use chroot = " (if use-chroot? "true" "false") "\n"
-      (if uid (string-append "uid = " uid "\n") "")
-      "gid = " (if gid gid "nogroup") "\n" ; no group nobody
-      "\n"
-      "[files]\n"
-      "path = " share-path "\n"
-      "comment = " share-comment "\n"
-      "read only = " (if read-only? "true" "false") "\n"
-      "timeout = " (number->string timeout) "\n"))))
+  (define (module-config module)
+    (match-record module <rsync-module>
+      (name file-name comment chroot? read-only? timeout)
+      (list "[" name "]\n"
+            "  path = " file-name "\n"
+            "  use chroot = " (if chroot? "true" "false") "\n"
+            "  comment = " comment "\n"
+            "  read only = " (if read-only? "true" "false") "\n"
+            "  timeout = " (number->string timeout) "\n")))
+
+  (define modules
+    (rsync-configuration-modules config))
+
+  (match-record config <rsync-configuration>
+    (package address port-number pid-file lock-file log-file
+             user group uid gid)
+    (unless (string=? user "root")
+      (cond
+       ((<= port-number 1024)
+        (error (string-append "rsync-service: to run on port "
+                              (number->string port-number)
+                              ", user must be root.")))
+       ((find rsync-module-chroot? modules)
+        (error (string-append "rsync-service: to run in a chroot"
+                              ", user must be root.")))
+       (uid
+        (error "rsync-service: to use uid, user must be root."))
+       (gid
+        (error "rsync-service: to use gid, user must be root."))))
+
+    (apply mixed-text-file "rsync.conf"
+           "# Generated by 'rsync-service'.\n\n"
+           "pid file = " pid-file "\n"
+           "lock file = " lock-file "\n"
+           "log file = " log-file "\n"
+           (if address (string-append "address = " address "\n") "")
+           "port = " (number->string port-number) "\n"
+           (if uid (string-append "uid = " uid "\n") "")
+           "gid = " (if gid gid "nogroup") "\n"   ; no group nobody
+           "\n\n"
+           (append-map module-config modules))))
 
 (define (rsync-shepherd-service config)
   "Return a <shepherd-service> for rsync with CONFIG."
@@ -172,4 +245,7 @@ (define rsync-service-type
     (list (service-extension shepherd-root-service-type rsync-shepherd-service)
           (service-extension account-service-type rsync-account)
           (service-extension activation-service-type rsync-activation)))
-   (default-value (rsync-configuration))))
+   (default-value (rsync-configuration))
+   (description
+    "Run the rsync file copying tool in daemon mode.  This allows remote hosts
+to keep synchronized copies of the files exported by rsync.")))
-- 
2.33.0

Message #16 received at 52703-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 52703-done <at> debbugs.gnu.org
Subject: Re: bug#52703: [PATCH 0/2] Improving rsync-service-type
Date: Tue, 21 Dec 2021 15:50:19 +0100

Ludovic Courtès <ludo <at> gnu.org> skribis:

>   services: rsync: Allow configuring several rsync "modules".
>   tests: rsync: Define several modules.

We’ve super-fast-tracked these patches on #guix.  Pushed as
3b3bef3e4e3a8f38e8686262cd12e1786a9ac2b0.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.