From unknown Sat Jul 26 21:31:03 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#33508 <33508@debbugs.gnu.org> To: bug#33508 <33508@debbugs.gnu.org> Subject: Status: [PATCH] gnu: Add ability to restart services on system reconfigure Reply-To: bug#33508 <33508@debbugs.gnu.org> Date: Sun, 27 Jul 2025 04:31:03 +0000 retitle 33508 [PATCH] gnu: Add ability to restart services on system reconf= igure reassign 33508 guix-patches submitter 33508 Carlo Zancanaro severity 33508 normal tag 33508 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 26 06:41:48 2018 Received: (at submit) by debbugs.gnu.org; 26 Nov 2018 11:41:48 +0000 Received: from localhost ([127.0.0.1]:47887 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gRFGt-0004qK-Hv for submit@debbugs.gnu.org; Mon, 26 Nov 2018 06:41:48 -0500 Received: from eggs.gnu.org ([208.118.235.92]:51839) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gRFGr-0004q6-ND for submit@debbugs.gnu.org; Mon, 26 Nov 2018 06:41:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gRFGi-0003X7-6b for submit@debbugs.gnu.org; Mon, 26 Nov 2018 06:41:40 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:37938) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gRFGg-0003W2-GT for submit@debbugs.gnu.org; Mon, 26 Nov 2018 06:41:36 -0500 Received: from eggs.gnu.org ([208.118.235.92]:41741) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gRFGZ-0007Ly-Ec for guix-patches@gnu.org; Mon, 26 Nov 2018 06:41:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gRFGJ-0003Af-Jv for guix-patches@gnu.org; Mon, 26 Nov 2018 06:41:27 -0500 Received: from zancanaro.com.au ([45.76.117.151]:50490) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gRFGI-00036A-Pq for guix-patches@gnu.org; Mon, 26 Nov 2018 06:41:11 -0500 Received: from jolteon (210-1-202-160-cpe.spintel.net.au [210.1.202.160]) by zancanaro.com.au (Postfix) with ESMTPSA id ADF3623B19 for ; Mon, 26 Nov 2018 11:41:04 +0000 (UTC) User-agent: mu4e 1.0; emacs 26.1 From: Carlo Zancanaro To: guix-patches@gnu.org Subject: [PATCH] gnu: Add ability to restart services on system reconfigure Date: Mon, 26 Nov 2018 22:41:01 +1100 Message-ID: <87efb8m5gy.fsf@zancanaro.id.au> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.1 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.1 (-----) --=-=-= Content-Type: text/plain; format=flowed Hey Guix! A few months ago I mentioned the idea of adding the ability to have services automatically restarted when running "guix system reconfigure". These patches are a start on making that happen. They're incomplete (in particular, documentation is missing), but I'm offering them up for comment. The broad idea is to add a new field to our guix shepherd services: restart-strategy. There are three valid values: - always: this service is always safe to restart when running reconfigure - manual: this service may not be safe to restart when running reconfigure - a message will be printed telling the user to restart the service manually, or they can provide the --restart-services flag to reconfigure to automatically restart them - never: this service is never safe to restart when running reconfigure (eg. udev) I have added the flag to the guix daemon's shepherd service to show how it works. I tested this by changing my substitute servers in config.scm, and after running "reconfigure" I saw my updated substitute servers in ps without having to run "sudo herd restart guix-daemon". If nobody has any feedback in the next few days then I'll update the manual and send through another patch. Carlo --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-gnu-Add-ability-to-restart-services-on-system-reconf.patch >From 8b92ebac4fa13a2a89f279b249be152051f31d94 Mon Sep 17 00:00:00 2001 From: Carlo Zancanaro Date: Mon, 26 Nov 2018 22:38:08 +1100 Subject: [PATCH 1/3] gnu: Add ability to restart services on system reconfigure * gnu/services/herd.scm (restart-service): New procedure. * gnu/services/shepherd.scm ()[restart-strategy]: New field. (shepherd-service-upgrade): Return lists of services to automatically and manually restart. * guix/scripts/system.scm (call-with-service-upgrade-info): Pass through services to be automatically and manually restarted. (upgrade-shepherd-services): Automatically restart services that should be automatically restarted, and print a message about manually restarting services that should be manually restarted. --- gnu/services/herd.scm | 5 +++++ gnu/services/shepherd.scm | 35 ++++++++++++++++++++++-------- guix/scripts/system.scm | 45 ++++++++++++++++++++++++--------------- 3 files changed, 59 insertions(+), 26 deletions(-) diff --git a/gnu/services/herd.scm b/gnu/services/herd.scm index 8ff817759..c8d6eb04e 100644 --- a/gnu/services/herd.scm +++ b/gnu/services/herd.scm @@ -52,6 +52,7 @@ load-services load-services/safe start-service + restart-service stop-service)) ;;; Commentary: @@ -256,6 +257,10 @@ when passed a service with an already-registered name." (with-shepherd-action name ('start) result result)) +(define (restart-service name) + (with-shepherd-action name ('restart) result + result)) + (define (stop-service name) (with-shepherd-action name ('stop) result result)) diff --git a/gnu/services/shepherd.scm b/gnu/services/shepherd.scm index 49d08cc30..0c80e44f2 100644 --- a/gnu/services/shepherd.scm +++ b/gnu/services/shepherd.scm @@ -159,7 +159,9 @@ DEFAULT is given, use it as the service's default value." (auto-start? shepherd-service-auto-start? ;Boolean (default #t)) (modules shepherd-service-modules ;list of module names - (default %default-modules))) + (default %default-modules)) + (restart-strategy shepherd-service-restart-strategy + (default 'manual))) (define-record-type* shepherd-action make-shepherd-action @@ -344,9 +346,10 @@ symbols provided/required by a service." #t)))))) (define (shepherd-service-upgrade live target) - "Return two values: the subset of LIVE (a list of ) that needs -to be unloaded, and the subset of TARGET (a list of ) that -need to be restarted to complete their upgrade." + "Return three values: (a) the subset of LIVE (a list of ) that +needs to be unloaded, (b) the subset of TARGET (a list of ) +that can be restarted automatically, and (c) the subset of TARGET that must be +restarted manually." (define (essential? service) (memq (first (live-service-provision service)) '(root shepherd))) @@ -373,14 +376,28 @@ need to be restarted to complete their upgrade." (#f (every obsolete? (live-service-dependents service))) (_ #f))) - (define to-restart - ;; Restart services that are currently running. - (filter running? target)) - (define to-unload ;; Unload services that are no longer required. (remove essential? (filter obsolete? live))) - (values to-unload to-restart)) + (define to-automatically-restart + ;; Automatically restart services that are currently running and can + ;; always be restarted. + (filter (lambda (service) + (and (running? service) + (eq? (shepherd-service-restart-strategy service) + 'always))) + target)) + + (define to-manually-restart + ;; Manually restart services that are currently running and must be + ;; manually restarted. + (filter (lambda (service) + (and (running? service) + (eq? (shepherd-service-restart-strategy service) + 'manual))) + target)) + + (values to-unload to-automatically-restart to-manually-restart)) ;;; shepherd.scm ends here diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index d92ec7d5a..6f14b1395 100644 --- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -322,11 +322,12 @@ names of services to load (upgrade), and the list of names of services to unload." (match (current-services) ((services ...) - (let-values (((to-unload to-restart) + (let-values (((to-unload to-automatically-restart to-manually-restart) (shepherd-service-upgrade services new-services))) - (mproc to-restart - (map (compose first live-service-provision) - to-unload)))) + (mproc (map (compose first live-service-provision) + to-unload) + to-automatically-restart + to-manually-restart))) (#f (with-monad %store-monad (warning (G_ "failed to obtain list of shepherd services~%")) @@ -347,7 +348,7 @@ bring the system down." ;; Arrange to simply emit a warning if the service upgrade fails. (with-shepherd-error-handling (call-with-service-upgrade-info new-services - (lambda (to-restart to-unload) + (lambda (to-unload to-automatically-restart to-manually-restart) (for-each (lambda (unload) (info (G_ "unloading service '~a'...~%") unload) (unload-service unload)) @@ -355,27 +356,37 @@ bring the system down." (with-monad %store-monad (munless (null? new-services) - (let ((new-service-names (map shepherd-service-canonical-name new-services)) - (to-restart-names (map shepherd-service-canonical-name to-restart)) - (to-start (filter shepherd-service-auto-start? new-services))) - (info (G_ "loading new services:~{ ~a~}...~%") new-service-names) - (unless (null? to-restart-names) - ;; Listing TO-RESTART-NAMES in the message below wouldn't help - ;; because many essential services cannot be meaningfully - ;; restarted. See . - (format #t (G_ "To complete the upgrade, run 'herd restart SERVICE' to stop, -upgrade, and restart each service that was not automatically restarted.\n"))) + (let ((new-service-names (map shepherd-service-canonical-name new-services)) + (to-start-names (map shepherd-service-canonical-name (filter shepherd-service-auto-start? new-services))) + (to-automatically-restart-names (map shepherd-service-canonical-name to-automatically-restart)) + (to-manually-restart-names (map shepherd-service-canonical-name to-manually-restart))) + (set! to-start-names + (remove (lambda (name) + (or (member name to-automatically-restart-names) + (member name to-manually-restart-names))) + to-start-names)) + (mlet %store-monad ((files (mapm %store-monad (compose lower-object shepherd-service-file) new-services))) + (for-each restart-service to-automatically-restart-names) + ;; Here we assume that FILES are exactly those that were computed ;; as part of the derivation that built OS, which is normally the ;; case. + (info (G_ "loading new services:~{ ~a~}~%") new-service-names) (load-services/safe (map derivation->output-path files)) - + (info (G_ "starting services:~{ ~a~}~%") to-start-names) (for-each start-service - (map shepherd-service-canonical-name to-start)) + to-start-names) + (info (G_ "restarting services:~{ ~a~}~%") to-automatically-restart-names) + (for-each restart-service + to-automatically-restart-names) + + (unless (null? to-manually-restart-names) + (format #t (G_ "To complete the upgrade, the following services need to be manually restarted:~{ ~a~}~%") + to-manually-restart-names)) (return #t))))))))) (define* (switch-to-system os -- 2.19.1 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-system-Add-restart-services-flag-for-reconfigure.patch >From 3fdef27c8f11b6a0f013afa9b6e619659ce78dec Mon Sep 17 00:00:00 2001 From: Carlo Zancanaro Date: Mon, 26 Nov 2018 22:38:18 +1100 Subject: [PATCH 2/3] system: Add --restart-services flag for reconfigure * guix/scripts/system.scm (upgrade-shepherd-services): Add parameter to automatically restart services marked as needing manual restart. (switch-to-system): Pass through restart-services? flag. (perform-action): Pass through restart-services? flag. (%options): Add --restart-services flag. (%default-options): Add #f as default value for --restart-services flag. (process-action): Pass through restart-services? flag. --- guix/scripts/system.scm | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index 6f14b1395..bf632c534 100644 --- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -333,7 +333,7 @@ unload." (warning (G_ "failed to obtain list of shepherd services~%")) (return #f))))) -(define (upgrade-shepherd-services os) +(define (upgrade-shepherd-services os restart-services?) "Upgrade the Shepherd (PID 1) by unloading obsolete services and loading new services specified in OS and not currently running. @@ -360,6 +360,10 @@ bring the system down." (to-start-names (map shepherd-service-canonical-name (filter shepherd-service-auto-start? new-services))) (to-automatically-restart-names (map shepherd-service-canonical-name to-automatically-restart)) (to-manually-restart-names (map shepherd-service-canonical-name to-manually-restart))) + (when restart-services? + (set! to-automatically-restart-names (append to-automatically-restart-names + to-manually-restart-names)) + (set! to-manually-restart-names '())) (set! to-start-names (remove (lambda (name) (or (member name to-automatically-restart-names) @@ -389,7 +393,7 @@ bring the system down." to-manually-restart-names)) (return #t))))))))) -(define* (switch-to-system os +(define* (switch-to-system os restart-services? #:optional (profile %system-profile)) "Make a new generation of PROFILE pointing to the directory of OS, switch to it atomically, and then run OS's activation script." @@ -417,7 +421,7 @@ it atomically, and then run OS's activation script." (primitive-load (derivation->output-path script)))) ;; Finally, try to update system services. - (upgrade-shepherd-services os)))) + (upgrade-shepherd-services os restart-services?)))) (define-syntax-rule (unless-file-not-found exp) (catch 'system-error @@ -825,7 +829,8 @@ and TARGET arguments." use-substitutes? bootloader-target target image-size file-system-type full-boot? (mappings '()) - (gc-root #f)) + (gc-root #f) + (restart-services? #f)) "Perform ACTION for OS. INSTALL-BOOTLOADER? specifies whether to install bootloader; BOOTLOADER-TAGET is the target for the bootloader; TARGET is the target root directory; IMAGE-SIZE is the size of the image to be built, for @@ -907,7 +912,7 @@ static checks." (case action ((reconfigure) (mbegin %store-monad - (switch-to-system os) + (switch-to-system os restart-services?) (mwhen install-bootloader? (install-bootloader bootloader-script #:bootcfg bootcfg @@ -1090,6 +1095,9 @@ Some ACTIONS support additional ARGS.\n")) (option '(#\r "root") #t #f (lambda (opt name arg result) (alist-cons 'gc-root arg result))) + (option '("restart-services") #f #f + (lambda (opt name arg result) + (alist-cons 'restart-services? #t result))) %standard-build-options)) (define %default-options @@ -1104,7 +1112,8 @@ Some ACTIONS support additional ARGS.\n")) (verbosity . 0) (file-system-type . "ext4") (image-size . guess) - (install-bootloader? . #t))) + (install-bootloader? . #t) + (restart-services? . #f))) ;;; @@ -1177,7 +1186,8 @@ resulting from command-line parsing." #:install-bootloader? bootloader? #:target target #:bootloader-target bootloader-target - #:gc-root (assoc-ref opts 'gc-root))))) + #:gc-root (assoc-ref opts 'gc-root) + #:restart-services? (assoc-ref opts 'restart-services?))))) #:system system)) (warn-about-disk-space))) -- 2.19.1 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0003-services-Always-restart-guix-daemon.patch >From 099a8e2e6e28b38816ed1ba895c407f1d9efe62e Mon Sep 17 00:00:00 2001 From: Carlo Zancanaro Date: Mon, 26 Nov 2018 22:38:26 +1100 Subject: [PATCH 3/3] services: Always restart guix daemon * gnu/services/base.scm (guix-shepherd-service): Add restart-strategy of 'always. --- gnu/services/base.scm | 1 + 1 file changed, 1 insertion(+) diff --git a/gnu/services/base.scm b/gnu/services/base.scm index 228d3c592..7e0fdcb3e 100644 --- a/gnu/services/base.scm +++ b/gnu/services/base.scm @@ -1573,6 +1573,7 @@ failed to register hydra.gnu.org public key: ~a~%" status)))))))) (documentation "Run the Guix daemon.") (provision '(guix-daemon)) (requirement '(user-processes)) + (restart-strategy 'always) (modules '((srfi srfi-1))) (start #~(make-forkexec-constructor -- 2.19.1 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 26 07:42:08 2018 Received: (at 33508) by debbugs.gnu.org; 26 Nov 2018 12:42:09 +0000 Received: from localhost ([127.0.0.1]:47916 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gRGDH-0008M5-Uq for submit@debbugs.gnu.org; Mon, 26 Nov 2018 07:42:08 -0500 Received: from mail.lassieur.org ([83.152.10.219]:40922) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gRGDF-0008Lr-Js for 33508@debbugs.gnu.org; Mon, 26 Nov 2018 07:42:06 -0500 Received: from newt (smtp.parrot.biz [62.23.167.188]) by mail.lassieur.org (OpenSMTPD) with ESMTPSA id 8f97729a (TLSv1.2:ECDHE-RSA-CHACHA20-POLY1305:256:NO); Mon, 26 Nov 2018 12:42:02 +0000 (UTC) References: <87efb8m5gy.fsf@zancanaro.id.au> User-agent: mu4e 1.0; emacs 26.1 From: =?utf-8?Q?Cl=C3=A9ment?= Lassieur To: Carlo Zancanaro Subject: Re: [bug#33508] [PATCH] gnu: Add ability to restart services on system reconfigure In-reply-to: <87efb8m5gy.fsf@zancanaro.id.au> Date: Mon, 26 Nov 2018 13:42:02 +0100 Message-ID: <871s78ypr9.fsf@lassieur.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33508 Cc: 33508@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Carlo, It might be safer to 'reload' some services, rather than 'restarting' them. E.g. for nginx and prosody. Do you think it would be possible add a 'custom' value that would point to a custom Shepherd action? Thank you for your work on this! Cl=C3=A9ment Carlo Zancanaro writes: > Hey Guix! > > A few months ago I mentioned the idea of adding the ability to=20 > have services automatically restarted when running "guix system=20 > reconfigure". These patches are a start on making that happen.=20 > They're incomplete (in particular, documentation is missing), but=20 > I'm offering them up for comment. > > The broad idea is to add a new field to our guix shepherd=20 > services: restart-strategy. There are three valid values: > > - always: this service is always safe to restart when running=20 > reconfigure >=20=20 > - manual: this service may not be safe to restart when running=20 > reconfigure - a message will be printed telling the user to=20 > restart the service manually, or they can provide the=20 > --restart-services flag to reconfigure to automatically restart=20 > them > > - never: this service is never safe to restart when running=20 > reconfigure (eg. udev) > > I have added the flag to the guix daemon's shepherd service to=20 > show how it works. I tested this by changing my substitute servers=20 > in config.scm, and after running "reconfigure" I saw my updated=20 > substitute servers in ps without having to run "sudo herd restart=20 > guix-daemon". > > If nobody has any feedback in the next few days then I'll update=20 > the manual and send through another patch. > > Carlo > > From 8b92ebac4fa13a2a89f279b249be152051f31d94 Mon Sep 17 00:00:00 2001 > From: Carlo Zancanaro > Date: Mon, 26 Nov 2018 22:38:08 +1100 > Subject: [PATCH 1/3] gnu: Add ability to restart services on system > reconfigure > > * gnu/services/herd.scm (restart-service): New procedure. > * gnu/services/shepherd.scm ()[restart-strategy]: New > field. > (shepherd-service-upgrade): Return lists of services to automatically a= nd > manually restart. > * guix/scripts/system.scm (call-with-service-upgrade-info): Pass through > services to be automatically and manually restarted. > (upgrade-shepherd-services): Automatically restart services that should= be > automatically restarted, and print a message about manually restarting > services that should be manually restarted. > --- > gnu/services/herd.scm | 5 +++++ > gnu/services/shepherd.scm | 35 ++++++++++++++++++++++-------- > guix/scripts/system.scm | 45 ++++++++++++++++++++++++--------------- > 3 files changed, 59 insertions(+), 26 deletions(-) > > diff --git a/gnu/services/herd.scm b/gnu/services/herd.scm > index 8ff817759..c8d6eb04e 100644 > --- a/gnu/services/herd.scm > +++ b/gnu/services/herd.scm > @@ -52,6 +52,7 @@ > load-services > load-services/safe > start-service > + restart-service > stop-service)) >=20=20 > ;;; Commentary: > @@ -256,6 +257,10 @@ when passed a service with an already-registered nam= e." > (with-shepherd-action name ('start) result > result)) >=20=20 > +(define (restart-service name) > + (with-shepherd-action name ('restart) result > + result)) > + > (define (stop-service name) > (with-shepherd-action name ('stop) result > result)) > diff --git a/gnu/services/shepherd.scm b/gnu/services/shepherd.scm > index 49d08cc30..0c80e44f2 100644 > --- a/gnu/services/shepherd.scm > +++ b/gnu/services/shepherd.scm > @@ -159,7 +159,9 @@ DEFAULT is given, use it as the service's default val= ue." > (auto-start? shepherd-service-auto-start? ;Boolean > (default #t)) > (modules shepherd-service-modules ;list of module n= ames > - (default %default-modules))) > + (default %default-modules)) > + (restart-strategy shepherd-service-restart-strategy > + (default 'manual))) >=20=20 > (define-record-type* > shepherd-action make-shepherd-action > @@ -344,9 +346,10 @@ symbols provided/required by a service." > #t)))))) >=20=20 > (define (shepherd-service-upgrade live target) > - "Return two values: the subset of LIVE (a list of ) that= needs > -to be unloaded, and the subset of TARGET (a list of ) = that > -need to be restarted to complete their upgrade." > + "Return three values: (a) the subset of LIVE (a list of = ) that > +needs to be unloaded, (b) the subset of TARGET (a list of ) > +that can be restarted automatically, and (c) the subset of TARGET that m= ust be > +restarted manually." > (define (essential? service) > (memq (first (live-service-provision service)) > '(root shepherd))) > @@ -373,14 +376,28 @@ need to be restarted to complete their upgrade." > (#f (every obsolete? (live-service-dependents service))) > (_ #f))) >=20=20 > - (define to-restart > - ;; Restart services that are currently running. > - (filter running? target)) > - > (define to-unload > ;; Unload services that are no longer required. > (remove essential? (filter obsolete? live))) >=20=20 > - (values to-unload to-restart)) > + (define to-automatically-restart > + ;; Automatically restart services that are currently running and can > + ;; always be restarted. > + (filter (lambda (service) > + (and (running? service) > + (eq? (shepherd-service-restart-strategy service) > + 'always))) > + target)) > + > + (define to-manually-restart > + ;; Manually restart services that are currently running and must be > + ;; manually restarted. > + (filter (lambda (service) > + (and (running? service) > + (eq? (shepherd-service-restart-strategy service) > + 'manual))) > + target)) > + > + (values to-unload to-automatically-restart to-manually-restart)) >=20=20 > ;;; shepherd.scm ends here > diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm > index d92ec7d5a..6f14b1395 100644 > --- a/guix/scripts/system.scm > +++ b/guix/scripts/system.scm > @@ -322,11 +322,12 @@ names of services to load (upgrade), and the list o= f names of services to > unload." > (match (current-services) > ((services ...) > - (let-values (((to-unload to-restart) > + (let-values (((to-unload to-automatically-restart to-manually-resta= rt) > (shepherd-service-upgrade services new-services))) > - (mproc to-restart > - (map (compose first live-service-provision) > - to-unload)))) > + (mproc (map (compose first live-service-provision) > + to-unload) > + to-automatically-restart > + to-manually-restart))) > (#f > (with-monad %store-monad > (warning (G_ "failed to obtain list of shepherd services~%")) > @@ -347,7 +348,7 @@ bring the system down." > ;; Arrange to simply emit a warning if the service upgrade fails. > (with-shepherd-error-handling > (call-with-service-upgrade-info new-services > - (lambda (to-restart to-unload) > + (lambda (to-unload to-automatically-restart to-manually-restart) > (for-each (lambda (unload) > (info (G_ "unloading service '~a'...~%") unload) > (unload-service unload)) > @@ -355,27 +356,37 @@ bring the system down." >=20=20 > (with-monad %store-monad > (munless (null? new-services) > - (let ((new-service-names (map shepherd-service-canonical-na= me new-services)) > - (to-restart-names (map shepherd-service-canonical-na= me to-restart)) > - (to-start (filter shepherd-service-auto-star= t? new-services))) > - (info (G_ "loading new services:~{ ~a~}...~%") new-service= -names) > - (unless (null? to-restart-names) > - ;; Listing TO-RESTART-NAMES in the message below wouldn'= t help > - ;; because many essential services cannot be meaningfully > - ;; restarted. See . > - (format #t (G_ "To complete the upgrade, run 'herd resta= rt SERVICE' to stop, > -upgrade, and restart each service that was not automatically restarted.\= n"))) > + (let ((new-service-names (map shepherd-service-= canonical-name new-services)) > + (to-start-names (map shepherd-service-= canonical-name (filter shepherd-service-auto-start? new-services))) > + (to-automatically-restart-names (map shepherd-service-= canonical-name to-automatically-restart)) > + (to-manually-restart-names (map shepherd-service-= canonical-name to-manually-restart))) > + (set! to-start-names > + (remove (lambda (name) > + (or (member name to-automatically-restart-name= s) > + (member name to-manually-restart-names))) > + to-start-names)) > + > (mlet %store-monad ((files (mapm %store-monad > (compose lower-object > shepherd-service= -file) > new-services))) > + (for-each restart-service to-automatically-restart-names) > + > ;; Here we assume that FILES are exactly those that were= computed > ;; as part of the derivation that built OS, which is nor= mally the > ;; case. > + (info (G_ "loading new services:~{ ~a~}~%") new-service-= names) > (load-services/safe (map derivation->output-path files)) > - > + (info (G_ "starting services:~{ ~a~}~%") to-start-names) > (for-each start-service > - (map shepherd-service-canonical-name to-start)) > + to-start-names) > + (info (G_ "restarting services:~{ ~a~}~%") to-automatica= lly-restart-names) > + (for-each restart-service > + to-automatically-restart-names) > + > + (unless (null? to-manually-restart-names) > + (format #t (G_ "To complete the upgrade, the following= services need to be manually restarted:~{ ~a~}~%") > + to-manually-restart-names)) > (return #t))))))))) >=20=20 > (define* (switch-to-system os > --=20 > 2.19.1 > > From 3fdef27c8f11b6a0f013afa9b6e619659ce78dec Mon Sep 17 00:00:00 2001 > From: Carlo Zancanaro > Date: Mon, 26 Nov 2018 22:38:18 +1100 > Subject: [PATCH 2/3] system: Add --restart-services flag for reconfigure > > * guix/scripts/system.scm (upgrade-shepherd-services): Add parameter to > automatically restart services marked as needing manual restart. > (switch-to-system): Pass through restart-services? flag. > (perform-action): Pass through restart-services? flag. > (%options): Add --restart-services flag. > (%default-options): Add #f as default value for --restart-services flag. > (process-action): Pass through restart-services? flag. > --- > guix/scripts/system.scm | 24 +++++++++++++++++------- > 1 file changed, 17 insertions(+), 7 deletions(-) > > diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm > index 6f14b1395..bf632c534 100644 > --- a/guix/scripts/system.scm > +++ b/guix/scripts/system.scm > @@ -333,7 +333,7 @@ unload." > (warning (G_ "failed to obtain list of shepherd services~%")) > (return #f))))) >=20=20 > -(define (upgrade-shepherd-services os) > +(define (upgrade-shepherd-services os restart-services?) > "Upgrade the Shepherd (PID 1) by unloading obsolete services and loadi= ng new > services specified in OS and not currently running. >=20=20 > @@ -360,6 +360,10 @@ bring the system down." > (to-start-names (map shepherd-service-= canonical-name (filter shepherd-service-auto-start? new-services))) > (to-automatically-restart-names (map shepherd-service-= canonical-name to-automatically-restart)) > (to-manually-restart-names (map shepherd-service-= canonical-name to-manually-restart))) > + (when restart-services? > + (set! to-automatically-restart-names (append to-automati= cally-restart-names > + to-manually= -restart-names)) > + (set! to-manually-restart-names '())) > (set! to-start-names > (remove (lambda (name) > (or (member name to-automatically-restart-name= s) > @@ -389,7 +393,7 @@ bring the system down." > to-manually-restart-names)) > (return #t))))))))) >=20=20 > -(define* (switch-to-system os > +(define* (switch-to-system os restart-services? > #:optional (profile %system-profile)) > "Make a new generation of PROFILE pointing to the directory of OS, swi= tch to > it atomically, and then run OS's activation script." > @@ -417,7 +421,7 @@ it atomically, and then run OS's activation script." > (primitive-load (derivation->output-path script)))) >=20=20 > ;; Finally, try to update system services. > - (upgrade-shepherd-services os)))) > + (upgrade-shepherd-services os restart-services?)))) >=20=20 > (define-syntax-rule (unless-file-not-found exp) > (catch 'system-error > @@ -825,7 +829,8 @@ and TARGET arguments." > use-substitutes? bootloader-target target > image-size file-system-type full-boot? > (mappings '()) > - (gc-root #f)) > + (gc-root #f) > + (restart-services? #f)) > "Perform ACTION for OS. INSTALL-BOOTLOADER? specifies whether to inst= all > bootloader; BOOTLOADER-TAGET is the target for the bootloader; TARGET is= the > target root directory; IMAGE-SIZE is the size of the image to be built, = for > @@ -907,7 +912,7 @@ static checks." > (case action > ((reconfigure) > (mbegin %store-monad > - (switch-to-system os) > + (switch-to-system os restart-services?) > (mwhen install-bootloader? > (install-bootloader bootloader-script > #:bootcfg bootcfg > @@ -1090,6 +1095,9 @@ Some ACTIONS support additional ARGS.\n")) > (option '(#\r "root") #t #f > (lambda (opt name arg result) > (alist-cons 'gc-root arg result))) > + (option '("restart-services") #f #f > + (lambda (opt name arg result) > + (alist-cons 'restart-services? #t result))) > %standard-build-options)) >=20=20 > (define %default-options > @@ -1104,7 +1112,8 @@ Some ACTIONS support additional ARGS.\n")) > (verbosity . 0) > (file-system-type . "ext4") > (image-size . guess) > - (install-bootloader? . #t))) > + (install-bootloader? . #t) > + (restart-services? . #f))) >=20=20 > > ;;; > @@ -1177,7 +1186,8 @@ resulting from command-line parsing." > #:install-bootloader? bootloader? > #:target target > #:bootloader-target bootloader-target > - #:gc-root (assoc-ref opts 'gc-root))))) > + #:gc-root (assoc-ref opts 'gc-root) > + #:restart-services? (assoc-ref opts 'restar= t-services?))))) > #:system system)) > (warn-about-disk-space))) >=20=20 > --=20 > 2.19.1 > > From 099a8e2e6e28b38816ed1ba895c407f1d9efe62e Mon Sep 17 00:00:00 2001 > From: Carlo Zancanaro > Date: Mon, 26 Nov 2018 22:38:26 +1100 > Subject: [PATCH 3/3] services: Always restart guix daemon > > * gnu/services/base.scm (guix-shepherd-service): Add restart-strategy of > 'always. > --- > gnu/services/base.scm | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/gnu/services/base.scm b/gnu/services/base.scm > index 228d3c592..7e0fdcb3e 100644 > --- a/gnu/services/base.scm > +++ b/gnu/services/base.scm > @@ -1573,6 +1573,7 @@ failed to register hydra.gnu.org public key: ~a~%" = status)))))))) > (documentation "Run the Guix daemon.") > (provision '(guix-daemon)) > (requirement '(user-processes)) > + (restart-strategy 'always) > (modules '((srfi srfi-1))) > (start > #~(make-forkexec-constructor From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 26 15:11:07 2018 Received: (at 33508) by debbugs.gnu.org; 26 Nov 2018 20:11:07 +0000 Received: from localhost ([127.0.0.1]:50163 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gRNDn-0002RD-66 for submit@debbugs.gnu.org; Mon, 26 Nov 2018 15:11:07 -0500 Received: from zancanaro.com.au ([45.76.117.151]:44846) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gRNDl-0002R4-6v for 33508@debbugs.gnu.org; Mon, 26 Nov 2018 15:11:06 -0500 Received: from jolteon (210-1-202-160-cpe.spintel.net.au [210.1.202.160]) by zancanaro.com.au (Postfix) with ESMTPSA id BCD9623B15; Mon, 26 Nov 2018 20:11:02 +0000 (UTC) References: <87efb8m5gy.fsf@zancanaro.id.au> <871s78ypr9.fsf@lassieur.org> User-agent: mu4e 1.0; emacs 26.1 From: Carlo Zancanaro To: =?utf-8?Q?Cl=C3=A9ment?= Lassieur Subject: Re: [bug#33508] [PATCH] gnu: Add ability to restart services on system reconfigure In-reply-to: <871s78ypr9.fsf@lassieur.org> Date: Tue, 27 Nov 2018 07:11:00 +1100 Message-ID: <875zwj8uqz.fsf@zancanaro.id.au> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33508 Cc: 33508@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hey Cl=C3=A9ment! On Mon, Nov 26 2018, Cl=C3=A9ment Lassieur wrote: > It might be safer to 'reload' some services, rather than=20 > 'restarting' them. E.g. for nginx and prosody. Do you think it=20 > would be possible add a 'custom' value that would point to a=20 > custom Shepherd action? I can add this, but I don't think this is as useful as it=20 initially sounds. Most of our services are a specific version of a=20 service pointing to a specific version of a configuration file=20 (ie. that's in the store). That means that a "reload" shepherd=20 action won't be able to know where the new configuration file is=20 to load it. We could solve this in one of two ways: 1) by allowing an arbitrary procedure as the value of=20 restart-strategy, because it can then call a shepherd action with=20 the appropriate configuration file, but then our action will have=20 to detect whether the binary has been changed (which would also=20 detect any dependencies changing). This may also lead to an=20 inconsistent user experience where a "reconfigure" might lead to a=20 reload, or might lead to a restart, and it's not obvious which it=20 will be. 2) by changing our services to create configuration files in a=20 known location (ie. /etc/nginx/nginx.conf). This would make it so=20 a simple "reload" action in the service could meaningfully reload=20 the service, but only if the binary was unchanged (because the old=20 binary might not be able to read the new configuration format, for=20 instance). This still leads to the above problem around the=20 inconsistent user experience, and adds some complexity in terms of=20 how configuration files are managed. I lean towards option (1), because it gives us the ability to call=20 a shepherd action if we want, but also allows us to do arbitrary=20 other things with the extra knowledge on the Guix side. In the end, though, I'm unconvinced that this is a useful thing to=20 add. What do you think? Carlo From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 26 16:02:22 2018 Received: (at 33508) by debbugs.gnu.org; 26 Nov 2018 21:02:22 +0000 Received: from localhost ([127.0.0.1]:50184 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gRO1O-0003jl-4i for submit@debbugs.gnu.org; Mon, 26 Nov 2018 16:02:22 -0500 Received: from mail.lassieur.org ([83.152.10.219]:40934) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gRO1M-0003jc-67 for 33508@debbugs.gnu.org; Mon, 26 Nov 2018 16:02:20 -0500 Received: from rodion (88.191.118.83 [88.191.118.83]) by mail.lassieur.org (OpenSMTPD) with ESMTPSA id 2191bfc6 (TLSv1.2:ECDHE-RSA-CHACHA20-POLY1305:256:NO); Mon, 26 Nov 2018 21:02:18 +0000 (UTC) References: <87efb8m5gy.fsf@zancanaro.id.au> <871s78ypr9.fsf@lassieur.org> <875zwj8uqz.fsf@zancanaro.id.au> User-agent: mu4e 1.0; emacs 26.1 From: =?utf-8?Q?Cl=C3=A9ment?= Lassieur To: Carlo Zancanaro Subject: Re: [bug#33508] [PATCH] gnu: Add ability to restart services on system reconfigure In-reply-to: <875zwj8uqz.fsf@zancanaro.id.au> Date: Mon, 26 Nov 2018 22:02:17 +0100 Message-ID: <87k1kzd02u.fsf@lassieur.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33508 Cc: 33508@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hey Carlo! Carlo Zancanaro writes: > I can add this, but I don't think this is as useful as it initially > sounds. Most of our services are a specific version of a service pointing= to a > specific version of a configuration file (ie. that's in the store). That = means > that a "reload" shepherd action won't be able to know where the new > configuration file is to load it. > > We could solve this in one of two ways: > > 1) by allowing an arbitrary procedure as the value of restart-strategy, > because it can then call a shepherd action with the appropriate configura= tion > file > but then our action will have to detect whether the binary has been > changed (which would also detect any dependencies changing). I don't think it needs to detect whether the binary has changed, because 'reload' signals are usually implemented so that they can safely fail. So, if the configuration file has changed in an incompatible way, the 'reload' action won't work, but the service will keep running. > This may also lead to an inconsistent user experience where a > "reconfigure" might lead to a reload, or might lead to a restart, and > it's not obvious which it will be. Your patch also leads to this inconsistency, because it allows a service to either be restarted or not, in my opinion :-) > 2) by changing our services to create configuration files in a known loca= tion > (ie. /etc/nginx/nginx.conf). This would make it so a simple "reload" acti= on in > the service could meaningfully reload the service, but only if the binary= was > unchanged (because the old binary might not be able to read the new > configuration format, for instance). This still leads to the above problem > around the inconsistent user experience, and adds some complexity in term= s of > how configuration files are managed. > > I lean towards option (1), because it gives us the ability to call a shep= herd > action if we want, but also allows us to do arbitrary other things with t= he > extra knowledge on the Guix side. I think both (1) and (2) make sense because both kind of services (the ones pointing to configuration files in the store and the ones using /etc/some-file.conf) already exist. Ideally, the mechanism should be generic enough to handle both cases. > In the end, though, I'm unconvinced that this is a useful thing to add. W= hat > do you think? I don't agree :-). A 'restart' is inherently dangerous because there is a chance for the restart to fail (say, if the new configuration file is erroneous), whereas the 'reload' action cannot fail (if it is correctly implemented). That being said, I agree that adding support for 'reload' would lead to more complexity, and I would understand if you don't add it :-), but I still think it's a very useful feature. One question though: my understanding is that the default value for 'restart-strategy' is set in the Guix repository, but a user would be able to customize it in their config.scm. Can you confirm it? Thank you, Cl=C3=A9ment From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 26 16:59:35 2018 Received: (at 33508) by debbugs.gnu.org; 26 Nov 2018 21:59:35 +0000 Received: from localhost ([127.0.0.1]:50216 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gROul-00055U-2d for submit@debbugs.gnu.org; Mon, 26 Nov 2018 16:59:35 -0500 Received: from zancanaro.com.au ([45.76.117.151]:48306) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gROui-00055K-87 for 33508@debbugs.gnu.org; Mon, 26 Nov 2018 16:59:33 -0500 Received: from jolteon (110-175-91-30.tpgi.com.au [110.175.91.30]) by zancanaro.com.au (Postfix) with ESMTPSA id 0D103240A1; Mon, 26 Nov 2018 21:59:29 +0000 (UTC) References: <87efb8m5gy.fsf@zancanaro.id.au> <871s78ypr9.fsf@lassieur.org> <875zwj8uqz.fsf@zancanaro.id.au> <87k1kzd02u.fsf@lassieur.org> User-agent: mu4e 1.0; emacs 26.1 From: Carlo Zancanaro To: =?utf-8?Q?Cl=C3=A9ment?= Lassieur Subject: Re: [bug#33508] [PATCH] gnu: Add ability to restart services on system reconfigure In-reply-to: <87k1kzd02u.fsf@lassieur.org> Date: Tue, 27 Nov 2018 08:59:28 +1100 Message-ID: <87lg5fsdof.fsf@zancanaro.id.au> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33508 Cc: 33508@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hey Cl=C3=A9ment, Thanks for your thoughts! I think you're right that the approach=20 I've implemented isn't flexible enough. I potentially haven't=20 thought through the failure cases enough. I was more thinking of=20 reload as providing "zero downtime" upgrades, rather than=20 providing a safer way to upgrade. I'll respond more specifically inline. On Tue, Nov 27 2018, Cl=C3=A9ment Lassieur wrote: > I don't think it needs to detect whether the binary has changed,=20 > because 'reload' signals are usually implemented so that they=20 > can safely fail. So, if the configuration file has changed in=20 > an incompatible way, the 'reload' action won't work, but the=20 > service will keep running. We do need to detect whether the binary has changed for the sake=20 of security updates, or similar. It would be bad if a user=20 reconfigured their system and it didn't upgrade the version of=20 nginx (or its dependencies) that they're running. Broadly speaking, I conceptualise reconfigure as "bring my system=20 into this state". Now, thus far we haven't been able to do that,=20 because we have lacked the ability to restart services properly,=20 but in my mind the ideal situation is that after running "guix=20 system reconfigure" our system is completely put into the state=20 specified by the config.scm file used. Although, now that I type that out, I notice that there is one=20 obvious way in which that is not true: the kernel. We can't=20 hot-swap the kernel, so there can always be a difference between=20 what the configuration file specifies and what the system is=20 actually running. At any rate, even if we give services the ability to reload=20 without restarting, they would need to print out a message to=20 prompt the user to manually restart them if the binary has=20 changed. I would also then expect the --restart-services flag to=20 fully restart those services, rather than just reloading them. > Your patch also leads to this inconsistency, because it allows a=20 > service to either be restarted or not, in my opinion :-) Yes, that's true, but then there's no middle-ground. Reloading is=20 "new configuration, old binary", whereas the current options are=20 "old configuration, old binary" or "new configuration, new binary"=20 (or, I guess, "not running because of a failed restart"). > I think both (1) and (2) make sense because both kind of=20 > services (the ones pointing to configuration files in the store=20 > and the ones using /etc/some-file.conf) already exist. Ideally,=20 > the mechanism should be generic enough to handle both cases. (1) actually subsumes (2), so I think I'll implement that. It=20 actually ends up being slightly easier, because the restart=20 strategy can just always be a procedure, with three predefined=20 procedures: always-restart, manually-restart, and never-restart. > That being said, I agree that adding support for 'reload' would=20 > lead to more complexity, and I would understand if you don't add=20 > it :-), but I still think it's a very useful feature. I think you've convinced me that there's value in having more=20 flexibility around restarts. I'll change the restart-strategy=20 value to be a procedure rather than a bare symbol. The downside is=20 that we'll lose the ability to statically analyse how services=20 will behave when restarting, but the increased flexibility is=20 useful to us. > One question though: my understanding is that the default value=20 > for 'restart-strategy' is set in the Guix repository, but a user=20 > would be able to customize it in their config.scm. Can you=20 > confirm it? That is not the current implementation. Individual services can=20 add a field to their configuration objects if it's meaningful for=20 them to customise their restart behaviour, but there isn't a=20 general-purpose mechanism for a user to change the restart=20 behaviour of any service (beyond the ability to write arbitrary=20 Scheme to do it). Carlo From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 30 07:13:01 2018 Received: (at 33508) by debbugs.gnu.org; 30 Nov 2018 12:13:01 +0000 Received: from localhost ([127.0.0.1]:55608 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gShfI-0007pu-Lb for submit@debbugs.gnu.org; Fri, 30 Nov 2018 07:13:00 -0500 Received: from mail.lassieur.org ([83.152.10.219]:55298) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gShfG-0007pj-TD for 33508@debbugs.gnu.org; Fri, 30 Nov 2018 07:12:59 -0500 Received: from rodion (88.191.118.83 [88.191.118.83]) by mail.lassieur.org (OpenSMTPD) with ESMTPSA id e5abbc47 (TLSv1.2:ECDHE-RSA-CHACHA20-POLY1305:256:NO); Fri, 30 Nov 2018 12:12:57 +0000 (UTC) References: <87efb8m5gy.fsf@zancanaro.id.au> <871s78ypr9.fsf@lassieur.org> <875zwj8uqz.fsf@zancanaro.id.au> <87k1kzd02u.fsf@lassieur.org> <87lg5fsdof.fsf@zancanaro.id.au> User-agent: mu4e 1.0; emacs 26.1 From: =?utf-8?Q?Cl=C3=A9ment?= Lassieur To: Carlo Zancanaro Subject: Re: [bug#33508] [PATCH] gnu: Add ability to restart services on system reconfigure In-reply-to: <87lg5fsdof.fsf@zancanaro.id.au> Date: Fri, 30 Nov 2018 13:12:57 +0100 Message-ID: <87bm66higm.fsf@lassieur.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33508 Cc: 33508@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Carlo, Carlo Zancanaro writes: > Hey Cl=C3=A9ment, > > Thanks for your thoughts! I think you're right that the approach I've > implemented isn't flexible enough. I potentially haven't thought through = the > failure cases enough. I was more thinking of reload as providing "zero > downtime" upgrades, rather than providing a safer way to upgrade. > > I'll respond more specifically inline. > > On Tue, Nov 27 2018, Cl=C3=A9ment Lassieur wrote: >> I don't think it needs to detect whether the binary has changed, because >> 'reload' signals are usually implemented so that they can safely fail. = So, >> if the configuration file has changed in an incompatible way, the 'reloa= d' >> action won't work, but the service will keep running. > > We do need to detect whether the binary has changed for the sake of secur= ity > updates, or similar. It would be bad if a user reconfigured their system = and > it didn't upgrade the version of nginx (or its dependencies) that they're > running. If there is a risk for a service to be broken on reconfigure, a user might want to do a safe reconfigure, and later on deal with each critical service one after another, so to avoid having several services down at the same time. I think we should at least allow a user to do a 'safe reconfigure' if they want. > Broadly speaking, I conceptualise reconfigure as "bring my system into th= is > state". Now, thus far we haven't been able to do that, because we have la= cked > the ability to restart services properly, but in my mind the ideal situat= ion > is that after running "guix system reconfigure" our system is completely = put > into the state specified by the config.scm file used. This is ideal, but most services depend on a state (Cuirass, mail servers...). > Although, now that I type that out, I notice that there is one obvious wa= y in > which that is not true: the kernel. We can't hot-swap the kernel, so ther= e can > always be a difference between what the configuration file specifies and = what > the system is actually running. And I don't think you can restart Xorg either... > At any rate, even if we give services the ability to reload without > restarting, they would need to print out a message to prompt the user to > manually restart them if the binary has changed. I would also then expect= the > --restart-services flag to fully restart those services, rather than just > reloading them. Agreed :-) >> Your patch also leads to this inconsistency, because it allows a service= to >> either be restarted or not, in my opinion :-) > > Yes, that's true, but then there's no middle-ground. Reloading is "new > configuration, old binary", whereas the current options are "old > configuration, old binary" or "new configuration, new binary" (or, I gues= s, > "not running because of a failed restart"). > >> I think both (1) and (2) make sense because both kind of services (the o= nes >> pointing to configuration files in the store and the ones using >> /etc/some-file.conf) already exist. Ideally, the mechanism should be >> generic enough to handle both cases. > > (1) actually subsumes (2), so I think I'll implement that. It actually en= ds up > being slightly easier, because the restart strategy can just always be a > procedure, with three predefined procedures: always-restart, manually-res= tart, > and never-restart. > >> That being said, I agree that adding support for 'reload' would lead to = more >> complexity, and I would understand if you don't add it :-), but I still >> think it's a very useful feature. > > I think you've convinced me that there's value in having more flexibility > around restarts. I'll change the restart-strategy value to be a procedure > rather than a bare symbol. The downside is that we'll lose the ability to > statically analyse how services will behave when restarting, but the incr= eased > flexibility is useful to us. It could also detect whether you pass a symbol or a procedure. In most cases that would be a symbol which would allow static analysis. But one could still customize it with a procedure. >> One question though: my understanding is that the default value for >> 'restart-strategy' is set in the Guix repository, but a user would be ab= le >> to customize it in their config.scm. Can you confirm it? > > That is not the current implementation. Individual services can add a fie= ld to > their configuration objects if it's meaningful for them to customise their > restart behaviour, but there isn't a general-purpose mechanism for a user= to > change the restart behaviour of any service (beyond the ability to write > arbitrary Scheme to do it). Ok thank you! Cl=C3=A9ment From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 30 21:32:03 2018 Received: (at 33508) by debbugs.gnu.org; 1 Dec 2018 02:32:03 +0000 Received: from localhost ([127.0.0.1]:56717 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gSv4d-0008Ni-0P for submit@debbugs.gnu.org; Fri, 30 Nov 2018 21:32:03 -0500 Received: from zancanaro.com.au ([45.76.117.151]:49738) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gSv4Z-0008NF-N0 for 33508@debbugs.gnu.org; Fri, 30 Nov 2018 21:32:00 -0500 Received: from jolteon (210-1-202-160-cpe.spintel.net.au [210.1.202.160]) by zancanaro.com.au (Postfix) with ESMTPSA id F2BA7215D2; Sat, 1 Dec 2018 02:31:55 +0000 (UTC) References: <87efb8m5gy.fsf@zancanaro.id.au> <871s78ypr9.fsf@lassieur.org> <875zwj8uqz.fsf@zancanaro.id.au> <87k1kzd02u.fsf@lassieur.org> <87lg5fsdof.fsf@zancanaro.id.au> <87bm66higm.fsf@lassieur.org> User-agent: mu4e 1.0; emacs 26.1 From: Carlo Zancanaro To: =?utf-8?Q?Cl=C3=A9ment?= Lassieur Subject: Re: [bug#33508] [PATCH] gnu: Add ability to restart services on system reconfigure In-reply-to: <87bm66higm.fsf@lassieur.org> Date: Sat, 01 Dec 2018 13:31:48 +1100 Message-ID: <87va4eugdn.fsf@zancanaro.id.au> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33508 Cc: 33508@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Hey Cl=C3=A9ment, I'm still working through my thoughts on how all of this should=20 work. I feel like there are a few different use-cases that change=20 the trade-offs (eg. servers vs desktops, multi-user vs=20 single-user) and I don't know what the best defaults are, or the=20 most useful ways to vary that behaviour. I've attached my most recent version of my patches. I haven't had=20 a chance to test them (so they may have really dumb mistakes), but=20 they should implement the idea of restart-actions as procedures. On Fri, Nov 30 2018, Cl=C3=A9ment Lassieur wrote: > It could also detect whether you pass a symbol or a procedure.=20 > In most cases that would be a symbol which would allow static=20 > analysis. But one could still customize it with a procedure. I don't like this way of having two different representations for=20 the same thing. In my current implementation there are three=20 procedures, {always,manually,never}-restart, which can be used=20 directly in the place of the old symbols (thus we can check for=20 those strategies with eq?). Carlo --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-gnu-Add-ability-to-restart-services-on-system-reconf.patch >From 25d631b33b84f1f48bc06a192c46eb3170e29b97 Mon Sep 17 00:00:00 2001 From: Carlo Zancanaro Date: Mon, 26 Nov 2018 22:38:08 +1100 Subject: [PATCH 1/3] gnu: Add ability to restart services on system reconfigure * gnu/services/herd.scm (restart-service): New procedure. * gnu/services/shepherd.scm ()[restart-strategy]: New field. (always-restart, manually-restart, never-restart): New procedures. * guix/scripts/system.scm (upgrade-shepherd-services): Automatically restart services that should be automatically restarted, and print a message about manually restarting services that should be manually restarted. Temporary commit --- gnu/services/herd.scm | 5 +++++ gnu/services/shepherd.scm | 25 ++++++++++++++++++++++++- guix/scripts/system.scm | 37 +++++++++++++++++++++++++------------ 3 files changed, 54 insertions(+), 13 deletions(-) diff --git a/gnu/services/herd.scm b/gnu/services/herd.scm index 8ff817759..c8d6eb04e 100644 --- a/gnu/services/herd.scm +++ b/gnu/services/herd.scm @@ -52,6 +52,7 @@ load-services load-services/safe start-service + restart-service stop-service)) ;;; Commentary: @@ -256,6 +257,10 @@ when passed a service with an already-registered name." (with-shepherd-action name ('start) result result)) +(define (restart-service name) + (with-shepherd-action name ('restart) result + result)) + (define (stop-service name) (with-shepherd-action name ('stop) result result)) diff --git a/gnu/services/shepherd.scm b/gnu/services/shepherd.scm index 49d08cc30..f7e690fb0 100644 --- a/gnu/services/shepherd.scm +++ b/gnu/services/shepherd.scm @@ -44,12 +44,17 @@ shepherd-service-provision shepherd-service-canonical-name shepherd-service-requirement + shepherd-service-restart-strategy shepherd-service-respawn? shepherd-service-start shepherd-service-stop shepherd-service-auto-start? shepherd-service-modules + always-restart + manually-restart + never-restart + shepherd-action shepherd-action? shepherd-action-name @@ -141,6 +146,22 @@ DEFAULT is given, use it as the service's default value." (guix build utils) (guix build syscalls))) +(define (always-restart service) + "Unconditionally restart SERVICE and return #f." + (let ((name (shepherd-service-canonical-name service))) + (info (G_ "restarting service: ~a~%") name) + (restart-service name) + #f)) + +(define (manually-restart service) + "Do not restart SERVICE, but return #t to indicate that the user should +restart it." + #t) + +(define (never-restart service) + "Do not restart SERVICE and return #f." + #f) + (define-record-type* shepherd-service make-shepherd-service shepherd-service? @@ -159,7 +180,9 @@ DEFAULT is given, use it as the service's default value." (auto-start? shepherd-service-auto-start? ;Boolean (default #t)) (modules shepherd-service-modules ;list of module names - (default %default-modules))) + (default %default-modules)) + (restart-strategy shepherd-service-restart-strategy ;procedure + (default manually-restart))) (define-record-type* shepherd-action make-shepherd-action diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index d92ec7d5a..26e35fe99 100644 --- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -355,16 +355,14 @@ bring the system down." (with-monad %store-monad (munless (null? new-services) - (let ((new-service-names (map shepherd-service-canonical-name new-services)) - (to-restart-names (map shepherd-service-canonical-name to-restart)) - (to-start (filter shepherd-service-auto-start? new-services))) - (info (G_ "loading new services:~{ ~a~}...~%") new-service-names) - (unless (null? to-restart-names) - ;; Listing TO-RESTART-NAMES in the message below wouldn't help - ;; because many essential services cannot be meaningfully - ;; restarted. See . - (format #t (G_ "To complete the upgrade, run 'herd restart SERVICE' to stop, -upgrade, and restart each service that was not automatically restarted.\n"))) + (let* ((new-service-names (map shepherd-service-canonical-name new-services)) + (to-restart-names (map shepherd-service-canonical-name to-restart)) + (to-start-names (map shepherd-service-canonical-name + (filter (lambda (service) + (and (shepherd-service-auto-start? service) + (not (member service to-restart)))) + new-services)))) + (mlet %store-monad ((files (mapm %store-monad (compose lower-object shepherd-service-file) @@ -372,10 +370,25 @@ upgrade, and restart each service that was not automatically restarted.\n"))) ;; Here we assume that FILES are exactly those that were computed ;; as part of the derivation that built OS, which is normally the ;; case. + (info (G_ "loading new services:~{ ~a~}~%") new-service-names) (load-services/safe (map derivation->output-path files)) - (for-each start-service - (map shepherd-service-canonical-name to-start)) + (info (G_ "starting services:~{ ~a~}~%") to-start-names) + (for-each (lambda (service-name) + (info (G_ "starting service: ~a~%") service-name) + (start-service service-name)) + to-start-names) + + (let* ((to-manually-restart (filter (lambda (service) + ((shepherd-service-restart-strategy service) + service)) + to-restart)) + (to-manually-restart-names (map shepherd-service-canonical-name + to-manually-restart))) + (unless (null? to-manually-restart-names) + (info (G_ "To complete the upgrade, the following services need to be manually restarted:~{ ~a~}~%") + to-manually-restart-names))) + (return #t))))))))) (define* (switch-to-system os -- 2.19.2 --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0002-system-Add-restart-services-flag-for-reconfigure.patch >From 270a126c6efd498798bb9342a12c0f671df51b4c Mon Sep 17 00:00:00 2001 From: Carlo Zancanaro Date: Mon, 26 Nov 2018 22:38:18 +1100 Subject: [PATCH 2/3] system: Add --restart-services flag for reconfigure * gnu/services/shepherd.scm (always-restart, manually-restart, never-restart): Add restart-services? argument. * guix/scripts/system.scm (upgrade-shepherd-services): Add parameter to automatically restart services marked as needing manual restart. (switch-to-system, perform-action, process-action): Pass through restart-services? flag. (%options): Add --restart-services flag. (%default-options): Add #f as default value for --restart-services flag. --- gnu/services/shepherd.scm | 14 ++++++++------ guix/scripts/system.scm | 23 +++++++++++++++-------- 2 files changed, 23 insertions(+), 14 deletions(-) diff --git a/gnu/services/shepherd.scm b/gnu/services/shepherd.scm index f7e690fb0..638f6440c 100644 --- a/gnu/services/shepherd.scm +++ b/gnu/services/shepherd.scm @@ -146,19 +146,21 @@ DEFAULT is given, use it as the service's default value." (guix build utils) (guix build syscalls))) -(define (always-restart service) +(define (always-restart service restart-services?) "Unconditionally restart SERVICE and return #f." (let ((name (shepherd-service-canonical-name service))) (info (G_ "restarting service: ~a~%") name) (restart-service name) #f)) -(define (manually-restart service) - "Do not restart SERVICE, but return #t to indicate that the user should -restart it." - #t) +(define (manually-restart service restart-services?) + "Restart SERVICE and return #f if RESTART-SERVICES? is true, otherwise return #t to +indicate that the user should manually restart SERVICE." + (if restart-services? + (always-restart service #t) + #t)) -(define (never-restart service) +(define (never-restart service restart-services?) "Do not restart SERVICE and return #f." #f) diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index 26e35fe99..7c2699065 100644 --- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -332,7 +332,7 @@ unload." (warning (G_ "failed to obtain list of shepherd services~%")) (return #f))))) -(define (upgrade-shepherd-services os) +(define (upgrade-shepherd-services os restart-services?) "Upgrade the Shepherd (PID 1) by unloading obsolete services and loading new services specified in OS and not currently running. @@ -381,7 +381,8 @@ bring the system down." (let* ((to-manually-restart (filter (lambda (service) ((shepherd-service-restart-strategy service) - service)) + service + restart-services?)) to-restart)) (to-manually-restart-names (map shepherd-service-canonical-name to-manually-restart))) @@ -391,7 +392,7 @@ bring the system down." (return #t))))))))) -(define* (switch-to-system os +(define* (switch-to-system os restart-services? #:optional (profile %system-profile)) "Make a new generation of PROFILE pointing to the directory of OS, switch to it atomically, and then run OS's activation script." @@ -419,7 +420,7 @@ it atomically, and then run OS's activation script." (primitive-load (derivation->output-path script)))) ;; Finally, try to update system services. - (upgrade-shepherd-services os)))) + (upgrade-shepherd-services os restart-services?)))) (define-syntax-rule (unless-file-not-found exp) (catch 'system-error @@ -827,7 +828,8 @@ and TARGET arguments." use-substitutes? bootloader-target target image-size file-system-type full-boot? (mappings '()) - (gc-root #f)) + (gc-root #f) + (restart-services? #f)) "Perform ACTION for OS. INSTALL-BOOTLOADER? specifies whether to install bootloader; BOOTLOADER-TAGET is the target for the bootloader; TARGET is the target root directory; IMAGE-SIZE is the size of the image to be built, for @@ -909,7 +911,7 @@ static checks." (case action ((reconfigure) (mbegin %store-monad - (switch-to-system os) + (switch-to-system os restart-services?) (mwhen install-bootloader? (install-bootloader bootloader-script #:bootcfg bootcfg @@ -1092,6 +1094,9 @@ Some ACTIONS support additional ARGS.\n")) (option '(#\r "root") #t #f (lambda (opt name arg result) (alist-cons 'gc-root arg result))) + (option '("restart-services") #f #f + (lambda (opt name arg result) + (alist-cons 'restart-services? #t result))) %standard-build-options)) (define %default-options @@ -1106,7 +1111,8 @@ Some ACTIONS support additional ARGS.\n")) (verbosity . 0) (file-system-type . "ext4") (image-size . guess) - (install-bootloader? . #t))) + (install-bootloader? . #t) + (restart-services? . #f))) ;;; @@ -1179,7 +1185,8 @@ resulting from command-line parsing." #:install-bootloader? bootloader? #:target target #:bootloader-target bootloader-target - #:gc-root (assoc-ref opts 'gc-root))))) + #:gc-root (assoc-ref opts 'gc-root) + #:restart-services? (assoc-ref opts 'restart-services?))))) #:system system)) (warn-about-disk-space))) -- 2.19.2 --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0003-services-Always-restart-guix-daemon.patch >From 2077919dca604c94b09cf105c33987daa8c46304 Mon Sep 17 00:00:00 2001 From: Carlo Zancanaro Date: Mon, 26 Nov 2018 22:38:26 +1100 Subject: [PATCH 3/3] services: Always restart guix daemon * gnu/services/base.scm (guix-shepherd-service): Add restart-strategy of always-restart. --- gnu/services/base.scm | 1 + 1 file changed, 1 insertion(+) diff --git a/gnu/services/base.scm b/gnu/services/base.scm index 228d3c592..37d60720d 100644 --- a/gnu/services/base.scm +++ b/gnu/services/base.scm @@ -1573,6 +1573,7 @@ failed to register hydra.gnu.org public key: ~a~%" status)))))))) (documentation "Run the Guix daemon.") (provision '(guix-daemon)) (requirement '(user-processes)) + (restart-strategy always-restart) (modules '((srfi srfi-1))) (start #~(make-forkexec-constructor -- 2.19.2 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 09 11:59:48 2018 Received: (at 33508) by debbugs.gnu.org; 9 Dec 2018 16:59:48 +0000 Received: from localhost ([127.0.0.1]:40190 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gW2Qm-0004m6-FH for submit@debbugs.gnu.org; Sun, 09 Dec 2018 11:59:48 -0500 Received: from eggs.gnu.org ([208.118.235.92]:48536) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gW2Qk-0004ls-N9 for 33508@debbugs.gnu.org; Sun, 09 Dec 2018 11:59:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gW2Qb-0005zx-Gw for 33508@debbugs.gnu.org; Sun, 09 Dec 2018 11:59:41 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:43506) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gW2Qb-0005zp-D2; Sun, 09 Dec 2018 11:59:37 -0500 Received: from [109.190.253.16] (port=58572 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gW2Qa-0006Sx-Un; Sun, 09 Dec 2018 11:59:37 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Carlo Zancanaro Subject: Re: [bug#33508] [PATCH] gnu: Add ability to restart services on system reconfigure References: <87efb8m5gy.fsf@zancanaro.id.au> Date: Sun, 09 Dec 2018 17:59:29 +0100 In-Reply-To: <87efb8m5gy.fsf@zancanaro.id.au> (Carlo Zancanaro's message of "Mon, 26 Nov 2018 22:41:01 +1100") Message-ID: <87lg4yws9a.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33508 Cc: 33508@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Hi Carlo, Sorry for not commenting earlier! Carlo Zancanaro skribis: > The broad idea is to add a new field to our guix shepherd services: > restart-strategy. There are three valid values: > > - always: this service is always safe to restart when running > reconfigure > > - manual: this service may not be safe to restart when running > reconfigure - a message will be printed telling the user to restart > the service manually, or they can provide the --restart-services flag > to reconfigure to automatically restart them > > - never: this service is never safe to restart when running > reconfigure (eg. udev) > > I have added the flag to the guix daemon's shepherd service to show > how it works. I tested this by changing my substitute servers in > config.scm, and after running "reconfigure" I saw my updated > substitute servers in ps without having to run "sudo herd restart > guix-daemon". In what sense is guix-daemon =E2=80=9Calways safe to restart=E2=80=9D? It= =E2=80=99s actually a difficult question for me. You could argue that its child guix-daemon processes will remain live when we restart it, meaning that client connections remain active and valid. I believe this is indeed the case, though it would be worth double-checking. Now, if safe-to-restart means that we automatically invoke the =E2=80=9Cres= tart=E2=80=9D action on guix-daemon, that means that anything that depends on it (=E2=80=98guix-publish=E2=80=99, =E2=80=98cuirass=E2=80=99, =E2=80=98hpcgui= x-web=E2=80=99, etc.) would be restarted as well (even though I *think* we don=E2=80=99t have to in this case.) But th= ese may not be safe to restart: for example, on may want =E2=80=98guix-publish= =E2=80=99 to run uninterrupted. Furthermore, whether something is =E2=80=9Csafe to restart=E2=80=9D is real= ly user policy. So the notion here should probably not be =E2=80=9Csafe to restart=E2=80=9D= but rather =E2=80=9Clive-upgradable=E2=80=9D. sshd, nginx, and maybe guix-daemon can more or less be live-upgraded, meaning that (1) existing connections are preserved but future connections will talk to the new daemon, and as a corollary, (2) dependent services do not need to be stopped & restarted. Does that make sense? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 13 09:22:46 2018 Received: (at 33508) by debbugs.gnu.org; 13 Dec 2018 14:22:46 +0000 Received: from localhost ([127.0.0.1]:45960 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXRsz-00084T-R2 for submit@debbugs.gnu.org; Thu, 13 Dec 2018 09:22:46 -0500 Received: from mail.lassieur.org ([83.152.10.219]:48468) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gXRsw-00084J-DM for 33508@debbugs.gnu.org; Thu, 13 Dec 2018 09:22:43 -0500 Received: from rodion (i577BC123.versanet.de [87.123.193.35]) by mail.lassieur.org (OpenSMTPD) with ESMTPSA id a5040de0 (TLSv1.2:ECDHE-RSA-CHACHA20-POLY1305:256:NO); Thu, 13 Dec 2018 14:22:40 +0000 (UTC) References: <87efb8m5gy.fsf@zancanaro.id.au> <871s78ypr9.fsf@lassieur.org> <875zwj8uqz.fsf@zancanaro.id.au> <87k1kzd02u.fsf@lassieur.org> <87lg5fsdof.fsf@zancanaro.id.au> <87bm66higm.fsf@lassieur.org> <87va4eugdn.fsf@zancanaro.id.au> User-agent: mu4e 1.0; emacs 26.1 From: =?utf-8?Q?Cl=C3=A9ment?= Lassieur To: Carlo Zancanaro Subject: Re: [bug#33508] [PATCH] gnu: Add ability to restart services on system reconfigure In-reply-to: <87va4eugdn.fsf@zancanaro.id.au> Date: Thu, 13 Dec 2018 15:22:39 +0100 Message-ID: <87h8fhtsk0.fsf@lassieur.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33508 Cc: 33508@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Carlo, Thank you for your modifications. I like your patches better now :-) But I prefer to let Ludovic comment on them, as I have seen an email from him about Guix-daemon restart strategy. Cheers! Cl=C3=A9ment Carlo Zancanaro writes: > Hey Cl=C3=A9ment, > > I'm still working through my thoughts on how all of this should=20 > work. I feel like there are a few different use-cases that change=20 > the trade-offs (eg. servers vs desktops, multi-user vs=20 > single-user) and I don't know what the best defaults are, or the=20 > most useful ways to vary that behaviour. > > I've attached my most recent version of my patches. I haven't had=20 > a chance to test them (so they may have really dumb mistakes), but=20 > they should implement the idea of restart-actions as procedures. > > On Fri, Nov 30 2018, Cl=C3=A9ment Lassieur wrote: >> It could also detect whether you pass a symbol or a procedure.=20 >> In most cases that would be a symbol which would allow static=20 >> analysis. But one could still customize it with a procedure. > > I don't like this way of having two different representations for=20 > the same thing. In my current implementation there are three=20 > procedures, {always,manually,never}-restart, which can be used=20 > directly in the place of the old symbols (thus we can check for=20 > those strategies with eq?). > > Carlo > > From 25d631b33b84f1f48bc06a192c46eb3170e29b97 Mon Sep 17 00:00:00 2001 > From: Carlo Zancanaro > Date: Mon, 26 Nov 2018 22:38:08 +1100 > Subject: [PATCH 1/3] gnu: Add ability to restart services on system > reconfigure > > * gnu/services/herd.scm (restart-service): New procedure. > * gnu/services/shepherd.scm ()[restart-strategy]: New > field. > (always-restart, manually-restart, never-restart): New procedures. > * guix/scripts/system.scm (upgrade-shepherd-services): Automatically rest= art > services that should be automatically restarted, and print a message ab= out > manually restarting services that should be manually restarted. > > Temporary commit > --- > gnu/services/herd.scm | 5 +++++ > gnu/services/shepherd.scm | 25 ++++++++++++++++++++++++- > guix/scripts/system.scm | 37 +++++++++++++++++++++++++------------ > 3 files changed, 54 insertions(+), 13 deletions(-) > > diff --git a/gnu/services/herd.scm b/gnu/services/herd.scm > index 8ff817759..c8d6eb04e 100644 > --- a/gnu/services/herd.scm > +++ b/gnu/services/herd.scm > @@ -52,6 +52,7 @@ > load-services > load-services/safe > start-service > + restart-service > stop-service)) >=20=20 > ;;; Commentary: > @@ -256,6 +257,10 @@ when passed a service with an already-registered nam= e." > (with-shepherd-action name ('start) result > result)) >=20=20 > +(define (restart-service name) > + (with-shepherd-action name ('restart) result > + result)) > + > (define (stop-service name) > (with-shepherd-action name ('stop) result > result)) > diff --git a/gnu/services/shepherd.scm b/gnu/services/shepherd.scm > index 49d08cc30..f7e690fb0 100644 > --- a/gnu/services/shepherd.scm > +++ b/gnu/services/shepherd.scm > @@ -44,12 +44,17 @@ > shepherd-service-provision > shepherd-service-canonical-name > shepherd-service-requirement > + shepherd-service-restart-strategy > shepherd-service-respawn? > shepherd-service-start > shepherd-service-stop > shepherd-service-auto-start? > shepherd-service-modules >=20=20 > + always-restart > + manually-restart > + never-restart > + > shepherd-action > shepherd-action? > shepherd-action-name > @@ -141,6 +146,22 @@ DEFAULT is given, use it as the service's default va= lue." > (guix build utils) > (guix build syscalls))) >=20=20 > +(define (always-restart service) > + "Unconditionally restart SERVICE and return #f." > + (let ((name (shepherd-service-canonical-name service))) > + (info (G_ "restarting service: ~a~%") name) > + (restart-service name) > + #f)) > + > +(define (manually-restart service) > + "Do not restart SERVICE, but return #t to indicate that the user should > +restart it." > + #t) > + > +(define (never-restart service) > + "Do not restart SERVICE and return #f." > + #f) > + > (define-record-type* > shepherd-service make-shepherd-service > shepherd-service? > @@ -159,7 +180,9 @@ DEFAULT is given, use it as the service's default val= ue." > (auto-start? shepherd-service-auto-start? ;Boolean > (default #t)) > (modules shepherd-service-modules ;list of module n= ames > - (default %default-modules))) > + (default %default-modules)) > + (restart-strategy shepherd-service-restart-strategy ;procedure > + (default manually-restart))) >=20=20 > (define-record-type* > shepherd-action make-shepherd-action > diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm > index d92ec7d5a..26e35fe99 100644 > --- a/guix/scripts/system.scm > +++ b/guix/scripts/system.scm > @@ -355,16 +355,14 @@ bring the system down." >=20=20 > (with-monad %store-monad > (munless (null? new-services) > - (let ((new-service-names (map shepherd-service-canonical-na= me new-services)) > - (to-restart-names (map shepherd-service-canonical-na= me to-restart)) > - (to-start (filter shepherd-service-auto-star= t? new-services))) > - (info (G_ "loading new services:~{ ~a~}...~%") new-service= -names) > - (unless (null? to-restart-names) > - ;; Listing TO-RESTART-NAMES in the message below wouldn'= t help > - ;; because many essential services cannot be meaningfully > - ;; restarted. See . > - (format #t (G_ "To complete the upgrade, run 'herd resta= rt SERVICE' to stop, > -upgrade, and restart each service that was not automatically restarted.\= n"))) > + (let* ((new-service-names (map shepherd-service-canonical-na= me new-services)) > + (to-restart-names (map shepherd-service-canonical-na= me to-restart)) > + (to-start-names (map shepherd-service-canonical-na= me > + (filter (lambda (service) > + (and (shepherd-serv= ice-auto-start? service) > + (not (member s= ervice to-restart)))) > + new-services)))) > + > (mlet %store-monad ((files (mapm %store-monad > (compose lower-object > shepherd-service= -file) > @@ -372,10 +370,25 @@ upgrade, and restart each service that was not auto= matically restarted.\n"))) > ;; Here we assume that FILES are exactly those that were= computed > ;; as part of the derivation that built OS, which is nor= mally the > ;; case. > + (info (G_ "loading new services:~{ ~a~}~%") new-service-= names) > (load-services/safe (map derivation->output-path files)) >=20=20 > - (for-each start-service > - (map shepherd-service-canonical-name to-start)) > + (info (G_ "starting services:~{ ~a~}~%") to-start-names) > + (for-each (lambda (service-name) > + (info (G_ "starting service: ~a~%") service-= name) > + (start-service service-name)) > + to-start-names) > + > + (let* ((to-manually-restart (filter (lambda (service) > + ((shepherd-service= -restart-strategy service) > + service)) > + to-restart)) > + (to-manually-restart-names (map shepherd-service-= canonical-name > + to-manually-resta= rt))) > + (unless (null? to-manually-restart-names) > + (info (G_ "To complete the upgrade, the following se= rvices need to be manually restarted:~{ ~a~}~%") > + to-manually-restart-names))) > + > (return #t))))))))) >=20=20 > (define* (switch-to-system os > --=20 > 2.19.2 > > From 270a126c6efd498798bb9342a12c0f671df51b4c Mon Sep 17 00:00:00 2001 > From: Carlo Zancanaro > Date: Mon, 26 Nov 2018 22:38:18 +1100 > Subject: [PATCH 2/3] system: Add --restart-services flag for reconfigure > > * gnu/services/shepherd.scm (always-restart, manually-restart, never-rest= art): > Add restart-services? argument. > * guix/scripts/system.scm (upgrade-shepherd-services): Add parameter to > automatically restart services marked as needing manual restart. > (switch-to-system, perform-action, process-action): Pass through > restart-services? flag. > (%options): Add --restart-services flag. > (%default-options): Add #f as default value for --restart-services flag. > --- > gnu/services/shepherd.scm | 14 ++++++++------ > guix/scripts/system.scm | 23 +++++++++++++++-------- > 2 files changed, 23 insertions(+), 14 deletions(-) > > diff --git a/gnu/services/shepherd.scm b/gnu/services/shepherd.scm > index f7e690fb0..638f6440c 100644 > --- a/gnu/services/shepherd.scm > +++ b/gnu/services/shepherd.scm > @@ -146,19 +146,21 @@ DEFAULT is given, use it as the service's default v= alue." > (guix build utils) > (guix build syscalls))) >=20=20 > -(define (always-restart service) > +(define (always-restart service restart-services?) > "Unconditionally restart SERVICE and return #f." > (let ((name (shepherd-service-canonical-name service))) > (info (G_ "restarting service: ~a~%") name) > (restart-service name) > #f)) >=20=20 > -(define (manually-restart service) > - "Do not restart SERVICE, but return #t to indicate that the user should > -restart it." > - #t) > +(define (manually-restart service restart-services?) > + "Restart SERVICE and return #f if RESTART-SERVICES? is true, otherwise= return #t to > +indicate that the user should manually restart SERVICE." > + (if restart-services? > + (always-restart service #t) > + #t)) >=20=20 > -(define (never-restart service) > +(define (never-restart service restart-services?) > "Do not restart SERVICE and return #f." > #f) >=20=20 > diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm > index 26e35fe99..7c2699065 100644 > --- a/guix/scripts/system.scm > +++ b/guix/scripts/system.scm > @@ -332,7 +332,7 @@ unload." > (warning (G_ "failed to obtain list of shepherd services~%")) > (return #f))))) >=20=20 > -(define (upgrade-shepherd-services os) > +(define (upgrade-shepherd-services os restart-services?) > "Upgrade the Shepherd (PID 1) by unloading obsolete services and loadi= ng new > services specified in OS and not currently running. >=20=20 > @@ -381,7 +381,8 @@ bring the system down." >=20=20 > (let* ((to-manually-restart (filter (lambda (service) > ((shepherd-service= -restart-strategy service) > - service)) > + service > + restart-services?= )) > to-restart)) > (to-manually-restart-names (map shepherd-service-= canonical-name > to-manually-resta= rt))) > @@ -391,7 +392,7 @@ bring the system down." >=20=20 > (return #t))))))))) >=20=20 > -(define* (switch-to-system os > +(define* (switch-to-system os restart-services? > #:optional (profile %system-profile)) > "Make a new generation of PROFILE pointing to the directory of OS, swi= tch to > it atomically, and then run OS's activation script." > @@ -419,7 +420,7 @@ it atomically, and then run OS's activation script." > (primitive-load (derivation->output-path script)))) >=20=20 > ;; Finally, try to update system services. > - (upgrade-shepherd-services os)))) > + (upgrade-shepherd-services os restart-services?)))) >=20=20 > (define-syntax-rule (unless-file-not-found exp) > (catch 'system-error > @@ -827,7 +828,8 @@ and TARGET arguments." > use-substitutes? bootloader-target target > image-size file-system-type full-boot? > (mappings '()) > - (gc-root #f)) > + (gc-root #f) > + (restart-services? #f)) > "Perform ACTION for OS. INSTALL-BOOTLOADER? specifies whether to inst= all > bootloader; BOOTLOADER-TAGET is the target for the bootloader; TARGET is= the > target root directory; IMAGE-SIZE is the size of the image to be built, = for > @@ -909,7 +911,7 @@ static checks." > (case action > ((reconfigure) > (mbegin %store-monad > - (switch-to-system os) > + (switch-to-system os restart-services?) > (mwhen install-bootloader? > (install-bootloader bootloader-script > #:bootcfg bootcfg > @@ -1092,6 +1094,9 @@ Some ACTIONS support additional ARGS.\n")) > (option '(#\r "root") #t #f > (lambda (opt name arg result) > (alist-cons 'gc-root arg result))) > + (option '("restart-services") #f #f > + (lambda (opt name arg result) > + (alist-cons 'restart-services? #t result))) > %standard-build-options)) >=20=20 > (define %default-options > @@ -1106,7 +1111,8 @@ Some ACTIONS support additional ARGS.\n")) > (verbosity . 0) > (file-system-type . "ext4") > (image-size . guess) > - (install-bootloader? . #t))) > + (install-bootloader? . #t) > + (restart-services? . #f))) >=20=20 > > ;;; > @@ -1179,7 +1185,8 @@ resulting from command-line parsing." > #:install-bootloader? bootloader? > #:target target > #:bootloader-target bootloader-target > - #:gc-root (assoc-ref opts 'gc-root))))) > + #:gc-root (assoc-ref opts 'gc-root) > + #:restart-services? (assoc-ref opts 'restar= t-services?))))) > #:system system)) > (warn-about-disk-space))) >=20=20 > --=20 > 2.19.2 > > From 2077919dca604c94b09cf105c33987daa8c46304 Mon Sep 17 00:00:00 2001 > From: Carlo Zancanaro > Date: Mon, 26 Nov 2018 22:38:26 +1100 > Subject: [PATCH 3/3] services: Always restart guix daemon > > * gnu/services/base.scm (guix-shepherd-service): Add restart-strategy of > always-restart. > --- > gnu/services/base.scm | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/gnu/services/base.scm b/gnu/services/base.scm > index 228d3c592..37d60720d 100644 > --- a/gnu/services/base.scm > +++ b/gnu/services/base.scm > @@ -1573,6 +1573,7 @@ failed to register hydra.gnu.org public key: ~a~%" = status)))))))) > (documentation "Run the Guix daemon.") > (provision '(guix-daemon)) > (requirement '(user-processes)) > + (restart-strategy always-restart) > (modules '((srfi srfi-1))) > (start > #~(make-forkexec-constructor From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 01 06:26:06 2019 Received: (at 33508) by debbugs.gnu.org; 1 Jan 2019 11:26:06 +0000 Received: from localhost ([127.0.0.1]:43788 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1geIBS-0006ch-7m for submit@debbugs.gnu.org; Tue, 01 Jan 2019 06:26:06 -0500 Received: from zancanaro.com.au ([45.76.117.151]:38996) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1geIBO-0006cE-Rr for 33508@debbugs.gnu.org; Tue, 01 Jan 2019 06:26:03 -0500 Received: from jolteon (210-1-202-160-cpe.spintel.net.au [210.1.202.160]) by zancanaro.com.au (Postfix) with ESMTPSA id 66A17259E6; Tue, 1 Jan 2019 11:25:59 +0000 (UTC) References: <87efb8m5gy.fsf@zancanaro.id.au> <87lg4yws9a.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Carlo Zancanaro To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#33508] [PATCH] gnu: Add ability to restart services on system reconfigure In-reply-to: <87lg4yws9a.fsf@gnu.org> Date: Tue, 01 Jan 2019 22:25:30 +1100 Message-ID: <87h8eszkk5.fsf@zancanaro.id.au> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33508 Cc: 33508@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hey Ludo=E2=80=99, Sorry for not responding to this email for so long. I've been=20 trying to think through some of the issues around this, and I'm=20 not confident that I have thought through the issues well enough=20 to actually decide on a good course of action, beyond what I have=20 already written. I'll respond to a few specific things in your=20 message, but I don't even know what a good solution would look=20 like, let alone how to build it. On Mon, Dec 10 2018, Ludovic Court=C3=A8s wrote: > In what sense is guix-daemon =E2=80=9Calways safe to restart=E2=80=9D? I= t=E2=80=99s=20 > actually a difficult question for me. I agree it's tricky. I had mostly intended that as an example,=20 because I used guix-daemon for my testing, but ... > You could argue that its child guix-daemon processes will remain=20 > live when we restart it, meaning that client connections remain=20 > active and valid. I believe this is indeed the case, though it=20 > would be worth double-checking. ... this is what I was thinking. I'm fairly sure this is the case,=20 given my observations while I was testing these patches. > Now, if safe-to-restart means that we automatically invoke the=20 > =E2=80=9Crestart=E2=80=9D action on guix-daemon, that means that anything= that=20 > depends on it (=E2=80=98guix-publish=E2=80=99, =E2=80=98cuirass=E2=80=99,= =E2=80=98hpcguix-web=E2=80=99, etc.)=20 > would be restarted as well (even though I *think* we don=E2=80=99t have=20 > to in this case.) But these may not be safe to restart: for=20 > example, on may want =E2=80=98guix-publish=E2=80=99 to run uninterrupted. At the moment we have no way to capture this, particularly in the=20 Shepherd. There's no way to restart a service without restarting=20 dependent services, but I particularly want to pick up on the=20 "uninterrupted" by talking about nginx below. > ... > sshd, nginx, and maybe guix-daemon can more or less be=20 > live-upgraded, meaning that (1) existing connections are=20 > preserved but future connections will talk to the new daemon,=20 > and as a corollary, (2) dependent services do not need to be=20 > stopped & restarted. I did some research into nginx, and it turns out that it is=20 possible to upgrade nginx with zero-downtime by running two=20 daemons simultaneously listening on the same port(s), then=20 shutting down the old daemon after the new one has successfully=20 started[1]. This allows for an "uninterrupted" upgrade, but I'm=20 not confident that I would be able to implement it within our=20 current framework. In all, I haven't done anything with this in the last month. I've=20 thought about it a few times, but it just feels a bit=20 overwhelming. Carlo [1]: https://nginx.org/en/docs/control.html#upgrade From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 05 09:00:13 2019 Received: (at 33508) by debbugs.gnu.org; 5 Jan 2019 14:00:13 +0000 Received: from localhost ([127.0.0.1]:47361 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gfmUm-0004os-O2 for submit@debbugs.gnu.org; Sat, 05 Jan 2019 09:00:12 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:60116) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gfmUk-0004ln-F0 for 33508@debbugs.gnu.org; Sat, 05 Jan 2019 09:00:10 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 68BEDC1C; Sat, 5 Jan 2019 15:00:09 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Btk7OarHaOrR; Sat, 5 Jan 2019 15:00:08 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 66B6E45A; Sat, 5 Jan 2019 15:00:08 +0100 (CET) From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Carlo Zancanaro Subject: Re: [bug#33508] [PATCH] gnu: Add ability to restart services on system reconfigure References: <87efb8m5gy.fsf@zancanaro.id.au> <87lg4yws9a.fsf@gnu.org> <87h8eszkk5.fsf@zancanaro.id.au> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 16 =?utf-8?Q?Niv=C3=B4se?= an 227 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 05 Jan 2019 15:00:07 +0100 In-Reply-To: <87h8eszkk5.fsf@zancanaro.id.au> (Carlo Zancanaro's message of "Tue, 01 Jan 2019 22:25:30 +1100") Message-ID: <87tvin5hnc.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 33508 Cc: 33508@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi Carlo, Carlo Zancanaro skribis: > Sorry for not responding to this email for so long. I've been trying > to think through some of the issues around this, and I'm not confident > that I have thought through the issues well enough to actually decide > on a good course of action, beyond what I have already written. I'll > respond to a few specific things in your message, but I don't even > know what a good solution would look like, let alone how to build it. Sure, we can take more time to think through it. You earlier work in this area has already greatly improved the situation so I feel less pressure now. > I did some research into nginx, and it turns out that it is possible > to upgrade nginx with zero-downtime by running two daemons > simultaneously listening on the same port(s), then shutting down the > old daemon after the new one has successfully started[1]. This allows > for an "uninterrupted" upgrade, but I'm not confident that I would be > able to implement it within our current framework. Nginx does all this for us. Basically if you run =E2=80=9Cnginx -s restart= =E2=80=9D, IIRC, it automatically does the multi-process dance and you eventually end up with only upgraded nginx processes. However it relies on being able to read its new configuration file from the same location as before, which is something that doesn=E2=80=99t quite work in our setting, unless we make the file available at a fixed location like /etc/nginx.conf. Cl=C3=A9ment looked into this a while back but I cannot find the reference. Anyway I think we should probably special-case the =E2=80=98restart=E2=80= =99 action for those live-upgradable services. That doesn=E2=80=99t require any change in= the Shepherd. Thoughts? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 09 11:26:56 2023 Received: (at control) by debbugs.gnu.org; 9 Jun 2023 15:26:56 +0000 Received: from localhost ([127.0.0.1]:60133 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q7e0u-0005Q2-L8 for submit@debbugs.gnu.org; Fri, 09 Jun 2023 11:26:56 -0400 Received: from eggs.gnu.org ([209.51.188.92]:57984) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q7e0r-0005Pk-Rn for control@debbugs.gnu.org; Fri, 09 Jun 2023 11:26:54 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q7e0m-0007cZ-G2 for control@debbugs.gnu.org; Fri, 09 Jun 2023 11:26:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to: references; bh=9cxGdv+WLMVOCMzOgiBXJOyBXwQAcSKqJNUdV/bBhpw=; b=d8t43vdzer3Am4 YjIP28K/RPxRFcwB0UQj4vngS5qqjINhZJ7rO1uJdSCv6liShr2Zxomb4wnpx28dU5M0orrQEQybv /LZSeHx8J5pE1rqivOx/7LqGZGoG8kSKI7sK+SKVSpFdy9N/MRMIz/UnyxmbGzSmV+JRO/BDhSwRq hvrTfOi9ztq2sX52KtCLnBizjlLQxdAvUTz/AkAoy5QLuTEvHksf3ctxz/O92COwUBRp5jnoCGvpm pqp151CqkK3qJKNGyZMb86wXYSfQ2YzdCPDDcSc87RDMHXJSrynmMNfpyRMpnyNMAhyWMFUnuAM17 5RL6QoAWanfp+uXLv8Vg==; Received: from [193.50.110.239] (helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q7e0m-0005oR-3J for control@debbugs.gnu.org; Fri, 09 Jun 2023 11:26:48 -0400 Date: Fri, 09 Jun 2023 17:26:46 +0200 Message-Id: <871qikwtah.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #33508 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) close 33508 quit From unknown Sat Jul 26 21:31:03 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 08 Jul 2023 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator