From unknown Tue Jun 17 22:18:11 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#32408] [PATCH shepherd] Allow replacement of services Resent-From: Carlo Zancanaro Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 09 Aug 2018 12:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 32408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 32408@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.153381855916952 (code B ref -1); Thu, 09 Aug 2018 12:43:02 +0000 Received: (at submit) by debbugs.gnu.org; 9 Aug 2018 12:42:39 +0000 Received: from localhost ([127.0.0.1]:45595 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fnkGu-0004PC-Ok for submit@debbugs.gnu.org; Thu, 09 Aug 2018 08:42:39 -0400 Received: from eggs.gnu.org ([208.118.235.92]:51532) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fnkGo-0004Ot-NX for submit@debbugs.gnu.org; Thu, 09 Aug 2018 08:42:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fnkGh-0006Vq-QK for submit@debbugs.gnu.org; Thu, 09 Aug 2018 08:42:21 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:60648) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fnkGh-0006V9-MH for submit@debbugs.gnu.org; Thu, 09 Aug 2018 08:42:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42502) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fnkGf-00046J-SA for guix-patches@gnu.org; Thu, 09 Aug 2018 08:42:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fnkGc-0006Ml-J5 for guix-patches@gnu.org; Thu, 09 Aug 2018 08:42:17 -0400 Received: from mail-pl0-x232.google.com ([2607:f8b0:400e:c01::232]:46389) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fnkGc-0006Kp-6e for guix-patches@gnu.org; Thu, 09 Aug 2018 08:42:14 -0400 Received: by mail-pl0-x232.google.com with SMTP id t17-v6so2505044ply.13 for ; Thu, 09 Aug 2018 05:42:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:user-agent:from:to:subject:date:message-id:mime-version; bh=ZsCJghigN7gZkaxu2jdzB7Ff38LVD9j2sZMThC+KPYw=; b=SPRyNNN+6E29IPXhtPTGJR98pgqkmW9Ed8u3hNHoifmNTQpQ1GThGuzyR/EazbptWJ IZvHYpuaYFYf0edtQe+5K3QYZAIxRkFW7bEM64CZ61x5BZ2DawKvZuM/OfLBlL/u+bB8 QU3kWatM46Vj1AkvY95xs+BSHHU1SIZMZ9tocVHpEDtIBwHByPHqVFvh7xD/ydktBK17 IN9iZoFa7Y/ZNmjm0Q3Ocx+geTNrAQ5nWNDQjNAL3l0r+y633hDAFciWuZgUTPcHERkv fF7AEgx3M9pgC+uwbaMA+j24GSRXdT+S9eL59QQBhaZa9dgOh711J5UfUm8zaA6UadhF 8Ueg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:user-agent:from:to:subject:date :message-id:mime-version; bh=ZsCJghigN7gZkaxu2jdzB7Ff38LVD9j2sZMThC+KPYw=; b=qZLXNxnRejzLtdhRYp8HRmSrZGbESvZKrerrgsk4yPu0PhHjmbbgMJMyhKbbPO2YpR skvMPJyAUw2LVU+vfFFOOp9Qu3EZpTEIR0D5C4YV4spk58g6wjwUtbIDoalgAGDp1UZY 7WaPXxGWLMSDwfSp35tw0xDwWL73Y9hWw1X3Q5VtDjoQcOp1E/NTsLELU/o0XkPzFQKK 1eq0CnKVpKUQJDU69uNkjBfIqwYIIEjVvnzvX00PqdPP+7Bgxm6NbSHF9SKdX1RNI2LW 7DhAxlUF5RuucU3oJ6zWtOQbjtjgeyey8afITSJCeuSbjGBrqVvZS/N0/J6t0KGFFSYC UJDg== X-Gm-Message-State: AOUpUlEhz2+zpgnIcYWJw+uYwbvrCerbxFbFk9BuMkHGm9JU5EPHKoWI Ijc9cv4sC2H/m/3TSylg/7ReHJqd X-Google-Smtp-Source: AA+uWPwQ4ZOieWaKiTB3Q/whU7xblJkEpgfqpAk3GPkvc/S61C4xCKwaAMwG6nKs71y0PHXzo6ni7w== X-Received: by 2002:a17:902:50ec:: with SMTP id c41-v6mr1952314plj.269.1533818532666; Thu, 09 Aug 2018 05:42:12 -0700 (PDT) Received: from pidgey (210-1-202-160-cpe.spintel.net.au. [210.1.202.160]) by smtp.gmail.com with ESMTPSA id f6-v6sm14899190pgf.52.2018.08.09.05.42.10 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 09 Aug 2018 05:42:11 -0700 (PDT) User-agent: mu4e 1.0; emacs 26.1 From: Carlo Zancanaro Date: Thu, 09 Aug 2018 22:42:02 +1000 Message-ID: <87wosz4spx.fsf@zancanaro.id.au> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; format=flowed This is a relatively simple patch adding a replacement slot to services in the Shepherd. When stopping a service, the replacement slot is checked and, if it has a value, is used to upgrade the current service. I've chosen to modify the existing service, rather than creating a new one, but that was mostly because it was easier for me to implement quickly, and I didn't have a huge amount of time. I'm hopeful that this, or something like it, can be used by GuixSD to allow people to restart services after reconfiguring without rebooting (or remembering to stop, reconfigure, start). Carlo --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0001-service-Add-a-replacement-slot-for-delayed-service-r.patch Content-Transfer-Encoding: quoted-printable From=20e03290041c91813f1a301c7e9c4dbb9ee768b400 Mon Sep 17 00:00:00 2001 From: Carlo Zancanaro Date: Thu, 9 Aug 2018 22:30:38 +1000 Subject: [PATCH] service: Add a replacement slot for delayed service replacement. * modules/shepherd/service.scm (): Add replacement slot (replace-service): New procedure. (stop): Call replace-service after stopping a service. * tests/replacement.sh: Add a test for it. * Makefile.am (TESTS): Add the new test. * doc/shepherd.texi (Slots of services): Document it. =2D-- Makefile.am | 1 + doc/shepherd.texi | 9 +++ modules/shepherd/service.scm | 23 +++++++- tests/replacement.sh | 106 +++++++++++++++++++++++++++++++++++ 4 files changed, 138 insertions(+), 1 deletion(-) create mode 100644 tests/replacement.sh diff --git a/Makefile.am b/Makefile.am index 8dad006..4322d7f 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -184,6 +184,7 @@ SUFFIXES =3D .go =20 TESTS =3D \ tests/basic.sh \ + tests/replacement.sh \ tests/respawn.sh \ tests/respawn-throttling.sh \ tests/misbehaved-client.sh \ diff --git a/doc/shepherd.texi b/doc/shepherd.texi index 7946f8b..1de6d80 100644 =2D-- a/doc/shepherd.texi +++ b/doc/shepherd.texi @@ -708,6 +708,15 @@ handler will not start it again. =20 otherwise @code{#f}. =20 +@item +@vindex replacement (slot of ) +@code{replacement} specifies a service to be used to replace this one +when it is stopped. This service will continue to function normally +until the @code{stop} action is invoked. After the service has been +successfully stopped, its definition will be replaced by the value of +this slot, which must itself be a service. This slot is ignored if +its value is @code{#f}. + @end itemize =20 @c @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ diff --git a/modules/shepherd/service.scm b/modules/shepherd/service.scm index 5653388..4f62dc1 100644 =2D-- a/modules/shepherd/service.scm +++ b/modules/shepherd/service.scm @@ -205,7 +205,10 @@ respawned, shows that it has been respawned more than = TIMES in SECONDS." (stop-delay? #:init-keyword #:stop-delay? #:init-value #f) ;; The times of the last respawns, most recent first. =2D (last-respawns #:init-form '())) + (last-respawns #:init-form '()) + ;; A replacement for when this service is stopped. + (replacement #:init-keyword #:replacement + #:init-value #f)) =20 (define (service? obj) "Return true if OBJ is a service." @@ -341,6 +344,21 @@ wire." (canonical-name obj))))) (slot-ref obj 'running)) =20 +(define (replace-service service) + (let ((replacement (slot-ref service 'replacement))) + (define (copy-slot! slot) + (slot-set! service slot (slot-ref replacement slot))) + (when replacement + (copy-slot! 'provides) + (copy-slot! 'requires) + (copy-slot! 'respawn?) + (copy-slot! 'start) + (copy-slot! 'stop) + (copy-slot! 'actions) + (copy-slot! 'running) + (copy-slot! 'docstring)) + service)) + ;; Stop the service, including services that depend on it. If the ;; latter fails, continue anyway. Return `#f' if it could be stopped. (define-method (stop (obj ) . args) @@ -385,6 +403,9 @@ wire." ;; Reset the list of respawns. (slot-set! obj 'last-respawns '()) =20 + ;; Replace the service with its replacement, if it has one + (replace-service obj) + ;; Status message. (let ((name (canonical-name obj))) (if (running? obj) diff --git a/tests/replacement.sh b/tests/replacement.sh new file mode 100644 index 0000000..585ab5a =2D-- /dev/null +++ b/tests/replacement.sh @@ -0,0 +1,106 @@ +# GNU Shepherd --- Ensure replacing services works properly +# Copyright =C2=A9 2014, 2016 Ludovic Court=C3=A8s +# Copyright =C2=A9 2018 Carlo Zancanaro +# +# This file is part of the GNU Shepherd. +# +# The GNU Shepherd is free software; you can redistribute it and/or modify= it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or (at +# your option) any later version. +# +# The GNU Shepherd is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with the GNU Shepherd. If not, see . + +shepherd --version +herd --version + +socket=3D"t-socket-$$" +conf=3D"t-conf-$$" +rconf=3D"t-rconf-$$" +log=3D"t-log-$$" +stamp=3D"t-stamp-$$" +pid=3D"t-pid-$$" + +herd=3D"herd -s $socket" + +trap "rm -f $socket $conf $rconf $stamp $log; + test -f $pid && kill \`cat $pid\` || true; rm -f $pid" EXIT + +cat > "$conf"< + #:provides '(test) + #:start (const #t) + #:actions (make-actions + (say-hello (lambda _ + (call-with-output-file "$stamp" + (lambda (port) + (display "Hello" port)))))) + #:respawn? #f)) +EOF + +rm -f "$pid" "$stamp" "$socket" +shepherd -I -s "$socket" -c "$conf" --pid=3D"$pid" --log=3D"$log" & + +while ! test -f "$pid"; do sleep 0.5 ; done + +$herd start test + +if ! $herd say-hello test; then + echo "say-hello failed" + exit 1 +fi + +cat - > "$rconf"< + #:provides '(test) + #:start (const #t) + #:actions (make-actions + (say-goodbye (lambda _ + (call-with-output-file "$stamp" + (lambda (port) + (display "Goodbye" port)))))) + #:respawn? #f))) +EOF + +$herd load root "$rconf" + +if ! $herd say-hello test; then + echo "say-hello failed after setting replacement" + exit 1 +fi + +if test `cat $stamp` !=3D "Hello"; then + echo "Output file had the wrong contents! Was:" + cat $stamp + exit 1 +fi + +$herd stop test + +$herd start test + +if $herd say-hello test; then + echo "say-hello should have failed after stop/start" + exit 1 +fi + +if ! $herd say-goodbye test; then + echo "say-goodbye should have failed" + exit 1 +fi + +if test `cat $stamp` !=3D "Goodbye"; then + echo "Output file had the wrong contents! Was:" + cat $stamp + exit 1 +fi =2D-=20 2.18.0 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE1lpncq7JnOkt+LaeqdyPv9awIbwFAltsNpoACgkQqdyPv9aw IbzJ3A//TD+Z+9GuR+bSoRxFTS5wrMfQ0hHANj22sEPQQk8xaYSVKuPMHkPe4+RB z+ECmxYKtQRUB5b4pJ/cg8G8ynvOW9ZAbKL1pGpIMfuyyHmCv8XdbugzSrphSLFD Pq34HfTPWKPN9JuSeXZ445h5vVJ4i2XP0bwHU8l3by5cvOJesupBge1lvGYwnBhc HIvfB0r+Cm7g/3m5jHF8k/mImHZpYANzKRHL9pRSFJz4ewk/+8i5wpIVrpm5KPFX yX2c3wvxa4BRsyKPGVhJ0v34ltLT1BGeUHQhHDju3Iilkt3r/b3+DmWMiSVZ+D8v Gn3CjPH1bhmTx6Ls3Kj7kZMxyhdam5S3Zl+TERQsYS0Kp5jZNvqY8fbk6LEcS9tX o6B5WIo4oKnqA7PmOwMg5GzX+9BkLVY6CGnMkIAhdKWANjSYMBizZ94DA3ZmrImA 6/nrHsNCzcjhg2wyhS7cOtMeis+xWMP7CMO0IASjikpq6e3YWJU3vSqGJKZwInyZ xZGBp45+v8wiiLDcc9q1+k3rSyLbAPc4UE4h8CQ0FBeA93Z12pCiZ2W2l+BkK9L+ S1kaZ3iLBu1qFPyzKT6nRvREawdgbwD3BeZoo2jqX2PF4HlstAH9noM/n2GfiL0v m0QKotRUXcf97aCBhQHs2yRVMt2oJz936Q44zJ+yQqBeu7CyCb0= =2Nj0 -----END PGP SIGNATURE----- --==-=-=-- From unknown Tue Jun 17 22:18:11 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#32408] [PATCH shepherd] Allow replacement of services Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 20 Aug 2018 20:34:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 32408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Carlo Zancanaro Cc: 32408@debbugs.gnu.org Received: via spool by 32408-submit@debbugs.gnu.org id=B32408.153479723711005 (code B ref 32408); Mon, 20 Aug 2018 20:34:01 +0000 Received: (at 32408) by debbugs.gnu.org; 20 Aug 2018 20:33:57 +0000 Received: from localhost ([127.0.0.1]:55932 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1frqs8-0002rR-RH for submit@debbugs.gnu.org; Mon, 20 Aug 2018 16:33:57 -0400 Received: from eggs.gnu.org ([208.118.235.92]:34460) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1frqs6-0002rB-Q6 for 32408@debbugs.gnu.org; Mon, 20 Aug 2018 16:33:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1frqrz-0003RO-K1 for 32408@debbugs.gnu.org; Mon, 20 Aug 2018 16:33:49 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42997) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1frqrx-0003M6-LW; Mon, 20 Aug 2018 16:33:46 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51678 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1frqrv-0002Hx-Mt; Mon, 20 Aug 2018 16:33:45 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <87wosz4spx.fsf@zancanaro.id.au> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 3 Fructidor an 226 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 20 Aug 2018 22:33:42 +0200 In-Reply-To: <87wosz4spx.fsf@zancanaro.id.au> (Carlo Zancanaro's message of "Thu, 09 Aug 2018 22:42:02 +1000") Message-ID: <87h8jobwwp.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Hi Carlo, Carlo Zancanaro skribis: > This is a relatively simple patch adding a replacement slot to > services in the Shepherd. When stopping a service, the replacement > slot is checked and, if it has a value, is used to upgrade the current > service. > > I've chosen to modify the existing service, rather than creating a new > one, but that was mostly because it was easier for me to implement > quickly, and I didn't have a huge amount of time. > > I'm hopeful that this, or something like it, can be used by GuixSD to > allow people to restart services after reconfiguring without rebooting > (or remembering to stop, reconfigure, start). Awesome! This is exactly what we need to address the problem. We=E2=80=99ll still want to be able to special-case things like nginx that = can be =E2=80=9Chot-replaced=E2=80=9D, though. So perhaps, in addition to this= patch on the Shepherd side, we=E2=80=99ll need extra stuff in (gnu services shepherd). For instance, the =E2=80=98actions=E2=80=99 field of cou= ld, by default, include an =E2=80=9Cupgrade=E2=80=9D action that simply sets the = =E2=80=98replacement=E2=80=99 slot. For nginx, we=E2=80=99d provide a custom =E2=80=9Cupgrade=E2=80=9D a= ction that does =E2=80=9Cnginx -s restart=E2=80=9D or whatever it is that needs to be done. =E2=80=98guix system reconfigure=E2=80=99 would automatically invoke the = =E2=80=98upgrade=E2=80=99 action for each new service. WDYT? > From e03290041c91813f1a301c7e9c4dbb9ee768b400 Mon Sep 17 00:00:00 2001 > From: Carlo Zancanaro > Date: Thu, 9 Aug 2018 22:30:38 +1000 > Subject: [PATCH] service: Add a replacement slot for delayed service > replacement. > > * modules/shepherd/service.scm (): Add replacement slot > (replace-service): New procedure. > (stop): Call replace-service after stopping a service. > * tests/replacement.sh: Add a test for it. > * Makefile.am (TESTS): Add the new test. > * doc/shepherd.texi (Slots of services): Document it. Overall LGTM. Some comments: > +(define (replace-service service) Please add a docstring. > + (let ((replacement (slot-ref service 'replacement))) > + (define (copy-slot! slot) > + (slot-set! service slot (slot-ref replacement slot))) > + (when replacement > + (copy-slot! 'provides) > + (copy-slot! 'requires) > + (copy-slot! 'respawn?) > + (copy-slot! 'start) > + (copy-slot! 'stop) > + (copy-slot! 'actions) > + (copy-slot! 'running) > + (copy-slot! 'docstring)) > + service)) Having a hardcoded list of slots sounds error-prone=E2=80=94surely we=E2=80= =99ll forget to update it down the road. I wonder what else could be done. One option would be to grab the block asyncs and atomically replace the service in the =E2=80=98%services=E2=80=99 hash table. Then we only need t= o copy the =E2=80=98last-respawns=E2=80=99 slot to the new service, I believe. (This = changes the object identity of the service but I think its OK.) Another option would be to use GOOPS tricks to iterate over the list of slots and have a list of slots *not* to copy. I=E2=80=99m not a big fan of= this option, though. > +cat - > "$rconf"< +(let ((service (lookup-running 'test))) > + (slot-set! service 'replacement > + (make I wonder if we should let users fiddle with =E2=80=98replacement=E2=80=99 d= irectly, or if we should provide a higher-level construct. For instance, =E2=80=98register-services=E2=80=99 could transparently set t= he =E2=80=98replacement=E2=80=99 field for services already registered instead= of doing: (assert (null? (lookup-services (canonical-name new)))) Not sure if there are cases where this behavior would be undesirable, though. Thoughts? > +if test `cat $stamp` !=3D "Goodbye"; then Nitpick: "`cat $stamp`". Thanks! Ludo=E2=80=99. From unknown Tue Jun 17 22:18:11 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#32408] [PATCH shepherd] Allow replacement of services Resent-From: Carlo Zancanaro Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 20 Aug 2018 21:18:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 32408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 32408@debbugs.gnu.org Received: via spool by 32408-submit@debbugs.gnu.org id=B32408.153479982915088 (code B ref 32408); Mon, 20 Aug 2018 21:18:02 +0000 Received: (at 32408) by debbugs.gnu.org; 20 Aug 2018 21:17:09 +0000 Received: from localhost ([127.0.0.1]:55970 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1frrXw-0003vI-NZ for submit@debbugs.gnu.org; Mon, 20 Aug 2018 17:17:09 -0400 Received: from mail-pf1-f170.google.com ([209.85.210.170]:44601) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1frrXt-0003ui-SF for 32408@debbugs.gnu.org; Mon, 20 Aug 2018 17:17:06 -0400 Received: by mail-pf1-f170.google.com with SMTP id k21-v6so7382603pff.11 for <32408@debbugs.gnu.org>; Mon, 20 Aug 2018 14:17:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:references:user-agent:from:to:cc:subject:in-reply-to:date :message-id:mime-version; bh=vG8MATBZ6+uCdwLUwHJU2m9z2ZMZeclMsCB80yLiS+M=; b=tu98VU9kaUigQp9qndu41f3lCYqaLzuqq/iVlrTaCfm851+wvtY45Q0n8GD5k9sOAX BCITJyzrjwrXpnQtjBdCQ/uvcGNJp4F5QjI3a6fKgEi0IThZ7O1pTc8xd1l+fZm5z6hK WqLyjIEAD4joqUWSf1nmaDo/feHDe3p+NJTjAEjejxJUOyH5E7NHzrjNH9FCqIlW2/Vb ujl2I+wB90j/KG8HK7m2vc9MMcPd3EEk/9ogMP9Vew0MzE4FYELX5zddtqJ6NLijWqwT y334wOHVSXZm7ZsoE7ZeMuUsrZhOrt4vS0SR0REvLeSE8+prj35/RrUIUr4L/pewQ9lQ /wSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:references:user-agent:from:to:cc:subject :in-reply-to:date:message-id:mime-version; bh=vG8MATBZ6+uCdwLUwHJU2m9z2ZMZeclMsCB80yLiS+M=; b=O4DGSwjkiZdu7wELaGaxhBvxmZegJMzEw3rPbBfp9nHPpmh5qipBMh8vtezy4TPkPK Vg7qHj44yXrnx8TEDzQlrvhT6kp2OScHrn5Eao+vO0d75iq7oPAR8Vyzd20WvdPwq4Lt 7wdKGUa7L7UxaEWp78p0DkuWWKqmXfiA8NmsUwOubKx5o00Xsb6rqMJvECGBJOys/1DH WeCh0cXKSnoL+EvobeZMAbxchExRKXkuAOciGitFxcTxiAMvIQTdnUZuV3tEfPCSZ7Nm ka5kM/VghFgVTE5RPpgTA9Z4ejZjB5QtqgGht1sfIoKdjqBNHFEH5OnTSO1a44ushgmM rZjA== X-Gm-Message-State: AOUpUlFcaIwT8Mb3pQ5sM8xdYIhAQQPjivmY5u2TNZkRM4YQoRYIzDeQ xZxhYXSY/WbqY0jLe/Fd0xr1B30B X-Google-Smtp-Source: AA+uWPwtXrLzTi1QgvU8X08O2ITlYA2VNHuMBp0Kh3cfdX9UuaU32Enk/wXuJxb8Iyg+yNSblZSI5g== X-Received: by 2002:a63:9412:: with SMTP id m18-v6mr7244544pge.440.1534799819795; Mon, 20 Aug 2018 14:16:59 -0700 (PDT) Received: from pidgey (210-1-202-160-cpe.spintel.net.au. [210.1.202.160]) by smtp.gmail.com with ESMTPSA id g4-v6sm6219618pgl.47.2018.08.20.14.16.56 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 20 Aug 2018 14:16:58 -0700 (PDT) References: <87wosz4spx.fsf@zancanaro.id.au> <87h8jobwwp.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Carlo Zancanaro In-reply-to: <87h8jobwwp.fsf@gnu.org> Date: Tue, 21 Aug 2018 07:16:48 +1000 Message-ID: <87k1okzqkf.fsf@zancanaro.id.au> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.2 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) --=-=-= Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Hey Ludo, On Tue, Aug 21 2018, Ludovic Court=C3=A8s wrote: > We=E2=80=99ll still want to be able to special-case things like nginx=20 > that can be =E2=80=9Chot-replaced=E2=80=9D, though. So perhaps, in addit= ion to=20 > this patch on the Shepherd side, we=E2=80=99ll need extra stuff in (gnu=20 > services shepherd). Yeah, if we expose the replacement field directly, then we'll need=20 some supporting code in (gnu services shepherd), even if it's just=20 to detect whether the service is stopped or not before doing a=20 replacement. Although ideally our interface wouldn't introduce=20 race conditions like that. (See below for more thoughts on this.) > For instance, the =E2=80=98actions=E2=80=99 field of c= ould,=20 > by default, include an =E2=80=9Cupgrade=E2=80=9D action that simply sets = the=20 > =E2=80=98replacement=E2=80=99 slot. For nginx, we=E2=80=99d provide a cu= stom =E2=80=9Cupgrade=E2=80=9D=20 > action that does =E2=80=9Cnginx -s restart=E2=80=9D or whatever it is tha= t needs=20 > to be done. > > =E2=80=98guix system reconfigure=E2=80=99 would automatically invoke the= =20 > =E2=80=98upgrade=E2=80=99 action for each new service. > > WDYT? How many services can we meaningfully upgrade like this? My=20 understanding is that most of our system services a fed an=20 immutable configuration file, and thus restarting/reloading won't=20 actually upgrade them. In order to make an upgrade action work the=20 service would have to mutate itself into a new correct=20 configuration, as well as restarting/reloading the underlying=20 daemon. It's even trickier if the daemon itself has been upgraded,=20 because then the process will have to be restarted anyway. At any rate, I think the replacement mechanism (this patch) is=20 just one way that a service can be reloaded. It would probably be=20 a good idea to create a higher-level abstraction over it. I think=20 other mechanisms (like a upgrade/reload action) should be handled=20 on the Guix side of things. >> + (let ((replacement (slot-ref service 'replacement))) >> + (define (copy-slot! slot) >> + (slot-set! service slot (slot-ref replacement slot))) >> + (when replacement >> + (copy-slot! 'provides) >> + (copy-slot! 'requires) >> + (copy-slot! 'respawn?) >> + (copy-slot! 'start) >> + (copy-slot! 'stop) >> + (copy-slot! 'actions) >> + (copy-slot! 'running) >> + (copy-slot! 'docstring)) >> + service)) > > Having a hardcoded list of slots sounds error-prone=E2=80=94surely we=E2= =80=99ll=20 > forget to update it down the road. I wonder what else could be=20 > done. > > One option would be to grab the block asyncs and atomically=20 > replace the service in the =E2=80=98%services=E2=80=99 hash table. Then = we only=20 > need to copy the =E2=80=98last-respawns=E2=80=99 slot to the new service,= I=20 > believe. (This changes the object identity of the service but I=20 > think its OK.) > > Another option would be to use GOOPS tricks to iterate over the=20 > list of slots and have a list of slots *not* to copy. I=E2=80=99m not a= =20 > big fan of this option, though. My favourite option for this would be to separate the =20 object into an immutable and a mutable .=20 The object would have a reference to a =20 object in order to invoke actions on it, and it could also hold a=20 second object as a replacement. Then the swap would be=20 much more straightforward. I haven't done any real work towards=20 this, though. In the short term, I'd rather replace it in the %services hash=20 table. I did it by copying slots because I wasn't sure I would get=20 the details of the swap right and didn't have time to properly=20 work out how to do it. I'll give it a go! >> +(let ((service (lookup-running 'test))) >> + (slot-set! service 'replacement >> + (make > > I wonder if we should let users fiddle with =E2=80=98replacement=E2=80=99= =20 > directly, or if we should provide a higher-level construct. > > For instance, =E2=80=98register-services=E2=80=99 could transparently set= the=20 > =E2=80=98replacement=E2=80=99 field for services already registered inste= ad of=20 > doing: > > (assert (null? (lookup-services (canonical-name new)))) > > Not sure if there are cases where this behavior would be=20 > undesirable, though. > > Thoughts? With this current patch the replacement field is only checked at=20 the point when the service is stopped, so the field could only be=20 set when the service is actually running. I think it makes the=20 most sense to just replace the service directly if it's not=20 stopped. I can't think of any undesirable cases, but having a higher-level=20 interface is a good idea. At the very least we need to control the=20 inherent race condition involved in (if running? do-x do-y) for if=20 the service is stopped after the running? check. At the moment I=20 think the only thing we have to worry about there is signals, but=20 if we're going to move to have more parallelism through fibers=20 then we might need to be even more careful. I'll try to send through an updated patch later this week. Carlo --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE1lpncq7JnOkt+LaeqdyPv9awIbwFAlt7L8AACgkQqdyPv9aw IbyKtw//X2xkrVY9HAtGQ7ROk1k7PMGAEFpCJyxEFbcMh3BkbmJxbhdX5Gov1+3M U67UX/t2D239Q07W28FPw5psm2PPowSZMx+QIcl+CA2H3efPuRmgdXlAPuMzn8gn CZxLcDoS9lhB1vFGHIG6AkHVzGFj6OB2HhDNG/id/KyumUOPMRrqrNDTtt9+60pB X/At8EpNp1QGtgq4ZOVXvmEufDX4TWzSdAFYh82AiiTuZZFeDhqZ7zDEk8jB/ng9 MPtyt8RdWfnqmxgzwHWf/Dniiow8nHw9Z9mYX+P4g7/6ZRWvDTLeGFKn5fjG9fVz Ntg71O7DDyuM7bm24R2/JW7cN1Qv9gP4AYOAfrYxJgzNntNmnOVniZyFRS+dtm0p z6iF/LqsDkit9eRnWW5K8MuEu9AwtV8HTANNmGgOnU4Oc7soVzBJnbvJYuxTPEm7 rfliFke/HNGBnvSfoXa5JzGPYJz8wVlnMdUc7CvATVobUqty4sJ4NbSLQDxNDlSE 9uTZH5irxE5E2RY1aPgsl+4uU1KbMw+rah5d40pe0ddVu1WI0llSmij0IpJTaFm9 +IzMoaiJhrk0ceocs8aEcmx9gjl09ELcUiEIvoorz/Z9NMdLLJ04eJHlWNXgTgu7 QA37arySGg6zxnQOiExJC+/B6l+7hw341+pG+YDrA6gBFIpp9ys= =5vP+ -----END PGP SIGNATURE----- --=-=-=-- From unknown Tue Jun 17 22:18:11 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#32408] [PATCH shepherd] Allow replacement of services Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 21 Aug 2018 10:28:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 32408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Carlo Zancanaro Cc: 32408@debbugs.gnu.org Received: via spool by 32408-submit@debbugs.gnu.org id=B32408.153484725910180 (code B ref 32408); Tue, 21 Aug 2018 10:28:01 +0000 Received: (at 32408) by debbugs.gnu.org; 21 Aug 2018 10:27:39 +0000 Received: from localhost ([127.0.0.1]:56200 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fs3sx-0002e8-4R for submit@debbugs.gnu.org; Tue, 21 Aug 2018 06:27:39 -0400 Received: from eggs.gnu.org ([208.118.235.92]:39556) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fs3sv-0002dw-LN for 32408@debbugs.gnu.org; Tue, 21 Aug 2018 06:27:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fs3sl-0000TL-B4 for 32408@debbugs.gnu.org; Tue, 21 Aug 2018 06:27:32 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:56899) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fs3sl-0000TF-6c; Tue, 21 Aug 2018 06:27:27 -0400 Received: from [193.50.110.186] (port=44842 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1fs3sk-0006FT-Sf; Tue, 21 Aug 2018 06:27:27 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <87wosz4spx.fsf@zancanaro.id.au> <87h8jobwwp.fsf@gnu.org> <87k1okzqkf.fsf@zancanaro.id.au> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 Fructidor an 226 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 21 Aug 2018 12:27:25 +0200 In-Reply-To: <87k1okzqkf.fsf@zancanaro.id.au> (Carlo Zancanaro's message of "Tue, 21 Aug 2018 07:16:48 +1000") Message-ID: <87in44ovzm.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Hey Carlo, Carlo Zancanaro skribis: > On Tue, Aug 21 2018, Ludovic Court=C3=A8s wrote: >> For instance, the =E2=80=98actions=E2=80=99 field of = could, by >> default, include an =E2=80=9Cupgrade=E2=80=9D action that simply sets the >> =E2=80=98replacement=E2=80=99 slot. For nginx, we=E2=80=99d provide a c= ustom =E2=80=9Cupgrade=E2=80=9D >> action that does =E2=80=9Cnginx -s restart=E2=80=9D or whatever it is th= at needs to >> be done. >> >> =E2=80=98guix system reconfigure=E2=80=99 would automatically invoke the= =E2=80=98upgrade=E2=80=99 >> action for each new service. >> >> WDYT? > > How many services can we meaningfully upgrade like this? Probably very few. > My understanding is that most of our system services a fed an immutable > configuration file, and thus restarting/reloading won't actually > upgrade them. In order to make an upgrade action work the service > would have to mutate itself into a new correct configuration, as well > as restarting/reloading the underlying daemon. It's even trickier if > the daemon itself has been upgraded, because then the process will > have to be restarted anyway. Yeah. > At any rate, I think the replacement mechanism (this patch) is just > one way that a service can be reloaded. It would probably be a good > idea to create a higher-level abstraction over it. I think other > mechanisms (like a upgrade/reload action) should be handled on the > Guix side of things. Sounds good. >>> + (let ((replacement (slot-ref service 'replacement))) >>> + (define (copy-slot! slot) >>> + (slot-set! service slot (slot-ref replacement slot))) >>> + (when replacement >>> + (copy-slot! 'provides) >>> + (copy-slot! 'requires) >>> + (copy-slot! 'respawn?) >>> + (copy-slot! 'start) >>> + (copy-slot! 'stop) >>> + (copy-slot! 'actions) >>> + (copy-slot! 'running) >>> + (copy-slot! 'docstring)) >>> + service)) >> >> Having a hardcoded list of slots sounds error-prone=E2=80=94surely we=E2= =80=99ll >> forget to update it down the road. I wonder what else could be >> done. >> >> One option would be to grab the block asyncs and atomically replace >> the service in the =E2=80=98%services=E2=80=99 hash table. Then we only= need to >> copy the =E2=80=98last-respawns=E2=80=99 slot to the new service, I beli= eve. (This >> changes the object identity of the service but I think its OK.) >> >> Another option would be to use GOOPS tricks to iterate over the list >> of slots and have a list of slots *not* to copy. I=E2=80=99m not a big = fan >> of this option, though. > > My favourite option for this would be to separate the object > into an immutable and a mutable . The > object would have a reference to a object in > order to invoke actions on it, and it could also hold a second > object as a replacement. Then the swap would be much more > straightforward. I haven't done any real work towards this, though. I agree that separating state is the right approach longer-term (it=E2=80= =99s beyond the scope of this patch.) > In the short term, I'd rather replace it in the %services hash > table. I did it by copying slots because I wasn't sure I would get the > details of the swap right and didn't have time to properly work out > how to do it. I'll give it a go! Alright! >>> +(let ((service (lookup-running 'test))) >>> + (slot-set! service 'replacement >>> + (make >> >> I wonder if we should let users fiddle with =E2=80=98replacement=E2=80= =99 directly, >> or if we should provide a higher-level construct. >> >> For instance, =E2=80=98register-services=E2=80=99 could transparently se= t the >> =E2=80=98replacement=E2=80=99 field for services already registered inst= ead of >> doing: >> >> (assert (null? (lookup-services (canonical-name new)))) >> >> Not sure if there are cases where this behavior would be >> undesirable, though. >> >> Thoughts? > > With this current patch the replacement field is only checked at the > point when the service is stopped, so the field could only be set when > the service is actually running. I think it makes the most sense to > just replace the service directly if it's not stopped. > > I can't think of any undesirable cases, but having a higher-level > interface is a good idea. Right. Currently the Guix side of things does the equivalent of: herd eval root '(register-services (load "a.scm") (load "b.scm"))' for services currently stopped, which is okay but not great. IWBN if it could directly use a higher-level action. > At the very least we need to control the inherent race condition > involved in (if running? do-x do-y) for if the service is stopped > after the running? check. At the moment I think the only thing we have > to worry about there is signals, but if we're going to move to have > more parallelism through fibers then we might need to be even more > careful. Indeed. Thank you, Ludo=E2=80=99. From unknown Tue Jun 17 22:18:11 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#32408] [PATCH shepherd] Allow replacement of services Resent-From: Carlo Zancanaro Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 23 Aug 2018 13:46:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 32408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 32408@debbugs.gnu.org Received: via spool by 32408-submit@debbugs.gnu.org id=B32408.153503193317655 (code B ref 32408); Thu, 23 Aug 2018 13:46:02 +0000 Received: (at 32408) by debbugs.gnu.org; 23 Aug 2018 13:45:33 +0000 Received: from localhost ([127.0.0.1]:58367 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fspvV-0004ab-CX for submit@debbugs.gnu.org; Thu, 23 Aug 2018 09:45:33 -0400 Received: from mail-pl0-f43.google.com ([209.85.160.43]:42450) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fspvP-0004aI-Ek for 32408@debbugs.gnu.org; Thu, 23 Aug 2018 09:45:27 -0400 Received: by mail-pl0-f43.google.com with SMTP id g23-v6so2433407plq.9 for <32408@debbugs.gnu.org>; Thu, 23 Aug 2018 06:45:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:references:user-agent:from:to:cc:subject:in-reply-to:date :message-id:mime-version; bh=m61n1/QJpwFb3VaRFnUuTFhSDFqa3MfRtc2NFZDxCQ8=; b=KVqfzAkg/0JBUwrQ+D4A2otUsdGCjbDW1VZx475kdhd8QGIcbf0l1ivi6SmhaFy/1z 79Lh3gm0a7zzwdlgel+Y9a34C9RNMgIV4rDyNnrW8B21kTN8Y7SvRAw+0DLoIBJd3ZiE YjzbQX2gapAXzozYOuNz+xhqqknRXjfpM/de4A5qJRLQQhPRW5yOkU7wfriLEpEuTXjP fTTZNvAOfs4xMjEl7qTSw8lgs9FLlu0SFlxwxVlZcnB8AA5NJCqlZozN14Mc9hjIFxgZ alrPVyuD/8qZt1fMsI+7F2ar8UzDCGAsetwovDkhSjmutT9J2GJpJNcGqZAC3gsI+3p2 4DAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:references:user-agent:from:to:cc:subject :in-reply-to:date:message-id:mime-version; bh=m61n1/QJpwFb3VaRFnUuTFhSDFqa3MfRtc2NFZDxCQ8=; b=a88dVk4O1jM0SAvEzNMABqxxHfI+rH52WoA/N3oi2OaZ542kF5FBUh7X+OoqsK/oNN ptAGNG+kKwgmjpgoi1a6OmFNf8XDyEpVindq0gCqSqDgX24+/wWBvwM9H7u+mhEI5/k3 FXNE6igNjS5Eeuy6iJxtr183SJ/tt0+NZmNZ3k1Hs8RNHyA4Pah/IRkUbM8jUBJ83z8h I4j8hmLxfsa1wd2De4RhFem8i/6Bc/VQ8rGwpErltBHao6FQ9y1LK331QJrKnHNWXKTS hforX+i0QUkVHWrk/y6RoFGPpIGY2fy9Vpv9xMBgETPL3XYUyRhSd9mcoPiNshXz2DbL oE6A== X-Gm-Message-State: APzg51DWLi0sXMh3I+fUSvyr55umxlWoT2x40gtxcZX2q+7sP6JfXiIL h5opA2BGneMbWc4zzhsiiLraQQJT X-Google-Smtp-Source: ANB0VdZiJ08Bme7Qw5yXxW7tYLr3N/vVDvV3JlfGE1tKY7GblUKYwTYWShW1QqFp0ub6lXK/Kmgfvg== X-Received: by 2002:a17:902:6a8a:: with SMTP id n10-v6mr11097406plk.302.1535031917309; Thu, 23 Aug 2018 06:45:17 -0700 (PDT) Received: from pidgey (210-1-202-160-cpe.spintel.net.au. [210.1.202.160]) by smtp.gmail.com with ESMTPSA id g5-v6sm6001638pgn.73.2018.08.23.06.45.14 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 23 Aug 2018 06:45:16 -0700 (PDT) References: <87wosz4spx.fsf@zancanaro.id.au> <87h8jobwwp.fsf@gnu.org> <87k1okzqkf.fsf@zancanaro.id.au> <87in44ovzm.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Carlo Zancanaro In-reply-to: <87in44ovzm.fsf@gnu.org> Date: Thu, 23 Aug 2018 23:45:05 +1000 Message-ID: <87o9dtqjry.fsf@zancanaro.id.au> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.3 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Hey Ludo=E2=80=99, I've attached an updated patch. I couldn't think of any unwanted=20 consequences, so I took your idea of making register-services=20 handle most of the details of replacement. With my patch,=20 something like > herd eval root '(register-services (load "a.scm") (load=20 > "b.scm"))' will deal with a conflict by either replacing the old service (if=20 it's not running), arranging for the old service to be replaced=20 when it's stopped, or raising an error. This seems like a sensible=20 way for things to function. >> At the very least we need to control the inherent race=20 >> condition [...] > > Indeed. Despite my desire to deal with the race condition, I haven't done=20 anything about it in this patch. The modification of %services=20 that was done in register-services was already racy, and I don't=20 think this patch will make it worse. If it hasn't been a problem=20 up until now, then I don't think this will make it a problem. Carlo --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-service-Add-a-replacement-slot-for-delayed-service-r.patch Content-Transfer-Encoding: quoted-printable From=209ec5c0000e9a45441417a6ee4138cdcbf1b1f2b2 Mon Sep 17 00:00:00 2001 From: Carlo Zancanaro Date: Thu, 9 Aug 2018 22:30:38 +1000 Subject: [PATCH] service: Add a replacement slot for delayed service replacement. * modules/shepherd/service.scm (): Add replacement slot (replace-service): New procedure. (stop): Call replace-service after stopping a service. (register-services): Replace existing services where possible, setting the = new replacement slot if they are currently running. * tests/replacement.sh: Add a test for it. * Makefile.am (TESTS): Add the new test. * doc/shepherd.texi (Slots of services): Document it. =2D-- Makefile.am | 1 + doc/shepherd.texi | 9 +++ modules/shepherd/service.scm | 68 ++++++++++++++++++----- tests/replacement.sh | 105 +++++++++++++++++++++++++++++++++++ 4 files changed, 168 insertions(+), 15 deletions(-) create mode 100644 tests/replacement.sh diff --git a/Makefile.am b/Makefile.am index 8dad006..4322d7f 100644 =2D-- a/Makefile.am +++ b/Makefile.am @@ -184,6 +184,7 @@ SUFFIXES =3D .go =20 TESTS =3D \ tests/basic.sh \ + tests/replacement.sh \ tests/respawn.sh \ tests/respawn-throttling.sh \ tests/misbehaved-client.sh \ diff --git a/doc/shepherd.texi b/doc/shepherd.texi index 7946f8b..1de6d80 100644 =2D-- a/doc/shepherd.texi +++ b/doc/shepherd.texi @@ -708,6 +708,15 @@ handler will not start it again. =20 otherwise @code{#f}. =20 +@item +@vindex replacement (slot of ) +@code{replacement} specifies a service to be used to replace this one +when it is stopped. This service will continue to function normally +until the @code{stop} action is invoked. After the service has been +successfully stopped, its definition will be replaced by the value of +this slot, which must itself be a service. This slot is ignored if +its value is @code{#f}. + @end itemize =20 @c @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ diff --git a/modules/shepherd/service.scm b/modules/shepherd/service.scm index 5653388..006309c 100644 =2D-- a/modules/shepherd/service.scm +++ b/modules/shepherd/service.scm @@ -205,7 +205,10 @@ respawned, shows that it has been respawned more than = TIMES in SECONDS." (stop-delay? #:init-keyword #:stop-delay? #:init-value #f) ;; The times of the last respawns, most recent first. =2D (last-respawns #:init-form '())) + (last-respawns #:init-form '()) + ;; A replacement for when this service is stopped. + (replacement #:init-keyword #:replacement + #:init-value #f)) =20 (define (service? obj) "Return true if OBJ is a service." @@ -341,6 +344,20 @@ wire." (canonical-name obj))))) (slot-ref obj 'running)) =20 +(define (replace-service old-service new-service) + "Replace OLD-SERVICE with NEW-SERVICE in the services registry. This +completely removes all references to OLD-SERVICE before registering +NEW-SERVICE." + (define (remove-service name) + (let* ((old (hashq-ref %services name)) + (new (delete old-service old))) + (if (null? new) + (hashq-remove! %services name) + (hashq-set! %services name new)))) + (when new-service + (for-each remove-service (provided-by old-service)) + (register-services new-service))) + ;; Stop the service, including services that depend on it. If the ;; latter fails, continue anyway. Return `#f' if it could be stopped. (define-method (stop (obj ) . args) @@ -385,6 +402,11 @@ wire." ;; Reset the list of respawns. (slot-set! obj 'last-respawns '()) =20 + ;; Replace the service with its replacement, if it has one + (let ((replacement (slot-ref obj 'replacement))) + (when replacement + (replace-service obj replacement))) + ;; Status message. (let ((name (canonical-name obj))) (if (running? obj) @@ -1038,25 +1060,41 @@ then disable it." =20 ;; Add NEW-SERVICES to the list of known services. (define (register-services . new-services) + "Add NEW-SERVICES to the list of known services. If a service has alrea= dy +been registered, arrange to have it replaced when it is next stopped. If = it +is currently stopped, replace it immediately." (define (register-single-service new) ;; Sanity-checks first. (assert (list-of-symbols? (provided-by new))) (assert (list-of-symbols? (required-by new))) (assert (boolean? (respawn? new))) =2D ;; Canonical name actually must be canonical. (FIXME: This test =2D ;; is incomplete, since we may add a service later that makes it =2D ;; non-cannonical.) =2D (assert (null? (lookup-services (canonical-name new)))) =2D ;; FIXME: Verify consistency: Check that there are no circular =2D ;; dependencies, check for bogus conflicts/dependencies, whatever =2D ;; else makes sense. =2D =2D ;; Insert into the hash table. =2D (for-each (lambda (name) =2D (let ((old (lookup-services name))) =2D ;; Actually add the new service now. =2D (hashq-set! %services name (cons new old)))) =2D (provided-by new))) + + ;; FIXME: Just because we have a unique canonical name now doesn't mea= n it + ;; will remain unique as other services are added. Whenever a service = is + ;; added it should check that it's not conflicting with any already + ;; registered canonical names. + (match (lookup-services (canonical-name new)) + (() ;; empty, so we can safely add ourselves + (for-each (lambda (name) + (let ((old (lookup-services name))) + (hashq-set! %services name (cons new old)))) + (provided-by new))) + ((old) ;; one service registered, so it may be an old version of us + (cond + ((not (eq? (canonical-name new) (canonical-name old))) + (local-output + "Cannot register service ~a: canonical name is not unique." + (canonical-name new)) + (throw 'non-canonical-name)) + ((running? old) + (slot-set! old 'replacement new)) + (#:else + (replace-service old new)))) + (_ ;; in any other case, there are too many services to register + (local-output + "Cannot register service ~a: canonical name is not unique." + (canonical-name new)) + (throw 'non-canonical-name)))) =20 (for-each register-single-service new-services)) =20 diff --git a/tests/replacement.sh b/tests/replacement.sh new file mode 100644 index 0000000..e06cb93 =2D-- /dev/null +++ b/tests/replacement.sh @@ -0,0 +1,105 @@ +# GNU Shepherd --- Ensure replacing services works properly +# Copyright =C2=A9 2014, 2016 Ludovic Court=C3=A8s +# Copyright =C2=A9 2018 Carlo Zancanaro +# +# This file is part of the GNU Shepherd. +# +# The GNU Shepherd is free software; you can redistribute it and/or modify= it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or (at +# your option) any later version. +# +# The GNU Shepherd is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with the GNU Shepherd. If not, see . + +shepherd --version +herd --version + +socket=3D"t-socket-$$" +conf=3D"t-conf-$$" +rconf=3D"t-rconf-$$" +log=3D"t-log-$$" +stamp=3D"t-stamp-$$" +pid=3D"t-pid-$$" + +herd=3D"herd -s $socket" + +trap "rm -f $socket $conf $rconf $stamp $log; + test -f $pid && kill \`cat $pid\` || true; rm -f $pid" EXIT + +cat > "$conf"< + #:provides '(test) + #:start (const #t) + #:actions (make-actions + (say-hello (lambda _ + (call-with-output-file "$stamp" + (lambda (port) + (display "Hello" port)))))) + #:respawn? #f)) +EOF + +rm -f "$pid" "$stamp" "$socket" +shepherd -I -s "$socket" -c "$conf" --pid=3D"$pid" --log=3D"$log" & + +while ! test -f "$pid"; do sleep 0.5 ; done + +$herd start test + +if ! $herd say-hello test; then + echo "say-hello failed" + exit 1 +fi + +cat > "$rconf"< + #:provides '(test) + #:start (const #t) + #:actions (make-actions + (say-goodbye (lambda _ + (call-with-output-file "$stamp" + (lambda (port) + (display "Goodbye" port)))))) + #:respawn? #f)) +EOF + +$herd load root "$rconf" + +if ! $herd say-hello test; then + echo "say-hello failed after setting replacement" + exit 1 +fi + +if test "`cat $stamp`" !=3D "Hello"; then + echo "Output file had the wrong contents! Was:" + cat $stamp + exit 1 +fi + +$herd stop test + +$herd start test + +if $herd say-hello test; then + echo "say-hello should have failed after stop/start" + exit 1 +fi + +if ! $herd say-goodbye test; then + echo "say-goodbye failed after replacement" + exit 1 +fi + +if test "`cat $stamp`" !=3D "Goodbye"; then + echo "Output file had the wrong contents! Was:" + cat $stamp + exit 1 +fi =2D-=20 2.18.0 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE1lpncq7JnOkt+LaeqdyPv9awIbwFAlt+umEACgkQqdyPv9aw Ibz9VQ/8CcWPyF3dAAxTQVS20gJs/b3NpTgQdTf4Vj+p9PTDrzY9NJgh2HMyvZJ4 OvoEZEzS7LfRV3InERRwJbEH8npN1PUrJz196EAD5AVwAynDHrwe/KwZDqk38ScP BIEpcOr34Jzn/CZ2c+CnXyYZsXf7omJtcP6LHLdq2yBW/KkKB5UagAxNjNzf/wPZ qzoZ1zb4XJ+tMzCaupUm+J4OyVgnEjqCvWy96L47QSD3TP9dl4701oJaZ7Vvzsm9 08+8ikDJ71ZKWhQ9PniKLj9pWrKibcBMMZ9PTY6zw69NJ2DHOH1AONWPzCmRKiy3 c776j3vkFXhQ2SUyprphzF7CfZ7OEgH2xwgy2TAphwNPZtHtamngQcR6MLECOQIP fKlGzxYdQ44u8TLCKkeR/ngMrJLMy5FuCymKuMqO3NUOHopp8qnK5qiUnp8RcaFV LFYOaWcugGqt0eugKTaGnHap5UtMVMXukmoQcLAzv6fBYYiyFOnFvXxCKOi0VXw0 6b+UDmjs2EuC5lmx/mMf3HlS3FxvyDfo41PdPoeKPXQBfDRzVepapBb+4nw6lqJj WLu+h6YsYrwWJxJx+9vP9mmGCTLDCXIiEfCmry+pkZsMndC+d0Tt0X/xYd6p3yHh bOW7NbNUG0/C3aYVjV62nIYTWSdpVQ3X14hd4OVYuAtYbrV7Vuk= =yrpC -----END PGP SIGNATURE----- --==-=-=-- From unknown Tue Jun 17 22:18:11 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#32408] [PATCH shepherd] Allow replacement of services Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 25 Aug 2018 14:21:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 32408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Carlo Zancanaro Cc: 32408@debbugs.gnu.org Received: via spool by 32408-submit@debbugs.gnu.org id=B32408.153520682512272 (code B ref 32408); Sat, 25 Aug 2018 14:21:01 +0000 Received: (at 32408) by debbugs.gnu.org; 25 Aug 2018 14:20:25 +0000 Received: from localhost ([127.0.0.1]:60698 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ftZQP-0003Br-JJ for submit@debbugs.gnu.org; Sat, 25 Aug 2018 10:20:25 -0400 Received: from eggs.gnu.org ([208.118.235.92]:33385) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ftZQN-0003Bc-SP for 32408@debbugs.gnu.org; Sat, 25 Aug 2018 10:20:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ftZQH-0000Cr-Q7 for 32408@debbugs.gnu.org; Sat, 25 Aug 2018 10:20:18 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:47283) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ftZQH-0000Cf-Mn; Sat, 25 Aug 2018 10:20:17 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=43998 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ftZQH-0005lA-Br; Sat, 25 Aug 2018 10:20:17 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <87wosz4spx.fsf@zancanaro.id.au> <87h8jobwwp.fsf@gnu.org> <87k1okzqkf.fsf@zancanaro.id.au> <87in44ovzm.fsf@gnu.org> <87o9dtqjry.fsf@zancanaro.id.au> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 8 Fructidor an 226 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 25 Aug 2018 16:20:15 +0200 In-Reply-To: <87o9dtqjry.fsf@zancanaro.id.au> (Carlo Zancanaro's message of "Thu, 23 Aug 2018 23:45:05 +1000") Message-ID: <877ekecyu8.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Hello! Carlo Zancanaro skribis: > I've attached an updated patch. I couldn't think of any unwanted > consequences, so I took your idea of making register-services handle > most of the details of replacement. With my patch, something like >> herd eval root '(register-services (load "a.scm") (load >> "b.scm"))' > will deal with a conflict by either replacing the old service (if it's > not running), arranging for the old service to be replaced when it's > stopped, or raising an error. This seems like a sensible way for > things to function. Awesome. Thus, the only thing we need to change in =E2=80=98guix system reconfigure=E2=80=99 is to make the =E2=80=98register-services=E2=80=99 cal= l unconditional (currently it=E2=80=99s limited to services that are stopped.) > Despite my desire to deal with the race condition, I haven't done > anything about it in this patch. The modification of %services that > was done in register-services was already racy, and I don't think this > patch will make it worse. If it hasn't been a problem up until now, > then I don't think this will make it a problem. Yeah, sounds reasonable. > From 9ec5c0000e9a45441417a6ee4138cdcbf1b1f2b2 Mon Sep 17 00:00:00 2001 > From: Carlo Zancanaro > Date: Thu, 9 Aug 2018 22:30:38 +1000 > Subject: [PATCH] service: Add a replacement slot for delayed service > replacement. > > * modules/shepherd/service.scm (): Add replacement slot > (replace-service): New procedure. > (stop): Call replace-service after stopping a service. > (register-services): Replace existing services where possible, setting th= e new > replacement slot if they are currently running. > * tests/replacement.sh: Add a test for it. > * Makefile.am (TESTS): Add the new test. > * doc/shepherd.texi (Slots of services): Document it. Awesome, please push! And sorry about the delay. Thank you, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 02 05:50:36 2018 Received: (at control) by debbugs.gnu.org; 2 Oct 2018 09:50:36 +0000 Received: from localhost ([127.0.0.1]:33797 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g7HK8-0002qj-FG for submit@debbugs.gnu.org; Tue, 02 Oct 2018 05:50:36 -0400 Received: from eggs.gnu.org ([208.118.235.92]:43610) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g7HK6-0002qW-N3 for control@debbugs.gnu.org; Tue, 02 Oct 2018 05:50:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g7HJw-0004iW-Jk for control@debbugs.gnu.org; Tue, 02 Oct 2018 05:50:29 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:46095) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g7HJu-0004eK-NG for control@debbugs.gnu.org; Tue, 02 Oct 2018 05:50:22 -0400 Received: from [193.50.110.166] (port=57856 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1g7HJu-0004TB-1T for control@debbugs.gnu.org; Tue, 02 Oct 2018 05:50:22 -0400 Date: Tue, 02 Oct 2018 11:50:20 +0200 Message-Id: <87r2h8r82r.fsf@gnu.org> To: control@debbugs.gnu.org From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: control message for bug #32408 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) tags 32408 fixed close 32408