GNU bug report logs - #76698
Activations interfere with each other modules

Previous Next

To reply to this bug, email your comments to 76698 AT debbugs.gnu.org.
There is no need to reopen the bug first.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: bug-guix <at> gnu.org
Subject: Activations interfere with each other modules
Date: Mon, 03 Mar 2025 02:02:22 +0100

Hello,

when using `activation-service-type' combined with `with-extension', the
various modules seem to interfere with each other.

In my case the samba-service-type interferes with my custom service:

--8<---------------cut here---------------start------------->8---
(define (watcher-activation* config)
  (match-record config <watcher-configuration>
                (user group cache-dir state-dir)
    (with-extension/guile-wolfsden
     #~(begin
         (use-modules (wolfsden fs)
                      (wolfsden sh))
         (for-each (lambda (dir)
                     (mkdir-p dir)
                     (invoke* #$(file-append coreutils "/bin/chown")
                              "-Rv" (string-append #$user ":" #$group) dir)
                     (chmod dir #o700))
                   '(#$cache-dir
                     #$state-dir))))))
--8<---------------cut here---------------end--------------->8---

samba-service-type's activation starts with:

--8<---------------cut here---------------start------------->8---
(define (samba-activation config)
  (let ((package (samba-configuration-package config))
        (config-file (samba-configuration-config-file config)))
    (with-imported-modules '((guix build utils))
--8<---------------cut here---------------end--------------->8---

Both (guix build utils) and (wolfsden sh) provide (invoke) procedure,
however with incompatible parameters, leading to somewhat annoying to
debug issues.  In my case the error was:

--8<---------------cut here---------------start------------->8---
guix deploy: error: failed to deploy x: failed to switch systems while deploying '127.0.0.1':
keyword-argument-error #f "Invalid keyword" () ("/etc/samba/smb.conf") 
--8<---------------cut here---------------end--------------->8---

Which makes sense now when I understand what the problem is, but it took
while to get to it.

I do not think this should be the case.  Either each activation service
should be isolated into a fresh user module, or the services provided by
Guix should not introduce any additional modules into the scope.
Alternatively, each activation service could be wrapped into
program-file, that would ensure isolation as well.

Tomas
-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

Message #8 received at 76698 <at> debbugs.gnu.org (full text, mbox):

From: Hilton Chain <hako <at> ultrarare.space>
To: Tomas Volf <~@wolfsden.cz>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 76698 <at> debbugs.gnu.org
Subject: Re: bug#76698: Activations interfere with each other modules
Date: Thu, 06 Mar 2025 11:13:14 +0800

Hi Tomas,

On Mon, 03 Mar 2025 09:02:22 +0800,
Tomas Volf wrote:
>
> Hello,
>
> when using `activation-service-type' combined with `with-extension', the
> various modules seem to interfere with each other.
>
> In my case the samba-service-type interferes with my custom service:
>
> --8<---------------cut here---------------start------------->8---
> (define (watcher-activation* config)
>   (match-record config <watcher-configuration>
>                 (user group cache-dir state-dir)
>     (with-extension/guile-wolfsden
>      #~(begin
>          (use-modules (wolfsden fs)
>                       (wolfsden sh))
>          (for-each (lambda (dir)
>                      (mkdir-p dir)
>                      (invoke* #$(file-append coreutils "/bin/chown")
>                               "-Rv" (string-append #$user ":" #$group) dir)
>                      (chmod dir #o700))
>                    '(#$cache-dir
>                      #$state-dir))))))
> --8<---------------cut here---------------end--------------->8---
>
> samba-service-type's activation starts with:
>
> --8<---------------cut here---------------start------------->8---
> (define (samba-activation config)
>   (let ((package (samba-configuration-package config))
>         (config-file (samba-configuration-config-file config)))
>     (with-imported-modules '((guix build utils))
> --8<---------------cut here---------------end--------------->8---
>
> Both (guix build utils) and (wolfsden sh) provide (invoke) procedure,
> however with incompatible parameters, leading to somewhat annoying to
> debug issues.  In my case the error was:
>
> --8<---------------cut here---------------start------------->8---
> guix deploy: error: failed to deploy x: failed to switch systems while deploying '127.0.0.1':
> keyword-argument-error #f "Invalid keyword" () ("/etc/samba/smb.conf")
> --8<---------------cut here---------------end--------------->8---
>
> Which makes sense now when I understand what the problem is, but it took
> while to get to it.
>
> I do not think this should be the case.  Either each activation service
> should be isolated into a fresh user module, or the services provided by
> Guix should not introduce any additional modules into the scope.
> Alternatively, each activation service could be wrapped into
> program-file, that would ensure isolation as well.

Activation scripts are already ‘program-file’s, but unfortunately
(gnu build activation) and (guix build utils) are implicit dependencies.  This
is hard to change at the moment.

I have sent a patch which might partially address your issue:

[PATCH v2 3/3] services: activation: Continue on exceptions.
https://issues.guix.gnu.org/73494#26

It executes activation scripts by ‘invoke’, so they won't change the
environment.

What I'm trying to achieve is to avoid blocking the activation process when one
script fails.

With that patch, when one script fails, its backtrace is printed out by
‘invoke’, additionally a warning is emitted to indicate the failed script:
--8<---------------cut here---------------start------------->8---
activating system...
The following derivation will be built:
  /gnu/store/78x8n7b58nqcllfqfpr4p73q6n2wazak-switch-to-system.scm.drv

building /gnu/store/78x8n7b58nqcllfqfpr4p73q6n2wazak-switch-to-system.scm.drv...
making '/var/guix/profiles/system-110-link' the current system...
populating /etc from /gnu/store/v206hfm2idagh2nczc6ii0m628xyg5dk-etc...
setting up privileged programs in '/run/privileged/bin'...
Backtrace:
           3 (primitive-load "/gnu/store/59651880cmil350zf5hagyhdi7y…")
In ice-9/eval.scm:
    619:8  2 (_ #f)
    619:8  1 (_ #(#(#(#(#<directory (guile-user) 7fdc0c317…>) …) …) …))
In unknown file:
           0 (rmdir "/var/run/dbus")

ERROR: In procedure rmdir:
In procedure rmdir: Directory not empty
guix system: warning: failed to activate '/gnu/store/59651880cmil350zf5hagyhdi7y66dgz-activate-service.scm'.
The following derivation will be built:
  /gnu/store/m1c65sxmqvvpfdsazcka51miixcpsy9g-install-bootloader.scm.drv

building /gnu/store/m1c65sxmqvvpfdsazcka51miixcpsy9g-install-bootloader.scm.drv...
guix system: bootloader successfully installed on '(/efi)'
--8<---------------cut here---------------end--------------->8---

(Cc'd Ludo) Any concern on this change?  Does the "TODO: Use 'load-compiled'."
still apply?

Message #11 received at 76698 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: Hilton Chain <hako <at> ultrarare.space>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 76698 <at> debbugs.gnu.org
Subject: Re: bug#76698: Activations interfere with each other modules
Date: Fri, 07 Mar 2025 22:29:51 +0100

Hilton Chain <hako <at> ultrarare.space> writes:

> Hi Tomas,
>
> On Mon, 03 Mar 2025 09:02:22 +0800,
> Tomas Volf wrote:
>>
>> Hello,
>>
>> when using `activation-service-type' combined with `with-extension', the
>> various modules seem to interfere with each other.
>>
>> In my case the samba-service-type interferes with my custom service:
>>
>> --8<---------------cut here---------------start------------->8---
>> (define (watcher-activation* config)
>>   (match-record config <watcher-configuration>
>>                 (user group cache-dir state-dir)
>>     (with-extension/guile-wolfsden
>>      #~(begin
>>          (use-modules (wolfsden fs)
>>                       (wolfsden sh))
>>          (for-each (lambda (dir)
>>                      (mkdir-p dir)
>>                      (invoke* #$(file-append coreutils "/bin/chown")
>>                               "-Rv" (string-append #$user ":" #$group) dir)
>>                      (chmod dir #o700))
>>                    '(#$cache-dir
>>                      #$state-dir))))))
>> --8<---------------cut here---------------end--------------->8---
>>
>> samba-service-type's activation starts with:
>>
>> --8<---------------cut here---------------start------------->8---
>> (define (samba-activation config)
>>   (let ((package (samba-configuration-package config))
>>         (config-file (samba-configuration-config-file config)))
>>     (with-imported-modules '((guix build utils))
>> --8<---------------cut here---------------end--------------->8---
>>
>> Both (guix build utils) and (wolfsden sh) provide (invoke) procedure,
>> however with incompatible parameters, leading to somewhat annoying to
>> debug issues.  In my case the error was:
>>
>> --8<---------------cut here---------------start------------->8---
>> guix deploy: error: failed to deploy x: failed to switch systems while deploying '127.0.0.1':
>> keyword-argument-error #f "Invalid keyword" () ("/etc/samba/smb.conf")
>> --8<---------------cut here---------------end--------------->8---
>>
>> Which makes sense now when I understand what the problem is, but it took
>> while to get to it.
>>
>> I do not think this should be the case.  Either each activation service
>> should be isolated into a fresh user module, or the services provided by
>> Guix should not introduce any additional modules into the scope.
>> Alternatively, each activation service could be wrapped into
>> program-file, that would ensure isolation as well.
>
> Activation scripts are already ‘program-file’s,

Ah, yes, on second look you are right.  I did not realized at first,
since they are sourced using primitive-load instead of executed.

> but unfortunately (gnu build activation) and (guix build utils) are
> implicit dependencies.  This is hard to change at the moment.

I do not think this is true.  When I look at the activation script for
my service, I do not see any additional code then my service has.  So
assuming it would be executed, instead of loaded, (guix build utils)
should not be in the environment.  Am I missing something?

>
> I have sent a patch which might partially address your issue:
>
> [PATCH v2 3/3] services: activation: Continue on exceptions.
> https://issues.guix.gnu.org/73494#26
>
> It executes activation scripts by ‘invoke’, so they won't change the
> environment.
>
> What I'm trying to achieve is to avoid blocking the activation process when one
> script fails.

Well, that patch would probably resolve my issues, correct.  What I am
unsure about is whether ignoring errors is a good idea and what (if any)
impact it will have on the rest of the deploy process.  Like, if the
activation fails, would it not be better to rollback instead of push
forward?  Dunno, I probably just do not know enough about this. :)

Tomas

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

Message #14 received at 76698 <at> debbugs.gnu.org (full text, mbox):

From: Hilton Chain <hako <at> ultrarare.space>
To: Tomas Volf <~@wolfsden.cz>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 76698 <at> debbugs.gnu.org
Subject: Re: bug#76698: Activations interfere with each other modules
Date: Sun, 09 Mar 2025 13:20:28 +0800

On Sat, 08 Mar 2025 05:29:51 +0800,
Tomas Volf wrote:
>
> > but unfortunately (gnu build activation) and (guix build utils) are
> > implicit dependencies.  This is hard to change at the moment.
>
> I do not think this is true.  When I look at the activation script for
> my service, I do not see any additional code then my service has.  So
> assuming it would be executed, instead of loaded, (guix build utils)
> should not be in the environment.  Am I missing something?

(gnu build activation) and (guix build utils) are added to the environment
before loading activation scripts, existing activation services may depend on
this.  For example firmware-service-type uses ‘activate-firmware’ directly
without importing (gnu build activation).

> > I have sent a patch which might partially address your issue:
> >
> > [PATCH v2 3/3] services: activation: Continue on exceptions.
> > https://issues.guix.gnu.org/73494#26
> >
> > It executes activation scripts by ‘invoke’, so they won't change the
> > environment.
> >
> > What I'm trying to achieve is to avoid blocking the activation process when one
> > script fails.
>
> Well, that patch would probably resolve my issues, correct.  What I am
> unsure about is whether ignoring errors is a good idea and what (if any)
> impact it will have on the rest of the deploy process.  Like, if the
> activation fails, would it not be better to rollback instead of push
> forward?  Dunno, I probably just do not know enough about this. :)

Makes sense, but too late at this stage since it's already switched to the new
generation before activation.

The current behavior is, one failed activation breaks the whole activation
process, without a clear indication.  As a result, all services depending on
activation may fail.  And I want to limit failed services to relevant ones.

Message #17 received at 76698 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: Hilton Chain <hako <at> ultrarare.space>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 76698 <at> debbugs.gnu.org
Subject: Re: bug#76698: Activations interfere with each other modules
Date: Sun, 09 Mar 2025 12:38:51 +0100

[Message part 1 (text/plain, inline)]

Hilton Chain <hako <at> ultrarare.space> writes:

> On Sat, 08 Mar 2025 05:29:51 +0800,
> Tomas Volf wrote:
>>
>> > but unfortunately (gnu build activation) and (guix build utils) are
>> > implicit dependencies.  This is hard to change at the moment.
>>
>> I do not think this is true.  When I look at the activation script for
>> my service, I do not see any additional code then my service has.  So
>> assuming it would be executed, instead of loaded, (guix build utils)
>> should not be in the environment.  Am I missing something?
>
> (gnu build activation) and (guix build utils) are added to the environment
> before loading activation scripts, existing activation services may depend on
> this.  For example firmware-service-type uses ‘activate-firmware’ directly
> without importing (gnu build activation).

I am not sure I follow.  When I have an activation service looking like:

--8<---------------cut here---------------start------------->8---
#~(mkdir "/x")
--8<---------------cut here---------------end--------------->8---

The generated file looks like the following:

--8<---------------cut here---------------start------------->8---
#!/gnu/store/ylwk2vn18dkzkj0nxq2h4vjzhz17bm7c-guile-3.0.9/bin/guile --no-auto-compile
!#
(mkdir "/x")
--8<---------------cut here---------------end--------------->8---

Wait... When I looked again I have now noticed that your patch actually
modifies the activation-script procedure to explicitly add the modules.
It did not do that before.  That is a shame.  Would it not be better to
just add the appropriate use-modules into the services that are missing
them?

I am not sure these two modules being available is documented anywhere,
so in-tree users can be fixed, and out-of-tree users are relying on
something they should not assume (so they should be fixed as well,
possibly with some deprecation period).

>
>> > I have sent a patch which might partially address your issue:
>> >
>> > [PATCH v2 3/3] services: activation: Continue on exceptions.
>> > https://issues.guix.gnu.org/73494#26
>> >
>> > It executes activation scripts by ‘invoke’, so they won't change the
>> > environment.
>> >
>> > What I'm trying to achieve is to avoid blocking the activation process when one
>> > script fails.
>>
>> Well, that patch would probably resolve my issues, correct.  What I am
>> unsure about is whether ignoring errors is a good idea and what (if any)
>> impact it will have on the rest of the deploy process.  Like, if the
>> activation fails, would it not be better to rollback instead of push
>> forward?  Dunno, I probably just do not know enough about this. :)
>
> Makes sense, but too late at this stage since it's already switched to the new
> generation before activation.
>
> The current behavior is, one failed activation breaks the whole activation
> process, without a clear indication.  As a result, all services depending on
> activation may fail.  And I want to limit failed services to relevant
> ones.

I see.  Makes sense, thanks for explanation.

Tomas

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 76698 <at> debbugs.gnu.org (full text, mbox):

From: Hilton Chain <hako <at> ultrarare.space>
To: Tomas Volf <~@wolfsden.cz>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 76698 <at> debbugs.gnu.org
Subject: Re: bug#76698: Activations interfere with each other modules
Date: Wed, 12 Mar 2025 16:45:01 +0800

On Sun, 09 Mar 2025 19:38:51 +0800,
Tomas Volf wrote:
>
> I am not sure these two modules being available is documented anywhere,
> so in-tree users can be fixed, and out-of-tree users are relying on
> something they should not assume (so they should be fixed as well,
> possibly with some deprecation period).

Sure, the period is at least one year I think, and a news entry is needed.  I'll
change Guix Home to use ‘invoke’ for activation as well.

Message #23 received at 76698 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Hilton Chain <hako <at> ultrarare.space>
Cc: Tomas Volf <~@wolfsden.cz>, 76698 <at> debbugs.gnu.org
Subject: Re: bug#76698: Activations interfere with each other modules
Date: Mon, 17 Mar 2025 20:55:53 +0100

Hello,

Hilton Chain <hako <at> ultrarare.space> skribis:

> Activation scripts are already ‘program-file’s, but unfortunately
> (gnu build activation) and (guix build utils) are implicit dependencies.  This
> is hard to change at the moment.
>
> I have sent a patch which might partially address your issue:
>
> [PATCH v2 3/3] services: activation: Continue on exceptions.
> https://issues.guix.gnu.org/73494#26
>
> It executes activation scripts by ‘invoke’, so they won't change the
> environment.

Activation snippets all run in the same Guile process so far, mostly out
of a concern to be more efficient than running each one in a separate
process as you suggest.

The downside is what Tomas found out: the load path is global to the
entire set of activation snippets.

Now, maybe we could have the best of both worlds: instead of
‘activation-script’ doing

  (for-each primitive-load '#$actions)

it could do something like:

  (for-each (lambda (action)
              (guard …   ;like Hilton suggests
                (save-module-excursion
                  (lambda ()
                    (set-current-module (resolve-module '(guile-user)))
                    (primitive-load action)))))
            '#$actions)

Thoughts?

Ludo’.

Message #26 received at 76698 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Hilton Chain <hako <at> ultrarare.space>, 76698 <at> debbugs.gnu.org
Subject: Re: bug#76698: Activations interfere with each other modules
Date: Mon, 17 Mar 2025 22:36:37 +0100

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hello,
>
> Hilton Chain <hako <at> ultrarare.space> skribis:
>
>> Activation scripts are already ‘program-file’s, but unfortunately
>> (gnu build activation) and (guix build utils) are implicit dependencies.  This
>> is hard to change at the moment.
>>
>> I have sent a patch which might partially address your issue:
>>
>> [PATCH v2 3/3] services: activation: Continue on exceptions.
>> https://issues.guix.gnu.org/73494#26
>>
>> It executes activation scripts by ‘invoke’, so they won't change the
>> environment.
>
> Activation snippets all run in the same Guile process so far, mostly out
> of a concern to be more efficient than running each one in a separate
> process as you suggest.
>
> The downside is what Tomas found out: the load path is global to the
> entire set of activation snippets.
>
> Now, maybe we could have the best of both worlds: instead of
> ‘activation-script’ doing
>
>   (for-each primitive-load '#$actions)
>
> it could do something like:
>
>   (for-each (lambda (action)
>               (guard …   ;like Hilton suggests
>                 (save-module-excursion
>                   (lambda ()
>                     (set-current-module (resolve-module '(guile-user)))
>                     (primitive-load action)))))
>             '#$actions)
>
> Thoughts?

+1 for the approach, for implementation I think you need
(make-fresh-user-module) instead of (resolve-module '(guile-user)).

>
> Ludo’.

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[signature.asc (application/pgp-signature, inline)]

Message #29 received at 76698 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Hilton Chain <hako <at> ultrarare.space>, Tomas Volf <~@wolfsden.cz>,
 76698 <at> debbugs.gnu.org
Subject: Re: bug#76698: Activations interfere with each other modules
Date: Wed, 02 Apr 2025 14:41:16 +0900

Hi,

[...]

>> Activation snippets all run in the same Guile process so far, mostly out
>> of a concern to be more efficient than running each one in a separate
>> process as you suggest.
>>
>> The downside is what Tomas found out: the load path is global to the
>> entire set of activation snippets.
>>
>> Now, maybe we could have the best of both worlds: instead of
>> ‘activation-script’ doing
>>
>>   (for-each primitive-load '#$actions)
>>
>> it could do something like:
>>
>>   (for-each (lambda (action)
>>               (guard …   ;like Hilton suggests
>>                 (save-module-excursion
>>                   (lambda ()
>>                     (set-current-module (resolve-module '(guile-user)))
>>                     (primitive-load action)))))
>>             '#$actions)
>>
>> Thoughts?

Sounds like a nice improvement over what we currently have, though
without having measured the impact of using multiple processes perhaps
it is premature to optimize to use a single process (guile has
a notoriously fast startup time so it may not matter much).

Still, +1 on this improvement from me.

-- 
Thanks,
Maxim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.