Subject: Re: bug#78355: guix-ownership inconsistent state

From: Ben Sturmfels <[email-masked]>
Date: Fri, 30 May 2025 19:30:21 +1000

(CCing the bug tracker)

Hi Ruther,

Thanks for all your work and help on this issue! Sorry for the 
delay - I
needed to find a quiet time to reconfigure and watch what happens 
more
carefully.

Rutherther <[email-masked]> writes:

> I am CCing you to get more information about the inconsistent 
> ownership,
> if you could help with that.
>
> The most important questions are probably:
> 0. Are you sure the service actually ran after you reconfigured 
> to root?
>   It should definitely run after reboot, not sure if after 
>   reconfigure
>   as the service already exists, it's just modified

To be honest, I had no understanding of how the change was taking 
place. I
read about it briefly in the `guix pull --news --details` and may 
have
briefly referred to the manual - I can't recall exactly. Possibly 
naive of
me.

I'll give it another shot now. I've set "privileged? #f" again and 
run
"system reconfigure". I see that it prints:

   ...
   building
   /gnu/store/yyn762lwzra0nqrrwhgbwwlz2k0qyn0h-upgrade-shepherd-services.scm.drv...
   shepherd: Starting service host-name...
   shepherd: Service host-name started.
   shepherd: Service host-name running with value "Marseille".
   shepherd: Service host-name has been started.
   shepherd: Starting service user-homes...
   shepherd: Service user-homes has been started.
   shepherd: Starting service sysctl...
   shepherd: Service sysctl has been started.

Then appears to hang, though I now realise that it's busy updating
ownership. Unfortunately there is no feedback to the user or 
advice not to
interrupt the process. I watched the ownership in /gnu/store be
progressively updated to "guix-daemon".

It took about 5 mins, then quickly printed:

   shepherd: Service user-homes has been started.
   shepherd: Starting service guix-ownership...
   shepherd: Changing to unprivileged guix-daemon.
   shepherd: Service guix-ownership has been started.
   shepherd: Starting service x11-socket-directory...
   shepherd: Service x11-socket-directory has been started.
   shepherd: Service user-homes has been started.
   shepherd: Service tor is currently disabled.
   shepherd: Service user-homes has been started.
   ...

The message "Changing to unprivileged guix-daemon" needs to be 
visible
before the work is done though. Is it possible that stdout is 
being
buffered?

> 1. Do you think you could've killed the service when it was 
> running
> after you reconfigured back to privileged daemon, ie. by 
> rebooting when
> it was running?

Absolutely I could have killed it - to me it just looks like 
reconfigure
had locked up for some reason. I'd probably forgotten that I even 
set the
"privileged?" option, or didn't consider the pause might be 
related.

So yes, user error on my part, but reflecting on it, the mental 
model I
have of Guix is that it's immune to issues such as power outages 
during
upgrades. I realise now that this is a complex migration, so I 
appreciate
that there's some state to manage and it takes time.

Possibly the uprivileged transition might need to be either:

a. more obvious to the user, eg. red text and interactive: "The 
migration
to the unprivileged daemon needs to update a very large number of 
files and
must not be interrupted. Do not proceed on battery power. Proceed? 
(y/N)"

b. robust to being interrupted part way, such as starting the 
daemon as
privileged unless the transition process ran successfully. Maybe 
that's not
technically feasible though - just a thought.

> 2. Do you know what folders had wrong owners?
>   - Was everything under /var/guix fully owned by 971?
>   - Was everything under /etc/guix fully owned by 971?
>   - Was everything under store fully owned by 971?

Not everything, just some. I'm not 100% sure it was 971, but 
something like
that - the ID of a user that didn't exist on the system 
(presumably because
I rolled back/reconfigured to "privileged? #t" again).

Now that it's run successfully, all of /gnu/store and /etc/guix 
are
guix-daemon:guix-daemon. All of /var/guix is 
guix-daemon:guix-daemon except
for files /var/guix/profiles and /var/guix/userpool, which are 
root:root.

I haven't rebooted yet, but if you don't hear back from me, assume 
that
worked fine.

Thanks again for all your work!

Ben