GNU bug report logs - #79321
A pile of problems with unprivileged Guix daemon and 'guix gc'

Previous Next

Full log

View this message in rfc822 format

From: "Zack Weinberg" <zack <at> owlfolio.org>
To: help-guix <at> gnu.org, 79321 <at> debbugs.gnu.org
Subject: bug#79321: A pile of problems with unprivileged Guix daemon and 'guix gc'
Date: Tue, 26 Aug 2025 17:17:22 -0400

# guix gc
finding garbage collector roots...
cannot read potential root `/var/guix/gcroots/auto/idj3k6kjlqi7y8sc4c5xschqh8zkhfvb'
cannot read potential root `/var/guix/gcroots/auto/5si4fzk79j7v27rqaic4lc2qfpk42ilb'
cannot read potential root `/var/guix/gcroots/auto/825grbfhqdfav4g6827d2d3hb8hyhhzl'
cannot read potential root `/var/guix/gcroots/auto/6l77c1c97vij2gg3p95d9zi2k7l0yx29'
cannot read potential root `/var/guix/gcroots/auto/r2x8d211bfp2y3y6wvgp8740ram26ipv'
cannot read potential root `/var/guix/gcroots/auto/jbrxxz57056g8393kh9zyyj325lwq5c5'
guix gc: error: program `/gnu/store/6px1m9n904j8s4hyrmlds707sfnq52d9-guix-1.4.0-41.826e305/bin/guix' failed with exit code 1

So first off, these error messages fail to comply with the first law of
Unix error messages; they don't print strerror(errno), and they don't
name the actual system call that failed, so they don't tell me *why* the
GC roots can't be read.  But leave that aside for now...

# guix gc 2>&1 |
  sed -ne 's:^cannot read potential root `\([a-z0-9/]*\)'\''$:\1:p' >
  /tmp/bad-roots
# ls -l $(cat /tmp/bad-roots)
lrwxrwxrwx 1 guix-daemon guix-daemon 80 Aug 10 01:41 /var/guix/gcroots/auto/5si4fzk79j7v27rqaic4lc2qfpk42ilb -> /root/.cache/guix/inferiors/bpo6zmuuzeya74vbpqn2innq7vw4xzxn7azgjarsmg756jdrsika
lrwxrwxrwx 1 guix-daemon guix-daemon 79 Mar 16 22:20 /var/guix/gcroots/auto/6l77c1c97vij2gg3p95d9zi2k7l0yx29 -> /root/.cache/guix/profiles/simr3ylizyyss24c25azsqfl4vjtw2t4ywvgpbh3iinbrsljgfea
lrwxrwxrwx 1 guix-daemon guix-daemon 80 Jul 27 02:02 /var/guix/gcroots/auto/825grbfhqdfav4g6827d2d3hb8hyhhzl -> /root/.cache/guix/inferiors/zy7a627k6aubd32iun2ibyoy4ulbj4xas55yaibwaayctx6qehta
lrwxrwxrwx 1 guix-daemon guix-daemon 80 Jul 13 01:41 /var/guix/gcroots/auto/idj3k6kjlqi7y8sc4c5xschqh8zkhfvb -> /root/.cache/guix/inferiors/72tvmmz43muzwd4lml3xsfdxw55idd742433w4kylm7yyyohed6a
lrwxrwxrwx 1 guix-daemon guix-daemon 80 Aug  3 01:39 /var/guix/gcroots/auto/jbrxxz57056g8393kh9zyyj325lwq5c5 -> /root/.cache/guix/inferiors/qgxsppsml7olednljz273sdygm5zsxjrrpey2q7ysh5on6evneza
lrwxrwxrwx 1 guix-daemon guix-daemon 80 Jul 20 01:41 /var/guix/gcroots/auto/r2x8d211bfp2y3y6wvgp8740ram26ipv -> /root/.cache/guix/inferiors/whqagcgua6af2zpw3xpaiiifny6pvevcpque3kstsu74ufx6rrda

# ls -ld /root /root/.cache /root/.cache/guix /root/.cache/guix/{inferiors,profiles}
drwx------ 5 root root 4096 Aug 26 20:46 /root/
drwxr-xr-x 4 root root 4096 Jul 22  2024 /root/.cache/
drwxr-xr-x 6 root root 4096 Mar 16 22:19 /root/.cache/guix/
drwxr-xr-x 2 root root 4096 Aug 10 01:41 /root/.cache/guix/inferiors/
drwxr-xr-x 2 root root 4096 Mar 16 22:21 /root/.cache/guix/profiles/

After seeing this I suspected the problem might be that the *Guix daemon*,
which is running unprivileged, cannot access these files.  And indeed, if
I do `chmod 711 /root`, then `guix gc` stops printing the "cannot read
potential root" messages.  But it still doesn't _work_:

# guix gc
finding garbage collector roots...
guix gc: error: program `/gnu/store/6px1m9n904j8s4hyrmlds707sfnq52d9-guix-1.4.0-41.826e305/bin/guix' failed with exit code 1

So that's _really_ bad UX, but again, not the immediate problem.  Since I
do now know that it's the daemon that's having problems, I check the logs:

# tail -3 /var/log/guix-daemon.log
2025-08-26 20:56:21 accepted connection from pid 172, user root
2025-08-26 20:56:21 accepted connection from pid 176, user guix-daemon
2025-08-26 20:56:21 guix gc: error: creating directory `/var/guix/profiles/per-user/guix-daemon': Permission denied

Well, that's suggestive...

# ls -la /var/guix/profiles/per-user
total 28
drwxr-xr-x 7 root         root         4096 Apr 25 20:03 ./
drwxr-xr-x 3 root         root         4096 Aug 26 20:25 ../
drwxr-xr-x 2 root         root         4096 Aug 26 20:25 root/
drwxr-xr-x 2 user1        user1        4096 Apr 25 20:03 user1/
drwxr-xr-x 2 user2        user2        4096 Apr 25 20:03 user2/
drwxr-xr-x 2 user3        user3        4096 Apr 25 20:03 user3/

(actual user names redacted)

# mkdir /var/guix/profiles/per-user/guix-daemon
# chown guix-daemon:guix-daemon /var/guix/profiles/per-user/guix-daemon
# guix gc
finding garbage collector roots...
deleting garbage...
[7 MiB] deleting '/gnu/store/r993z4wdyqqwzxlif1hvqzp6cqhqr2bw-rustc-1.76.0-src.tar.zst.drv'
[7 MiB] deleting '/gnu/store/afnyx8a8qj4wlhywv0zsf57lmk8yskzc-rustc-1.76.0-src.tar.gz.drv'
...
[38344 MiB] deleting '/gnu/store/yd1hpyjjmzmq5qmlv6q2ycqlymsma9rh-freeglut-3.4.0-builder'
[38344 MiB] deleting '/gnu/store/3z0np2ad898193wws74k54rzppr356cv-ipxe-qemu-1.21.1-3.24db39f-builder'
deleting `/gnu/store/trash'
guix gc: error: making `var/empty' writable: Operation not permitted

Well yeah! /var/empty is supposed to be empty at all times!  Is this
not how it's supposed to be?

# ls -ld /var/empty
dr-xr-xr-x 2 root root 4096 Jul 22  2024 /var/empty/

But if I set it to be world-writable I still get the same error.  In
fact, even if I make it *owned by guix-daemon*, I still get the same
error!

And now I'm stuck.  guix gc moved all the trash to /gnu/store/trash,
but it didn't actually delete any of it.  There's 11G in there, and
I can't delete it by hand because the store is mounted read-only
and I don't know how to temporarily override that for this kind of
manual repair job.  Any advice would be most appreciated.

zw

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.