GNU bug report logs - #75810
[PATCH 0/6] Rootless guix-daemon

Previous Next

Full log

Message #32 received at 75810 <at> debbugs.gnu.org (full text, mbox):

From: Reepca Russelstein <reepca <at> russelstein.xyz>
To: 75810 <at> debbugs.gnu.org
Cc: ludo <at> gnu.org
Subject: Re: [bug#75810] [PATCH 0/6] Rootless guix-daemon
Date: Sat, 25 Jan 2025 18:39:04 -0600

[Message part 1 (text/plain, inline)]

> Hello Guix!
> 
> That guix-daemon runs as root is not confidence-inspiring for many.
> Initially, the main reason for running it as root was, in the absence
> of user namespaces, the fact that builders would be started under one
> of the build user accounts, which only root can do.  Now that
> unprivileged user namespaces are almost ubiquitous (even on HPC
> clusters), this is no longer a good reason.

Without the build users, we're relying entirely on kernel-specific
sandboxing mechanisms to protect the system from rogue builders.  It's
probably (?) not impossible to make it work, but, as with every time
security mechanisms are changed, it does require some very careful
thought.

For example, consider the following:

--8<---------------cut here---------------start------------->8---
(use-modules (guix)
             (gnu)
             (guix build-system trivial))

(define-public sneakysneaky
  (package
    (name "sneakysneaky")
    (version "0")
    (source #f)
    (build-system trivial-build-system)
    (arguments
     (list
      #:builder
      #~(let ((hello (string-append #$(this-package-input "hello")
                                    "/bin/hello")))
          (chmod (dirname hello) #o775)
          (chmod hello #o775)
          (delete-file hello)
          (call-with-output-file hello
            (lambda (port)
              (chmod port #o775)
              (display "#!/bin/sh
echo \"GOOOOOD BYYEEEEEE\""
                       port)))
          (mkdir #$output))))
    (inputs (list (@ (gnu packages base) hello)))
    (home-page "")
    (synopsis "")
    (description "")
    (license #f)))

sneakysneaky
--8<---------------cut here---------------end--------------->8---


If we save this as /tmp/mal-test.scm on a debian VM with these patches
applied, we can see the following:


--8<---------------cut here---------------start------------->8---
user <at> debian:~$ guix build --no-grafts hello
/gnu/store/8bjy9g0cssjrw9ljz2r8ww1sma95isfj-hello-2.12.1
user <at> debian:~$ /gnu/store/8bjy9g0cssjrw9ljz2r8ww1sma95isfj-hello-2.12.1/bin/hello
Hello, world!
user <at> debian:~$ guix build --no-grafts -f /tmp/mal-test.scm
substitute: looking for substitutes on 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: looking for substitutes on 'https://ci.guix.gnu.org'... 100.0%
The following derivation will be built:
  /gnu/store/p15g92hfs7254pqfa3kss63dprw2clis-sneakysneaky-0.drv
building /gnu/store/p15g92hfs7254pqfa3kss63dprw2clis-sneakysneaky-0.drv...
successfully built /gnu/store/p15g92hfs7254pqfa3kss63dprw2clis-sneakysneaky-0.drv
/gnu/store/y1jzqg30cgkydl8kymjsh99zqgzh1yj1-sneakysneaky-0
user <at> debian:~$ /gnu/store/8bjy9g0cssjrw9ljz2r8ww1sma95isfj-hello-2.12.1/bin/hello
GOOOOOD BYYEEEEEE
user <at> debian:~$ 
--8<---------------cut here---------------end--------------->8---

This happens because the daemon bind-mounts store items into the
container, so it's the same underlying inode both inside and out of the
container.  The build runs as the same user as the store owner, so
there's nothing stopping it from freely modifying its input store items
and any of their transitive references.

I suppose we could try to perform these bind-mounts with the MS_RDONLY
flag, but we would need some way to ensure that the builder can't just
remount them read-write (I haven't yet looked into how to do this).  The
nuclear option, of course, would be to simply do a full copy of the
store items in question instead of a bind-mount.

> This patch changes guix-daemon so it can run as an unprivileged
> user, using unprivileged user namespaces to still support isolated
> builds.  There’s a couple of cases where root is/was still necessary:
> 
>   1. To create /var/guix/profiles/per-user/$USER and chown it
>      as $USER (see CVE-2019-18192).
> 
>   2. To chown /tmp/guix-build-* when using ‘--keep-failed’.
> 
> Both can be addressed by giving CAP_CHOWN to guix-daemon, and this is
> what this patch series does on distros using systemd.  (For some
> reason CAP_CHOWN had to be added to the set of “ambient capabilities”,
> which are inherited by child processes; this is why there’s a patch
> to drop ambient capabilities in build processes.)
> 
> On Guix System (not implemented here), we could address (1) by
> creating /var/guix/profiles/per-user/$USER upfront for all the
> user accounts.  We could leave (2) unaddressed (so failed build
> directories would be owned by guix-daemon:guix-daemon) or we’d
> have to pass CAP_CHOWN as well.

The automatic chown of /tmp/guix-build-* has always been a litte strange
considering that multiple users could attempt the same doomed-to-failure
derivation build at the same time, and it comes down to a race to see
who gets the build (and therefore the build directory).  This does raise
the question, though, of how these failed build directories would get
deleted, aside from rebooting the system.  Perhaps the garbage collector
could be modified to get rid of them?  In which case it may be best to
make it so that the failed build directories are automatically added to
the temp roots for a client, and the client takes care to copy the
failed build directory to a fresh path owned by the current user?  Or we
could make it so that the failed build directory gets sent over the wire
in nar form to the client.  Not sure what the best approach there is.

> There’s another issue: /gnu/store can no longer be remounted
> read-only (like we do on Guix System and on systemd with
> ‘gnu-store.mount’) because then unprivileged guix-daemon would
> be unable to remount it read-write (or at least I couldn’t find
> a way to do that).  Thus ‘guix-install.sh’ no longer installs
> ‘gnu-store.mount’ in that case.  It’s a bit sad to lose that
> so if anyone can think of a way to achieve it, that’d be great.

We currently remount /gnu/store read-write at LocalStore-creation-time,
which happens in the newly-forked guix-daemon process at the start of a
connection.  I don't think there's any particularly elevated risk from
instead doing that before the per-connection process is forked.  There
are a number of ways we could do this: we could make it the
responsibility of the init system to create the mount namespace and do
the remounting, or we could have guix-daemon do it immediately on
startup and subsequently switch its uid and gid to
guix-daemon:guix-daemon.  These lack the slick appeal of "see, you never
have to give it root, and you can prove it just by looking at the
service file", but realistically should be just as secure.  It may be
useful to provide a small wrapper around guix-daemon that does the
remount and privilege-dropping, to more succinctly express this to
anybody wishing to see for themselves.

> The next step (in another patch series) would be Guix System support
> with automatic transition (essentially “chown -R
> guix-daemon:guix-daemon /gnu/store”).
> 
> Thoughts?

There are, effectively, 3 platforms that guix currently supports: posix,
linux, and hurd.  Posix doesn't get much attention since we don't chase
Mac like nix does, but there do exist configurations where we use
neither linux-specific nor hurd-specific functionality.  Additionally, a
given guix-daemon may be either privileged or unprivileged.  Thus, we
end up with a total of 6 configurations.  Except there is now also the
question of whether less-than-fully-trusted users are allowed access to
the guix-daemon's socket.  Now we're in theory at 12 configurations.
Which of these configurations to use is, in some circumstances, going to
come down to judgement calls.  For example, one user may not care at all
about the risk of malicious builders (e.g. "the admins on this shared
system all use the debian tools anyway"), but be quite concerned about
the possibility of a root-granting exploit being found in guix-daemon.
Another (like myself and other Guix System users) may consider a risk to
the store to be the same as a risk to the entire system itself.  In
theory splitting between "privileged-with-root" and
"privileged-with-capabilities" will only increase the number of
configurations further.

Personally, I think that if a guix-daemon can use privilege separation
users, it would probably be a good idea to.  We're certainly going to
need to support them on non-linux systems either way.  Could it be
possible to have guix-install.sh modify /etc/sudoers on systems that use
it to allow the guix-daemon user to run processes under guix builder
users?  I am currently less worried about arbitrary code execution
vulnerabilities being found in the daemon than about the possibility of
malicious builders (but it is possible I am underexposed to the ways
those can happen in C++).

Additionally, CAP_CHOWN, while not having a direct path to privilege
escalation due to setuid and setgid bits being reset when chown is
called, can nevertheless be easily leveraged into privilege escalation
in most real-world situations where arbitrary code execution is
possible, so switching to using just that capability would realistically
only add defense in less-than-arbitrary-code-execution scenarios.

Using unprivileged user namespaces would, however, be an excellent
addition for unprivileged daemons, like the one started by test-env, or
one started by an unprivileged user on a system without a whole-system
guix installation.

Hope that helps.

- reepca

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.