Subject: dockerd fails to start on boot
From: Luciano Laratelli <[email-masked]>
To: [email-masked]
Date: Sun, 12 Jun 2022 18:56:38 -0400
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi, hope you are doing well.

I’m running Guix System and am seeing that `dockerd' fails to start on boot due to not being able to find `containerd':

$ sudo tail /var/log/docker.log
2022-06-12 18:25:29 time=“2022-06-12T18:25:29.969005384-04:00” level=warning msg=“Error (Unable to complete atomic operation, key modified) deleting object [endpoint 062e6856b7776daf35f1d570dc7e055d3c0f3eefc0f58c5e279eba20035c8e9e eb10082295c7a53d882e36d93a8b5eb20e980a5950c4a67fa03444274448b232], retrying….”
2022-06-12 18:25:30 time=“2022-06-12T18:25:30.068910364-04:00” level=info msg=“Removing stale sandbox e35667a7ef1441bced213cf035efc9d6c71a0dce7f8941e3fbb63f5a27265bca (91314e5594f72585f9df121ba16cc8d67c4e1fcb91fc3c7b9b0660aed1b3054a)”
2022-06-12 18:25:30 time=“2022-06-12T18:25:30.080685302-04:00” level=warning msg=“Error (Unable to complete atomic operation, key modified) deleting object [endpoint 062e6856b7776daf35f1d570dc7e055d3c0f3eefc0f58c5e279eba20035c8e9e 825f4a6f68b1b81b24b2edc0b382deca116e72a75e6207036f24e18ba6434c81], retrying….”
2022-06-12 18:25:30 time=“2022-06-12T18:25:30.143624227-04:00” level=info msg=“Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option –bip can be used to set a preferred IP address”
2022-06-12 18:25:31 time=“2022-06-12T18:25:31.400700443-04:00” level=info msg=“Loading containers: done.”
2022-06-12 18:25:31 time=“2022-06-12T18:25:31.689183684-04:00” level=info msg=“Docker daemon” commit=v19.03.15 graphdriver(s)=overlay2 version=19.03.15-ce
2022-06-12 18:25:31 time=“2022-06-12T18:25:31.691171101-04:00” level=info msg=“Daemon has completed initialization”
2022-06-12 18:25:31 time=“2022-06-12T18:25:31.961049886-04:00” level=info msg=“API listen on /var/run/docker.sock”
2022-06-12 18:43:43 time=“2022-06-12T18:43:43.503118343-04:00” level=info msg=“Starting up”
2022-06-12 18:43:43 failed to start containerd: exec: “containerd”: executable file not found in $PATH

$ sudo docker ps
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

$ sudo herd status dockerd
Status of dockerd:
  It is stopped.
  It is enabled.
  Provides (dockerd).
  Requires (containerd dbus-system elogind file-system-/sys/fs/cgroup/blkio file-system-/sys/fs/cgroup/cpu file-system-/sys/fs/cgroup/cpuset file-system-/sys/fs/cgroup/devices file-system-/sys/fs/cgroup/memory file-system-/sys/fs/cgroup/pids networking udev).
  Conflicts with ().
  Will be respawned.

I can start it myself, though:

$ sudo herd start dockerd
Service dockerd has been started.

I found a [past issue] on this list with someone experiencing a similar problem,
but adding `kmod' as suggested did not resolve the issue on my end..

Here’s my config.scm - I’ve redcated my host name and file-system/swap-devices
blocks, but everything else is verbatim what the machine is running.

(use-modules (gnu)
             (nongnu packages linux)
             (nongnu system linux-initrd))

(use-service-modules desktop networking ssh xorg docker)

(operating-system
 (kernel linux)
 (firmware (list linux-firmware))
 (initrd microcode-initrd)
 (locale “en_US.utf8”)
 (timezone “America/New_York”)
 (keyboard-layout (keyboard-layout “us”))

(users (cons* (user-account
               (name “luciano”)
               (comment “Luciano Laratelli”)
               (group “users”)
               (home-directory “/home/luciano”)
               (supplementary-groups
                ’(“wheel” “netdev” “audio” “video”)))
              %base-user-accounts))
(packages
 (append
  (list (specification->package “st”)
        (specification->package “nss-certs”)
        (specification->package “docker”)
        (specification->package “docker-compose”)
        (specification->package “containerd”)
        (specification->package “kmod”)
        (specification->package “vim”)
        (specification->package “emacs-no-x-toolkit”)
        (specification->package “parted”))
  %base-packages))
(services
 (append
  (list (service openssh-service-type
                 (openssh-configuration
                  (password-authentication? #f)))

        (service network-manager-service-type)
        (service wpa-supplicant-service-type)
        (service docker-service-type)
        (elogind-service)
        (set-xorg-configuration
         (xorg-configuration
          (keyboard-layout keyboard-layout))))
  (modify-services
   %base-services
   (guix-service-type
    config =>
    (guix-configuration
     (inherit config)
     (substitute-urls
      (append
       (list "<https://substitutes.nonguix.org>”)
       %default-substitute-urls))
     (authorized-keys
      (append
       (list (plain-file “signing-key.pub” “
                 (public-key
                   (ecc
                     (curve Ed25519)
                     (q #C1FD53E5D4CE971933EC50C9F307AE2171A2D3B52C804642A7A35F84F3A4EA98#)
                     )
                   )”))
       %default-authorized-guix-keys)))))))
(bootloader
 (bootloader-configuration
  (bootloader grub-efi-bootloader)
  (targets ’(“/boot/efi”))
  (keyboard-layout keyboard-layout))))

I’m not sure how to debug this issue any further and would appreciate some pointers there.

Thank you,

Luciano

[past issue] <https://issues.guix.gnu.org/issue/34333>