GNU bug report logs - #55936
dockerd fails to start on boot

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#55936: closed (dockerd fails to start on boot)
Date: Thu, 14 Jul 2022 01:42:02 +0000

[Message part 1 (text/plain, inline)]

Your message dated Wed, 13 Jul 2022 21:40:53 -0400
with message-id <877d4g1nfe.fsf <at> gmail.com>
and subject line Re: bug#55936: dockerd fails to start on boot
has caused the debbugs.gnu.org bug report #55936,
regarding dockerd fails to start on boot
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
55936: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=55936
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Luciano Laratelli <luciano <at> laratel.li>
To: bug-guix <at> gnu.org
Subject: dockerd fails to start on boot
Date: Sun, 12 Jun 2022 18:56:38 -0400

[Message part 3 (text/plain, inline)]

Hi, hope you are doing well.

I’m running Guix System and am seeing that `dockerd' fails to start on boot due to not being able to find `containerd':

$ sudo tail /var/log/docker.log
2022-06-12 18:25:29 time=“2022-06-12T18:25:29.969005384-04:00” level=warning msg=“Error (Unable to complete atomic operation, key modified) deleting object [endpoint 062e6856b7776daf35f1d570dc7e055d3c0f3eefc0f58c5e279eba20035c8e9e eb10082295c7a53d882e36d93a8b5eb20e980a5950c4a67fa03444274448b232], retrying….”
2022-06-12 18:25:30 time=“2022-06-12T18:25:30.068910364-04:00” level=info msg=“Removing stale sandbox e35667a7ef1441bced213cf035efc9d6c71a0dce7f8941e3fbb63f5a27265bca (91314e5594f72585f9df121ba16cc8d67c4e1fcb91fc3c7b9b0660aed1b3054a)”
2022-06-12 18:25:30 time=“2022-06-12T18:25:30.080685302-04:00” level=warning msg=“Error (Unable to complete atomic operation, key modified) deleting object [endpoint 062e6856b7776daf35f1d570dc7e055d3c0f3eefc0f58c5e279eba20035c8e9e 825f4a6f68b1b81b24b2edc0b382deca116e72a75e6207036f24e18ba6434c81], retrying….”
2022-06-12 18:25:30 time=“2022-06-12T18:25:30.143624227-04:00” level=info msg=“Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option –bip can be used to set a preferred IP address”
2022-06-12 18:25:31 time=“2022-06-12T18:25:31.400700443-04:00” level=info msg=“Loading containers: done.”
2022-06-12 18:25:31 time=“2022-06-12T18:25:31.689183684-04:00” level=info msg=“Docker daemon” commit=v19.03.15 graphdriver(s)=overlay2 version=19.03.15-ce
2022-06-12 18:25:31 time=“2022-06-12T18:25:31.691171101-04:00” level=info msg=“Daemon has completed initialization”
2022-06-12 18:25:31 time=“2022-06-12T18:25:31.961049886-04:00” level=info msg=“API listen on /var/run/docker.sock”
2022-06-12 18:43:43 time=“2022-06-12T18:43:43.503118343-04:00” level=info msg=“Starting up”
2022-06-12 18:43:43 failed to start containerd: exec: “containerd”: executable file not found in $PATH

$ sudo docker ps
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

$ sudo herd status dockerd
Status of dockerd:
  It is stopped.
  It is enabled.
  Provides (dockerd).
  Requires (containerd dbus-system elogind file-system-/sys/fs/cgroup/blkio file-system-/sys/fs/cgroup/cpu file-system-/sys/fs/cgroup/cpuset file-system-/sys/fs/cgroup/devices file-system-/sys/fs/cgroup/memory file-system-/sys/fs/cgroup/pids networking udev).
  Conflicts with ().
  Will be respawned.


I can start it myself, though:


$ sudo herd start dockerd
Service dockerd has been started.


I found a [past issue] on this list with someone experiencing a similar problem,
but adding `kmod' as suggested did not resolve the issue on my end..

Here’s my config.scm - I’ve redcated my host name and file-system/swap-devices
blocks, but everything else is verbatim what the machine is running.

(use-modules (gnu)
             (nongnu packages linux)
             (nongnu system linux-initrd))

(use-service-modules desktop networking ssh xorg docker)

(operating-system
 (kernel linux)
 (firmware (list linux-firmware))
 (initrd microcode-initrd)
 (locale “en_US.utf8”)
 (timezone “America/New_York”)
 (keyboard-layout (keyboard-layout “us”))

(users (cons* (user-account
               (name “luciano”)
               (comment “Luciano Laratelli”)
               (group “users”)
               (home-directory “/home/luciano”)
               (supplementary-groups
                ’(“wheel” “netdev” “audio” “video”)))
              %base-user-accounts))
(packages
 (append
  (list (specification->package “st”)
        (specification->package “nss-certs”)
        (specification->package “docker”)
        (specification->package “docker-compose”)
        (specification->package “containerd”)
        (specification->package “kmod”)
        (specification->package “vim”)
        (specification->package “emacs-no-x-toolkit”)
        (specification->package “parted”))
  %base-packages))
(services
 (append
  (list (service openssh-service-type
                 (openssh-configuration
                  (password-authentication? #f)))

        (service network-manager-service-type)
        (service wpa-supplicant-service-type)
        (service docker-service-type)
        (elogind-service)
        (set-xorg-configuration
         (xorg-configuration
          (keyboard-layout keyboard-layout))))
  (modify-services
   %base-services
   (guix-service-type
    config =>
    (guix-configuration
     (inherit config)
     (substitute-urls
      (append
       (list "<https://substitutes.nonguix.org>”)
       %default-substitute-urls))
     (authorized-keys
      (append
       (list (plain-file “signing-key.pub” “
                 (public-key
                   (ecc
                     (curve Ed25519)
                     (q #C1FD53E5D4CE971933EC50C9F307AE2171A2D3B52C804642A7A35F84F3A4EA98#)
                     )
                   )”))
       %default-authorized-guix-keys)))))))
(bootloader
 (bootloader-configuration
  (bootloader grub-efi-bootloader)
  (targets ’(“/boot/efi”))
  (keyboard-layout keyboard-layout))))


I’m not sure how to debug this issue any further and would appreciate some pointers there.

Thank you,

Luciano


[past issue] <https://issues.guix.gnu.org/issue/34333>

[Message part 4 (text/html, inline)]

[Message part 5 (message/rfc822, inline)]

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Oleg Pykhalov <go.wigust <at> gmail.com>
Cc: 55936-done <at> debbugs.gnu.org
Subject: Re: bug#55936: dockerd fails to start on boot
Date: Wed, 13 Jul 2022 21:40:53 -0400

Hi Oleg,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:

> Hi,
>
> Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:
>
>> Hi Oleg,
>>
>> Oleg Pykhalov <go.wigust <at> gmail.com> writes:
>>
>>> Fixes <https://issues.guix.gnu.org/38432>.
>>>
>>> * gnu/packages/patches/containerd-create-pid-file.patch: New file.
>>> * gnu/local.mk (dist_patch_DATA): Add this.
>>> * gnu/packages/docker.scm (containerd)[source]: Add this patch.
>>> * gnu/services/docker.scm
>>> (containerd-shepherd-service): Add #:pid-file and #:pid-file-timeout.
>>> * gnu/services/docker.scm (docker-shepherd-service): Add --containerd flag.
>>
>> Thanks for this, it looks promising!

[...]

> All that to say that I shall be reviewing your patches shortly :-).

Now done; it all looks good to me!  I've run the docker system test, and
installed it on my machine, rebooted, confirmed it was up, restarted
containerd a couple times and checked the PID content matched its actual
PID, and it seems to behave as expected!

Pushed as b33e1a183f6756514e6b6a3b84054a232dbddad4.

Thank you!

Maxim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.