Subject: bug#77407: shepherd hangs at "herd restart nginx"

From: Ludovic Courtès <[email-masked]>
Date: Tue, 01 Apr 2025 14:19:32 +0200

Hello,

Andreas Enge <[email-masked]> skribis:

> Today I noticed that the QA webpage was 502 on bayfront.
> I restarted qa-frontpage and bffe without problem.
> When trying to do "herd restart nginx", the command did not show any
> output and did not come back. Moreover, "herd status" also hang without
> any output.

FWIW, I tested ‘bare-bones.tmpl’ with just nginx added; running:

  while herd restart nginx; do : ; done

in that VM didn’t trigger the hang.

So I’m a bit at loss as to why this fails on bayfront.  If someone has
ideas (and time!) on how to get a config closer to that of bayfront that
we could test in a VM, that’d be great.  Otherwise, we’ll have to debug
in production…

> I could still ssh into the machine and then, thanks to Ludovic, reboot
> with the magic command
>   guix shell guile shepherd -- guile -c '(use-modules (shepherd system)) (sync) (reboot-kexec)'

To be clear: this should be a last resort.  :-)

Thanks,
Ludo’.