Subject: Re: bug#77634: [shepherd] Test failures on GNU/Hurd

From: Ludovic Courtès <[email-masked]>
Date: Mon, 14 Apr 2025 15:27:58 +0200

Hi yelninei,

[email-masked] writes:

>>> FAIL tests/logging-failure.sh (exit status: 124)

[...]

> I was able to reproduce this by extracting the shepherd conf from the
> test and and attempting to run the test manually on the childhurd with
> the cross compiled shepherd (so you should not need to have a native
> toolchain available)
>
>
> Might be the attempt to write to /proc.
>
> Starting the log-directory-not-writable service causes the entire shepherd to become completely unresponsive causing the timeout failure.
>
> A similar behavior occurs with
>
> (call-with-output-file "/proc/1/something"
>   (lambda (port)
>     (display "Hello" port)))
>
> which hangs my guile completely.
>
> When I change the log file to a more normal unwritable path the test passes.

Ah, interesting.  So /something instead of /proc/1/something would work,
right?  (I picked /proc/1/something because ‘guix shell -C’ currently
gives a writable root file system… Going to be fixed with
<https://issues.guix.gnu.org/77638>.)

You could also report the procfs issue to bug-hurd I guess.  :-)

>>> FAIL tests/services/system-log.sh (exit status: 124)
>>>
>>
>> This one also hanged more than 3m it seems.
>>
> The shepherd syslog seems to be extremely slow:
>
> I extracted the logger.scm and conf.scm from the test, removed the $
> from the shell variables variables, started shepherd, echoed the test
> msg into  the kmsg file and then tried to start the shepherd syslog.
>
> It took multiple minutes to see the "Service system-log running with
> value" in the log and then another couple of minutes for "herd start
> syslogd" to return. Afterwards trying to query the syslog status (or
> trying to interact with shepherd in any other way) takes forever to
> complete while syslogd is running.
>
> I did this with the 1.0.4rc1 shepherd to work around the blocking
> socket problem with guix-daemon (#77610, it still fails on the first
> connection now but idk if this is a problem with shepherd or the
> guix-daemon)
>
> With the 1.0.3 shepherd syslogd seems to be a lot quicker to start
> initally ( see the log from the first message) but still extremely
> slow to interact with afterwards.

Weird.  Could you bisect between 1.0.3 and HEAD to try and find the
source of slowness?

I wonder if f1a82845eaf8851af9a811e5a1d185b68b1c363f might explain it,
but that’s pre-1.0.3.

Alternatively, you can try running ‘shepherd’ under ‘rpctrace’ for the
syslog slowness issue, so we get an idea of what it’s doing.

Thanks a lot for helping out!

Ludo’.