Subject: Re: bug#62491: [berlin] certbot renewal appears to be broken

From: Ludovic Courtès <[email-masked]>
Date: Thu, 23 Nov 2023 09:46:56 +0100

Hi,

Giovanni Biscuolo <[email-masked]> skribis:

> Maxim Cournoyer <[email-masked]> writes:
>
> [...]
>
>>> AFAIU actually #56678 is (was?) caused by a duplicate certbot account:
>
> [...]
>
>>> The problem on berlin (#62491) is (was) due to a failed challenge:
>
> I'm almost sure those are different bugs and I'm almost sure the bugs
> are caused by _state_ (/etc/letsencrypt/[accounts|renewal])

Indeed, that’s part of the problem.

Another example: our cerbot service offers a ‘deploy-hook’, but the
/gnu/store/… file name of that hook gets recorded somewhere in
/etc/letsencrypt and thus becomes invalid once the hook has been GC’d or
the system has been reconfigured.

>> I don't think it was truly resolved.  The problem keeps coming and
>> someone (usually Ludovic) has to manually run some commands get it to
>> cooperate (IIUC).
>
> Bugs like this are very difficult to reproduce and to investigate if we
> wait the certs expiration and are forced to find a quick "workaround";
> we should force a renewal (via CLI) before the expiration date and share
> the logs to see what's happening.
>
> I'd like to help but I'm not a sysadmin on bayfront nor on berlin.
>
> I think this kind "statefulness issues" are affecting other users.

Yeah, I think anyone running a web server on Guix System gets hit by
this issue.  I’m not super knowledgeable about certbot either so I tend
to just hack around to get things to work, which is not great.

Ludo’.