GNU bug report logs - #52533
guix deploy breaks SSH access with a PAM error

Previous Next

Full log

Message #25 received at 52533 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 52533 <at> debbugs.gnu.org
Subject: Re: bug#52533: guix deploy breaks SSH access with a PAM error
Date: Thu, 13 Jan 2022 11:45:08 -0500

Hello,

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi,
>
> Mathieu Othacehe <othacehe <at> gnu.org> skribis:
>
>>> This sounds a lot like this:
>>>
>>>   https://issues.guix.gnu.org/32182#1
>>
>> I was just kicked out of my own server due to this PAM/SSH issue. It
>> happens quite frequently here. Time for a fix :).

Not a meaningful contribution to the discussion, but my workaround is to
disable PAM; as it is not enabled in OpenSSH by default, perhaps we
should also leave it off unless requested?  What are the advantages of
having it on?

> Note that ‘guix deploy’ now opens a single SSH session, starting from
> 7f20e59a13a6acc3331e04185b8f1ed2538dcd0a, which might help mitigate the
> problem.
>
>> Regarding the two potential solutions that you proposed in 2018, are
>> they still actual? If yes, I could maybe try to implement the second
>> suggestion: introducing service chain-loading.
>
> Service chain-loading was implemented in the Shepherd a few years ago.
> However, it doesn’t really help; consider these two scenario:
>
>   • You do ‘guix system reconfigure && herd restart term-tty1’.  In that
>     case, all is good: ‘term-tty1’, will run the new ‘mingetty’ process
>     (post-glibc upgrade, thanks to service chain-loading) and ‘login’
>     will happily load the .so files listed in /etc/pam.d/login (also
>     post-glibc upgrade).
>
>   • You run ‘guix system reconfigure’ but do not restart ‘term-tty1’,
>     ‘sshd’, and all the other services that depend on PAM: these
>     pre-glibc upgrade programs will try dlopening the post-glibc upgrade
>     PAM plugins, which will break.
>
> The crux of the problem rather is the global /etc/pam.d: it’s valid for
> pre-glibc upgrade programs, or for post-glibc upgrade programs, but not
> both.
>
> FHS distros have a similar problem though; how do they handle it?  Do
> they force services to be restarted when glibc is upgraded, or something
> along these lines?

I just asked this question in Debian's OFTC channel:

"how does debian handle glibc updates?  are services restarted when it
happens?  Or does it postpone updating glibc until the next reboot?"

And got for answer: "there is no magic postponing of updates"; the
external needrestart [0] program was also mentioned.

Researching some more, it seems this may be handled on Debian by the use
of postinst scripts (which is an arbitrary shell script run after a
package is installed); so the libc package of Debian for example
restarts the postgres service to avoid problems:

[0]  https://github.com/liske/needrestart
[1]  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=710275

> In our case, suppose libpam honors $PAM_DIRECTORY; we could tweak each
> PAM-using Shepherd service (login, sshd, etc.) so that it sets
> PAM_DIRECTORY… but how would we get the PAM_DIRECTORY value for the OS
> being configured?  Tricky!

Good question, but that seems a good path to pursue; old services would
be using their own old pam modules, allowing them to continue running
unimpacted, while new ones would get the updated pam modules.

> We could maybe sidestep the issue altogether with socket-activated
> services: they’d be started on-demand, so the second scenario above
> would be unlikely.  But getting there is quite a bit of work…

I fail to see how this would be a solution for openssh, which would
typically already be running unless you've never login ounce since the
machine was up (or am I missing something?).  Also, it seems to me inetd
can already do "socket activation", if this was somehow useful.

Thanks,

Maxim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.