GNU bug report logs - #53712
Guix System hangs after boot with linux-libre 5.15.17

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 53712 in the body.
You can then email your comments to 53712 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: bug-guix <at> gnu.org
Subject: Guix System hangs after boot with linux-libre 5.15.17
Date: Tue, 1 Feb 2022 15:39:27 -0500

Guix System on x86_64 hangs after boot when reconfigured to use
linux-libre 5.15.17.

The system becomes unresponsive to keyboard input after boot. Sometimes
it reaches the login prompt, sometimes not. Services that take a while
to start, typically finishing after the prompt is displayed, do not
ever start.

I bisected and confirmed that the problem is introduced in Guix Git
commit aad96ed54070 "gnu: linux-libre: Update to 5.15.17."

I could not reproduce the problem with `guix system vm [...]`. Maybe
this means that the problem only occurs after reconfiguring, rather than
a fresh system, or maybe it just doesn't occur at all in a KVM VM.

I'll try with 5.15.18 and the 5.10 series. We are still unable to deploy
linux-libre 5.16 for Guix System users due to this bug:

https://issues.guix.gnu.org/53554

Message #8 received at 53712 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Leo Famulari <leo <at> famulari.name>
Cc: 53712 <at> debbugs.gnu.org
Subject: Re: bug#53712: Guix System hangs after boot with linux-libre 5.15.17
Date: Tue, 01 Feb 2022 16:11:27 -0500

Hi Leo,

Leo Famulari <leo <at> famulari.name> writes:

> Guix System on x86_64 hangs after boot when reconfigured to use
> linux-libre 5.15.17.
>
> The system becomes unresponsive to keyboard input after boot. Sometimes
> it reaches the login prompt, sometimes not. Services that take a while
> to start, typically finishing after the prompt is displayed, do not
> ever start.
>
> I bisected and confirmed that the problem is introduced in Guix Git
> commit aad96ed54070 "gnu: linux-libre: Update to 5.15.17."
>
> I could not reproduce the problem with `guix system vm [...]`. Maybe
> this means that the problem only occurs after reconfiguring, rather than
> a fresh system, or maybe it just doesn't occur at all in a KVM VM.
>
> I'll try with 5.15.18 and the 5.10 series. We are still unable to deploy
> linux-libre 5.16 for Guix System users due to this bug:

Not sure if that helps, but my system is running smoothly on 5.15.18 as
I write this.  What are the specifics of your system?

Thanks,

Maxim

Message #11 received at 53712 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: 53712 <at> debbugs.gnu.org
Subject: Re: Guix System hangs after boot with linux-libre 5.15.17
Date: Tue, 1 Feb 2022 16:44:24 -0500

On Tue, Feb 01, 2022 at 03:39:27PM -0500, Leo Famulari wrote:
> I'll try with 5.15.18 and the 5.10 series. We are still unable to deploy
> linux-libre 5.16 for Guix System users due to this bug:

Same problem with 5.10.95.

Message #14 received at 53712 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 53712 <at> debbugs.gnu.org
Subject: Re: bug#53712: Guix System hangs after boot with linux-libre 5.15.17
Date: Tue, 1 Feb 2022 16:58:41 -0500

[Message part 1 (text/plain, inline)]

On Tue, Feb 01, 2022 at 04:11:27PM -0500, Maxim Cournoyer wrote:
> Not sure if that helps, but my system is running smoothly on 5.15.18 as
> I write this.  What are the specifics of your system?

Interestingly, for me 5.15.18 does boot to an interactive console, but
the system fails to bring up the wifi interface, and then it fails to
halt upon command, just hanging forever. At least, it did that for the 3
times I tried using that generation.

This is a Thinkpad x200s with 3 GB RAM. Normal BIOS. I've attached the
operating-system declaration.

There are similar reports on #guix IRC from two other users:

http://logs.guix.gnu.org/guix/2022-01-31.log#152606
http://logs.guix.gnu.org/guix/2022-01-30.log#005034

[system.scm (text/plain, attachment)]

Message #17 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: bug-guix <at> gnu.org
Subject: Re: Guix System hangs after boot with linux-libre 5.15.17
Date: Tue, 1 Feb 2022 17:09:54 -0500

On Tue, Feb 01, 2022 at 03:39:27PM -0500, Leo Famulari wrote:
> Guix System on x86_64 hangs after boot when reconfigured to use
> linux-libre 5.15.17.

More anecdata:

While I build Linux 5.15.17 (not linux-libre) using the Guix kernel
config, adapted for Debian, it works fine on Debian.

The only changes required for Debian are:

1) Switch from CONFIG_MODULE_COMPRESS_GZIP to CONFIG_MODULE_COMPRESS_XZ
2) Set CONFIG_MODPROBE_PATH=/sbin/modprobe

This Debian kernel doesn't use the config options set in ((gnu packages
linux) %default-extra-linux-options), so it's not a very useful point of
comparison.

Message #20 received at 53712 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: 53712 <at> debbugs.gnu.org
Subject: Re: bug#53712: Guix System hangs after boot with linux-libre 5.15.17
Date: Tue, 1 Feb 2022 17:11:28 -0500

On Tue, Feb 01, 2022 at 03:39:27PM -0500, Leo Famulari wrote:
> Guix System on x86_64 hangs after boot when reconfigured to use
> linux-libre 5.15.17.

More anecdata:

While I build Linux 5.15.17 (not linux-libre) using the Guix kernel
config, adapted for Debian, it works fine on Debian.

The only changes required for Debian are:

1) Switch from CONFIG_MODULE_COMPRESS_GZIP to CONFIG_MODULE_COMPRESS_XZ
2) Set CONFIG_MODPROBE_PATH=/sbin/modprobe

This Debian kernel doesn't use the config options set in ((gnu packages
linux) %default-extra-linux-options), so it's not a very useful point of
comparison.

Message #23 received at 53712 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: 53712 <at> debbugs.gnu.org
Subject: Re: bug#53712: Guix System hangs after boot with linux-libre 5.15.17
Date: Wed, 2 Feb 2022 14:51:31 -0500

Discussion of the problem on a reddit page about Fedora:

https://old.reddit.com/r/Fedora/comments/shhj6e/kernels_above_51516_are_crashing/

They say that 5.15.18 works for them.

Message #26 received at 53712 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: 53712 <at> debbugs.gnu.org
Subject: Re: bug#53712: Guix System hangs after boot with linux-libre 5.15.17
Date: Thu, 3 Feb 2022 13:06:41 -0500

The failure exists for me with the most recent versions of the kernel
series 5.4, 5.10, 5.15, and 5.16.

Sometimes the screen does display the end of a kernel panic, but I can
only see the last few lines.

Message #31 received at 53712 <at> debbugs.gnu.org (full text, mbox):

From: Katherine Cox-Buday <cox.katherine.e <at> gmail.com>
To: 53712 <at> debbugs.gnu.org
Subject: I'm also experiencing this
Date: Mon, 07 Feb 2022 10:54:53 -0600

I thought I'd chime in with another data-point:

- Lenovo ThinkPad T480s
- Normal BIOS
- Vanilla Linux kernel(s)
- kernel v5.15.16 boots fine; everything after seems to hit this bug
- Turning off bluetooth in BIOS seemed to get boot further
- Turning off wifi in BIOS allows me to boot

The spec sheet tells me it has a Intel Dual Band Wireless-AC 8265 card, so I guess it's something to do with that. I don't know if this is an upstream kernel bug or something to do with distros specifically?

I hope this helps narrow things down. Luckily my laptop is just used as a desktop, and I have ethernet, so I can just keep Wifi disabled.

-- 
Katherine

Message #34 received at 53712 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Katherine Cox-Buday <cox.katherine.e <at> gmail.com>
Cc: 53712 <at> debbugs.gnu.org
Subject: Re: bug#53712: I'm also experiencing this
Date: Mon, 7 Feb 2022 13:14:58 -0500

On Mon, Feb 07, 2022 at 10:54:53AM -0600, Katherine Cox-Buday wrote:
> I thought I'd chime in with another data-point:
> 
> - Lenovo ThinkPad T480s
> - Normal BIOS
> - Vanilla Linux kernel(s)
> - kernel v5.15.16 boots fine; everything after seems to hit this bug
> - Turning off bluetooth in BIOS seemed to get boot further
> - Turning off wifi in BIOS allows me to boot
> 
> The spec sheet tells me it has a Intel Dual Band Wireless-AC 8265 card, so I guess it's something to do with that. I don't know if this is an upstream kernel bug or something to do with distros specifically?
> 
> I hope this helps narrow things down. Luckily my laptop is just used as a desktop, and I have ethernet, so I can just keep Wifi disabled.

Thanks for the notes.

I'm using wifi supported by the drivers from Guix's
rtl8812au-aircrack-ng-linux-module with linux-libre, so it also affects
users who only use the GNU Guix channel.

I also noticed some improvement when removing the wifi dongle, although
I was still unable to properly halt or reboot.

Message #37 received at 53712 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: 53712 <at> debbugs.gnu.org
Subject: Re: bug#53712: Guix System hangs after boot with linux-libre 5.15.17
Date: Wed, 9 Feb 2022 13:23:53 -0500

I'm still experiencing this problem with 5.15.21, as well 5.16.7 and
5.4.177.

Message #40 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Squirrel <1953494819 <at> qq.com>
To: bug-guix <at> gnu.org
Subject: bug#53712: Guix System hangs after boot with linux-libre 5.15.17
Date: Fri, 11 Feb 2022 04:03:03 +0800

It seems that this is a bug from upstream linux, which may be patched 
soon. Please see 
https://lore.kernel.org/stable/164448100914.10463.9523338503936670263.kvalo <at> kernel.org/

Blacklisting the iwlwifi kernel module works as a workaround for now, as 
is suggested by Jason Self, the maintainer of the Freesh and libeRTy apt 
repositories of linux-libre kernel. He mentioned it in Trisquel forum at 
https://trisquel.info/en/forum/trisquel-9-linux-libre-51517-will-not-allow-login 
and at https://lore.kernel.org/all/20220203161959.3edf1d6e <at> valencia/

I'd removed the Intel AX201NGW Wi-Fi & Bluetooth card from my Acer Swift 
3x laptop two days ago. There isn't any wireless network card installed 
now. The system runs without errors, which may be because the iwlwifi 
kernel module is not loaded due to the lack of a Wi-Fi network card. 
Before that, the system hanged randomly with the most recent versions of 
the kernel series 5.4, 5.10 and 5.15.

Message #43 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Squirrel via Bug reports for GNU Guix <bug-guix <at> gnu.org>
Cc: 53712 <at> debbugs.gnu.org
Subject: Re: bug#53712: Guix System hangs after boot with linux-libre 5.15.17
Date: Sat, 12 Feb 2022 21:06:09 -0500

On Fri, Feb 11, 2022 at 04:03:03AM +0800, Squirrel via Bug reports for GNU Guix wrote:
> Blacklisting the iwlwifi kernel module works as a workaround for now, as is
> suggested by Jason Self, the maintainer of the Freesh and libeRTy apt
> repositories of linux-libre kernel. He mentioned it in Trisquel forum at https://trisquel.info/en/forum/trisquel-9-linux-libre-51517-will-not-allow-login
> and at https://lore.kernel.org/all/20220203161959.3edf1d6e <at> valencia/

Thanks for the info!

I can confirm that adding the modprobe incantation to my
operating-system declaration, reconfiguring, and rebooting does fix the
bug:

(kernel-arguments
 '(; https://issues.guix.gnu.org/53712#12
   "modprobe.blacklist=pcspkr,snd_pcsp,iwlwifi"))

https://guix.gnu.org/manual/devel/en/html_node/operating_002dsystem-Reference.html#FOOT24

Message #51 received at 53712 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 54010 <at> debbugs.gnu.org, 53712 <at> debbugs.gnu.org
Subject: Re: bug#53712: Guix System hangs after boot with linux-libre 5.15.17
Date: Thu, 17 Feb 2022 10:12:49 +0100

Hi,

Leo Famulari <leo <at> famulari.name> skribis:

> On Tue, Feb 15, 2022 at 03:56:27PM +0100, Ludovic Courtès wrote:
>> 2022 has left me without a working Linux-libre kernel.
>> 
>> Breakage occurred sometime between:
>> 
>>   • 92faad0adb93b8349bfd7c67911d3d95f0505eb2
>>     (Jan. 3rd; Linux-libre 5.15.12)
>> 
>>   • 43dd34c7777a212c99a97da7a2c237158faa9a1b
>>     (Jan. 31st; Linux-libre 5.15.17)
>
> Isn't this #53712?

It is!

> I recommend trying the workaround described there, which is to blacklist
> the iwlwifi kernel module, if you are not using it but have iwlwifi
> hardware:
>
> https://issues.guix.gnu.org/53712#13

I confirm that 5.15.23-gnu works for me with:

  (kernel-arguments
   '(; https://issues.guix.gnu.org/53712#12
     "modprobe.blacklist=usbmouse,usbkbd,iwlwifi"
     "quiet"))

Thank you!

Ludo’.

Message #54 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Squirrel via Bug reports for GNU Guix <bug-guix <at> gnu.org>
Cc: 53712-done <at> debbugs.gnu.org
Subject: Re: bug#53712: Guix System hangs after boot with linux-libre 5.15.17
Date: Thu, 24 Feb 2022 10:08:38 -0500

On Fri, Feb 11, 2022 at 04:03:03AM +0800, Squirrel via Bug reports for GNU Guix wrote:
> It seems that this is a bug from upstream linux, which may be patched soon.
> Please see https://lore.kernel.org/stable/164448100914.10463.9523338503936670263.kvalo <at> kernel.org/

Good news, this bug has been fixed in the latest group of stable Linux
releases:

5.16.11
5.15.25
5.10.102
5.4.181
4.19.231
4.14.268
4.9.303

For our default release series, 5.15, it was fixed as commit
ddd46059f7d99119b62d44c519df7a79f2e6a515:

https://cdn.kernel.org/pub/linux/kernel/v5.x/ChangeLog-5.15.25

Message #67 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Squirrel <1953494819 <at> qq.com>
To: bug-guix <at> gnu.org
Subject: bug#53712: Guix System hangs after boot with linux-libre 5.15.17
Date: Wed, 2 Mar 2022 23:39:10 +0800

On Thu, 24 Feb 2022 10:08:38 -0500, Leo Famulari <leo <at> famulari.name> wrote:
> Good news, this bug has been fixed in the latest group of stable Linux
> releases:
>
> 5.16.11
> 5.15.25
> 5.10.102
> 5.4.181
> 4.19.231
> 4.14.268
> 4.9.303
>
> For our default release series, 5.15, it was fixed as commit
> ddd46059f7d99119b62d44c519df7a79f2e6a515:
>
> https://cdn.kernel.org/pub/linux/kernel/v5.x/ChangeLog-5.15.25
Thank you! I eventually have time to reinstall the WiFi card to test the 
5.10.102 kernel, and it works!

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.