Subject: Re: bug#70423: 29.3; cc-mode hang at 100% cpu and consuming all available memory

From: [email-masked]
Date: Fri, 19 Apr 2024 20:49:20 +0000

Hello again, Alan!

I've been using a git revision of hyperbole with the regex removed while Bob works on the new regex. I've hit a few other small glitches while using it, but nothing anywhere as major as the main issue at hand, and they're most definitely unrelated.

I also had a look in the other bug report and it seemed like there was also some kind of patch on the cc-mode side in emacs 30/trunk ? Or did I misunderstand that conversation ? I could try running an emacs git build if that's the case, although I'm not sure how many of my other packages will remain compatible, so it's not really a priority.

My init.el script is really innocent, although I've later asked some friends and they agree that they would also probably never run something like that. I just thought it would be a fun challenge to do everything to reproduce the issue in one init script. The base64-encoded string is just a gzipped copy of the .cpp file with which I could reproduce the issue. You should get the same string by running "gzip --stdout drm_formats.cpp|base64" on the attached .cpp file. (However, I don't know if all gzip versions produce the same output given the same input, and it will make me look even more suspicious if the strings don't match.)

Regardless, I've been productively working on other C++ code with the regexp turned off and it's been a great relief not to have to struggle with having to restart my workspace/desktop so often.

Thank you for both of yours work! And good luck with the new regexp!
If you put it in some git tag/branch that I can pull easily, I could maybe assist with testing it ? Let me know if that's an option.

Kind regards,
M

On Thursday, April 18th, 2024 at 8:39 PM, Alan Mackenzie <[email-masked]> wrote:

> Hello, Pillowtrucker!

> On Tue, Apr 16, 2024 at 19:51:09 +0000, [email-masked] wrote:
> 

> > Greetings!

> > I have been able to replicate a (relatively) minimum environment for
> > this — hyperbole + clang-format.el. I think lsp-mode aggravates the
> > issue somehow, but is not necessary to enter the infinite loop. Without
> > lsp-mode, however, the memory usage seems flat, as opposed to
> > ballooning when it is present. I would also guess that if you are able
> > to plug in that regexp into cc-mode without installing all of
> > hyperbole.el, you would probably get the same effect, but I didn't know
> > how to do that.
> 

> > You will need gzip and the clang-format utility for this from your
> > linux distribution or compiled from source. Preferably version 18.1.3
> > of the llvm suite, but it might work with earlier versions. I don't
> > think /how/ it's formatting the file matters as much as the fact of it
> > being altered to some degree.
> 

> OK. I haven't actually evaluated your init.el (as I don't fully
> understand it), but I have a strong hypothesis for the looping.
> 

> You're loading hyperbole as part of the initialisation. This sets up the
> value of defun-prompt-regexp to a ~760 character long string, whereas
> normally in C++ Mode, this variable is nil.
> 

> This variable is what is seen in the second backtrace below, although it
> is bowdlerised to a very short string ending in ... . See the line below
> emphasised with <=========================.
> 

> This regexp is known to be faulty, and sometimes to cause infinite
> looping. By coincidence, I have recently rewritten this regexp at the
> request of Bob Weiner (in the Cc:), the maintainer of hyperbole, and he
> is currently testing the regexp. This faulty regexp may be the cause of
> your bug, too.
> 

> So, could I ask you please to be patient, and wait for Bob to complete
> his tests. If they are successful, a new version of hyperbole is the
> likely outcome, and this may solve your bug.
> 

> If there is any way you can run your test without installing the
> hyperbole package, please do so, and let me know whether or not the hang
> still occurs.
> 

> Thanks!

> > Please follow these steps:
> > 1. start with an empty directory
> > $ export MYDEBUGDIR=emacs-debugging
> > 2. use the attached init.el file as follows:
> > $ emacs-29.3 --init-directory=${MYDEBUGDIR} -l init.el
> > 3. Wait for the script to reach the point where it's in an infinite loop (check top/htop for emacs with 100% usage, maybe wait a few seconds or minutes to make sure it is not just taking a long time to run the formatting) and press C-g to send "quit".
> 

> > This results in a hang and the debugger should show something like this:
> 

> [ .... ]

> > or maybe you will see:

> > Debugger entered--Lisp error: (quit)
> > re-search-backward("^\\s(\\|\\(?:^[ \11]\\(template\\s-<[^>;.{}]+>\\s-*\\)?\\(..." nil move 1) <==========================================================================
> > beginning-of-defun-raw(nil)
> > beginning-of-defun()
> > c-get-fallback-scan-pos(18908)
> > c-parse-state-get-strategy(18908 1)
> > c-parse-state-1()
> > c-parse-state()
> > c-laomib-invalidate-cache(18908 18910)
> > c-before-change(18908 18910)
> > clang-format--replace(18908 2 "\n")
> > apply(clang-format--replace (18908 2 "\n"))
> > clang-format-region(1 18910 nil nil)
> > clang-format-buffer()
> > load-with-code-conversion("/home/wrath/emacs-debugging2/init.el" "/home/wrath/emacs-debugging2/init.el" nil t)
> > command-line-1(("-l" "emacs-debugging2/init.el"))
> > command-line()
> > normal-top-level()
> 

> > Either way it does seem like the problem lies somewhere in cc-mode
> > choking either on the file itself or on that regular expression.
> 

> > I license the attached init.el under the terms of the AGPL-3 or later
> > version. The drm_formats.cpp file is part of the MIR Server project and
> > is licensed under the terms of the GPL-v3. Both licenses attached. I'm
> > not sure if I am allowed to re-license the MIR file as AGPL, but I
> > believe they are in any case compatible to be distributed together.
> 

> > Thank you for having a look at this issue, it has been plaguing me
> > since november and even making rust look like an ever so slightly more
> > pleasant alternative to dealing with constant hangs/crashes and having
> > to restore frame/window layouts when editing C/C++..
> 

> > Kind regards,
> > M.

> --
> Alan Mackenzie (Nuremberg, Germany).