Subject: bug#76505: 31.0.50; igc: M-x project-compile is slow

From: Ihor Radchenko <[email-masked]>
Date: Sun, 23 Feb 2025 16:27:52 +0000

Pip Cet <[email-masked]> writes:

>> Any suggestions, ideas?
>
> Many!  First, may I ask you to save a core file (attach gdb, "gcore",
> shouldn't destroy the session) somewhere along with the emacs binary and
> .pdmp file that correspond to it?  That way, if we fix your session we
> can still figure out why it was broken in the first place.  (Of course,
> you should nevershare the core file).

Oh. It is not broken. I am writing this very email from the same
session. It is just that running compilation makes Emacs hang for a few
seconds. Same when switching magit branches (and also calling external
process).

> The next step would be to set garbage-collection-messages to t to see
> whether it's indeed MPS that's to blame.

Here is what I see alongside with Emacs hanging when I run M-x
project-compile on Org mode git repo. The hang only happens while make
is spawning many subprocesses to compile individual .el files. Once it
proceeds to testing (and running a single emacs subprocess) the hangs stop.

Garbage collecting...
Generation 0 of a chain has reached capacity: start a minor collection.
Garbage collecting...
Generation 0 of a chain has reached capacity: start a minor collection.
Garbage collecting...
Opportunism: client predicts plenty of idle time, so start full collection.
pixel-scroll-precision-scroll-up: Beginning of buffer [2 times]
Garbage collecting...
Generation 0 of a chain has reached capacity: start a minor collection.
Garbage collecting...
Opportunism: client predicts plenty of idle time, so start full collection.

> What are the values of igc-step-interval and igc--balance-intervals?

igc-step-interval = 0.04
igc--balance-intervals = nil

> Can you send a dump of the output of M-x igc-stats and M-x
> igc-roots-stats?

What I did:
1. M-x igc-roots-stats
2. M-x igc-stats
3. In each window, press a and s for initial snapshot
4. Run M-x project compile (make test) on Org git repo
5. Observe the hangs until they finish
6. In each stats window, press s
7. In each stats window, press d

*igc roots*
bc-stack                       ambig               1         4194304
buffer                         ambig               2            1200
charset-table                  ambig               1           60480
control stack                  ambig               1             nil
dump-pins                      ambig               1              24
exact                          exact               1            8192
exact-n                        exact            2874         2617368
exact-ptr                      exact               7              56
kbd-buffer                     ambig               1          262144
lispsym                        exact               1           99344
main-thread                    exact               1             536
pure                           ambig               1         5166672
rdstack                        exact               1            3912
realloc-ambig                  ambig               3            3456
specpdl                        exact               1           62176
staticvec                      exact               1           16384
terminal-list                  ambig               1               8
tty-list                       exact               1               8
xpalloc-ambig                  ambig               1           18368
xpalloc-exact                  exact               2            1696
xzalloc-ambig                  ambig             105           17024

*igc*
IGC_OBJ_CONS                            597337        14336088         24           nil
IGC_OBJ_FLOAT                             -260           -6240         24           nil
IGC_OBJ_IMAGE                               -2            -528        264           nil
IGC_OBJ_INTERVAL                         -9099         -582336         64           nil
IGC_OBJ_ITREE_NODE                          -1             -88         88           nil
IGC_OBJ_ITREE_TREE                          -1             -32         32           nil
IGC_OBJ_MARKER_VECTOR                     -444          -30096         67           nil
IGC_OBJ_PAD                                527          -66496        126           nil
IGC_OBJ_STRING                           -8334         -333360         40           nil
IGC_OBJ_STRING_DATA                     -15384       -14186960        922           nil
IGC_OBJ_SYMBOL                              -3            -168         56           nil
IGC_OBJ_VECTOR                          -26817        -2073928         77           nil
PVEC_BIGNUM                               -122           -3904         32           nil
PVEC_BUFFER                               -401         -397792        992           nil
PVEC_CLOSURE                               694           64704         93           nil
PVEC_FONT                                -1009         -121272        120           nil
PVEC_FREE                                 -311          -10920         35           nil
PVEC_HASH_TABLE                          -2271         -199848         88           nil
PVEC_MARKER                              -5938         -332528         56           nil
PVEC_NORMAL_VECTOR                      -18007        -1101912         61           nil
PVEC_OVERLAY                                -1             -40         40           nil
PVEC_RECORD                                553           30928         55           nil
PVEC_WINDOW                                 -2           -1104        552           nil
PVEC_WINDOW_CONFIGURATION                   -2            -240        120           nil

> Does a manual full collection (M-x igc-collect) help temporarily?  How
> long does it take?

No, it does not help.
M-: (benchmark-progn (igc-collect))
Elapsed time: 3.086109s

Doing it repeatedly does not make it faster.

> Are you using the debug version of libmps or the standard version?

Standard version.

> Attaching gdb Monte-Carlo-style a few times during the hang and sending
> the "bt full" might also provide some hints.

The hang is just for a few seconds or so.
It will be tricky. Also, I never tried attaching gdb to process.

>> In GNU Emacs 31.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version
>>  3.24.42, cairo version 1.18.2) of 2025-02-15 built on localhost
>> Repository revision: 67e602105774a31508239a6d2a6a05a4d4c5d363
>
> Hmm.  That commit's a bit old, but I'm not aware of any major
> performance problems that would have gone away entirely since then.

I did not have crashes, so did not risk updates yet.
If you think it will be helpful, I can.

>> Repository branch: feature/igc
>> Windowing system distributor 'The X.Org Foundation', version 11.0.12101014
>> System Description: Gentoo Linux
>>
>> Configured using:
>>  'configure --with-mps=yes --with-native-compilation 'CFLAGS=-g3
>>  -I/opt/mps/include -L/opt/mps/lib'
>>  JAVAC=/etc/java-config-2/current-system-vm/bin/javac
>>  PKG_CONFIG_PATH=/usr/share/guile-data/3.0/pkgconfig'
>
> IIUC, configuring Emacs that way will add a "-O" flag to the CFLAGS,
> which is unusual as -O2 and -O3 are much more common.  I'm not sure
> whether that's what you intended to do, but it always surprises me :-)

CFLAGS=-g3 gives me symbols in perf data. With more optimizations, they
will disappear.

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>