GNU bug report logs -
#76505
31.0.50; igc: M-x project-compile is slow
Previous Next
To reply to this bug, email your comments to 76505 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 15:54:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Ihor Radchenko <yantar92 <at> posteo.net>:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org.
(Sun, 23 Feb 2025 15:54:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Hi,
After running Emacs for a few days (no crashes!), I noticed significant
hangs when running compilation.
The perf data is silent, but Emacs becomes unresponsive right at the
beginning of compilations and sometimes in the middle, as the
compilation buffer is filled.
Any suggestions, ideas?
1.46% emacs ld-linux-x86-64.so.2 [.] do_lookup_x
1.11% emacs emacs [.] compact_buffer
1.11% emacs libc.so.6 [.] pthread_mutex_lock
1.09% emacs libc.so.6 [.] pthread_mutex_unlock
1.08% emacs emacs [.] ChunkOfAddr
0.95% emacs [unknown] [k] 0xffffffff90e001c8
0.84% emacs emacs [.] fix_lisp_obj
0.76% emacs emacs [.] ChainDeferral
0.67% emacs emacs [.] igc_on_idle
0.66% emacs emacs-30-vcs [.] 0x00000000001bdc09
0.59% emacs emacs [.] GenDescNewSize
0.58% emacs emacs [.] re_match_2_internal
0.53% emacs emacs [.] _mps_fix2
0.53% emacs emacs-30-vcs [.] 0x00000000001ed929
0.47% emacs emacs [.] ArenaStep
0.44% emacs [vdso] [.] __vdso_clock_gettime
0.40% emacs emacs [.] ShieldFlush
0.39% emacs emacs [.] shieldQueue
0.38% emacs emacs-30-vcs [.] 0x00000000001c907f
0.37% emacs emacs-30-vcs [.] re_search_2
0.37% emacs emacs [.] QuickSort
0.37% emacs emacs [.] amcSegFix
0.35% emacs emacs-30-vcs [.] 0x00000000001bdeda
0.35% emacs emacs [.] dflt_scanx
0.32% emacs emacs [.] PolicyStartTrace
0.30% emacs [unknown] [k] 0xffffffff90e018a8
0.29% cc1plus cc1plus [.] push_to_top_level
0.28% emacs macroexp-2c3e1495-60fafb77.eln [.] F6d6163726f6578702d2d657870616e642d616c6c_macroexp__expand_all_0
In GNU Emacs 31.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version
3.24.42, cairo version 1.18.2) of 2025-02-15 built on localhost
Repository revision: 67e602105774a31508239a6d2a6a05a4d4c5d363
Repository branch: feature/igc
Windowing system distributor 'The X.Org Foundation', version 11.0.12101014
System Description: Gentoo Linux
Configured using:
'configure --with-mps=yes --with-native-compilation 'CFLAGS=-g3
-I/opt/mps/include -L/opt/mps/lib'
JAVAC=/etc/java-config-2/current-system-vm/bin/javac
PKG_CONFIG_PATH=/usr/share/guile-data/3.0/pkgconfig'
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 16:09:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 76505 <at> debbugs.gnu.org (full text, mbox):
"Ihor Radchenko" <yantar92 <at> posteo.net> writes:
> Hi,
>
> After running Emacs for a few days (no crashes!), I noticed significant
> hangs when running compilation.
Thanks for the report, we're very interested in cases where feature/igc
becomes unusable (less usable) in large sessions!
> The perf data is silent, but Emacs becomes unresponsive right at the
> beginning of compilations and sometimes in the middle, as the
> compilation buffer is filled.
>
> Any suggestions, ideas?
Many! First, may I ask you to save a core file (attach gdb, "gcore",
shouldn't destroy the session) somewhere along with the emacs binary and
.pdmp file that correspond to it? That way, if we fix your session we
can still figure out why it was broken in the first place. (Of course,
you should nevershare the core file).
The next step would be to set garbage-collection-messages to t to see
whether it's indeed MPS that's to blame.
What are the values of igc-step-interval and igc--balance-intervals?
Can you send a dump of the output of M-x igc-stats and M-x
igc-roots-stats?
Does a manual full collection (M-x igc-collect) help temporarily? How
long does it take?
Are you using the debug version of libmps or the standard version?
IIUC, the debug version sometimes verifies so many things that it runs
into complexity issues.
Attaching gdb Monte-Carlo-style a few times during the hang and sending
the "bt full" might also provide some hints.
> In GNU Emacs 31.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version
> 3.24.42, cairo version 1.18.2) of 2025-02-15 built on localhost
> Repository revision: 67e602105774a31508239a6d2a6a05a4d4c5d363
Hmm. That commit's a bit old, but I'm not aware of any major
performance problems that would have gone away entirely since then.
> Repository branch: feature/igc
> Windowing system distributor 'The X.Org Foundation', version 11.0.12101014
> System Description: Gentoo Linux
>
> Configured using:
> 'configure --with-mps=yes --with-native-compilation 'CFLAGS=-g3
> -I/opt/mps/include -L/opt/mps/lib'
> JAVAC=/etc/java-config-2/current-system-vm/bin/javac
> PKG_CONFIG_PATH=/usr/share/guile-data/3.0/pkgconfig'
IIUC, configuring Emacs that way will add a "-O" flag to the CFLAGS,
which is unusual as -O2 and -O3 are much more common. I'm not sure
whether that's what you intended to do, but it always surprises me :-)
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 16:29:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 76505 <at> debbugs.gnu.org (full text, mbox):
Pip Cet <pipcet <at> protonmail.com> writes:
>> Any suggestions, ideas?
>
> Many! First, may I ask you to save a core file (attach gdb, "gcore",
> shouldn't destroy the session) somewhere along with the emacs binary and
> .pdmp file that correspond to it? That way, if we fix your session we
> can still figure out why it was broken in the first place. (Of course,
> you should nevershare the core file).
Oh. It is not broken. I am writing this very email from the same
session. It is just that running compilation makes Emacs hang for a few
seconds. Same when switching magit branches (and also calling external
process).
> The next step would be to set garbage-collection-messages to t to see
> whether it's indeed MPS that's to blame.
Here is what I see alongside with Emacs hanging when I run M-x
project-compile on Org mode git repo. The hang only happens while make
is spawning many subprocesses to compile individual .el files. Once it
proceeds to testing (and running a single emacs subprocess) the hangs stop.
Garbage collecting...
Generation 0 of a chain has reached capacity: start a minor collection.
Garbage collecting...
Generation 0 of a chain has reached capacity: start a minor collection.
Garbage collecting...
Opportunism: client predicts plenty of idle time, so start full collection.
pixel-scroll-precision-scroll-up: Beginning of buffer [2 times]
Garbage collecting...
Generation 0 of a chain has reached capacity: start a minor collection.
Garbage collecting...
Opportunism: client predicts plenty of idle time, so start full collection.
> What are the values of igc-step-interval and igc--balance-intervals?
igc-step-interval = 0.04
igc--balance-intervals = nil
> Can you send a dump of the output of M-x igc-stats and M-x
> igc-roots-stats?
What I did:
1. M-x igc-roots-stats
2. M-x igc-stats
3. In each window, press a and s for initial snapshot
4. Run M-x project compile (make test) on Org git repo
5. Observe the hangs until they finish
6. In each stats window, press s
7. In each stats window, press d
*igc roots*
bc-stack ambig 1 4194304
buffer ambig 2 1200
charset-table ambig 1 60480
control stack ambig 1 nil
dump-pins ambig 1 24
exact exact 1 8192
exact-n exact 2874 2617368
exact-ptr exact 7 56
kbd-buffer ambig 1 262144
lispsym exact 1 99344
main-thread exact 1 536
pure ambig 1 5166672
rdstack exact 1 3912
realloc-ambig ambig 3 3456
specpdl exact 1 62176
staticvec exact 1 16384
terminal-list ambig 1 8
tty-list exact 1 8
xpalloc-ambig ambig 1 18368
xpalloc-exact exact 2 1696
xzalloc-ambig ambig 105 17024
*igc*
IGC_OBJ_CONS 597337 14336088 24 nil
IGC_OBJ_FLOAT -260 -6240 24 nil
IGC_OBJ_IMAGE -2 -528 264 nil
IGC_OBJ_INTERVAL -9099 -582336 64 nil
IGC_OBJ_ITREE_NODE -1 -88 88 nil
IGC_OBJ_ITREE_TREE -1 -32 32 nil
IGC_OBJ_MARKER_VECTOR -444 -30096 67 nil
IGC_OBJ_PAD 527 -66496 126 nil
IGC_OBJ_STRING -8334 -333360 40 nil
IGC_OBJ_STRING_DATA -15384 -14186960 922 nil
IGC_OBJ_SYMBOL -3 -168 56 nil
IGC_OBJ_VECTOR -26817 -2073928 77 nil
PVEC_BIGNUM -122 -3904 32 nil
PVEC_BUFFER -401 -397792 992 nil
PVEC_CLOSURE 694 64704 93 nil
PVEC_FONT -1009 -121272 120 nil
PVEC_FREE -311 -10920 35 nil
PVEC_HASH_TABLE -2271 -199848 88 nil
PVEC_MARKER -5938 -332528 56 nil
PVEC_NORMAL_VECTOR -18007 -1101912 61 nil
PVEC_OVERLAY -1 -40 40 nil
PVEC_RECORD 553 30928 55 nil
PVEC_WINDOW -2 -1104 552 nil
PVEC_WINDOW_CONFIGURATION -2 -240 120 nil
> Does a manual full collection (M-x igc-collect) help temporarily? How
> long does it take?
No, it does not help.
M-: (benchmark-progn (igc-collect))
Elapsed time: 3.086109s
Doing it repeatedly does not make it faster.
> Are you using the debug version of libmps or the standard version?
Standard version.
> Attaching gdb Monte-Carlo-style a few times during the hang and sending
> the "bt full" might also provide some hints.
The hang is just for a few seconds or so.
It will be tricky. Also, I never tried attaching gdb to process.
>> In GNU Emacs 31.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version
>> 3.24.42, cairo version 1.18.2) of 2025-02-15 built on localhost
>> Repository revision: 67e602105774a31508239a6d2a6a05a4d4c5d363
>
> Hmm. That commit's a bit old, but I'm not aware of any major
> performance problems that would have gone away entirely since then.
I did not have crashes, so did not risk updates yet.
If you think it will be helpful, I can.
>> Repository branch: feature/igc
>> Windowing system distributor 'The X.Org Foundation', version 11.0.12101014
>> System Description: Gentoo Linux
>>
>> Configured using:
>> 'configure --with-mps=yes --with-native-compilation 'CFLAGS=-g3
>> -I/opt/mps/include -L/opt/mps/lib'
>> JAVAC=/etc/java-config-2/current-system-vm/bin/javac
>> PKG_CONFIG_PATH=/usr/share/guile-data/3.0/pkgconfig'
>
> IIUC, configuring Emacs that way will add a "-O" flag to the CFLAGS,
> which is unusual as -O2 and -O3 are much more common. I'm not sure
> whether that's what you intended to do, but it always surprises me :-)
CFLAGS=-g3 gives me symbols in perf data. With more optimizations, they
will disappear.
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 16:52:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 76505 <at> debbugs.gnu.org (full text, mbox):
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 16:55:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 76505 <at> debbugs.gnu.org (full text, mbox):
> Cc: 76505 <at> debbugs.gnu.org
> From: Ihor Radchenko <yantar92 <at> posteo.net>
> Date: Sun, 23 Feb 2025 16:27:52 +0000
>
> Pip Cet <pipcet <at> protonmail.com> writes:
>
> >> Any suggestions, ideas?
A couple more questions:
. what is the memory footprint (resident and VM total) of the Emacs
process when this happens?
. do you see any signs of paging (significant disk activity) when
those "hangs" happen?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 17:01:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 76505 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
> . what is the memory footprint (resident and VM total) of the Emacs
> process when this happens?
RES=3.4g VIRT=11.2g
It does not really change when I run project-compile.
It is also fairly normal for my Emacs sessions.
> . do you see any signs of paging (significant disk activity) when
> those "hangs" happen?
No.
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 17:06:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 76505 <at> debbugs.gnu.org (full text, mbox):
> From: Ihor Radchenko <yantar92 <at> posteo.net>
> Cc: pipcet <at> protonmail.com, 76505 <at> debbugs.gnu.org
> Date: Sun, 23 Feb 2025 17:00:00 +0000
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> > . what is the memory footprint (resident and VM total) of the Emacs
> > process when this happens?
>
> RES=3.4g VIRT=11.2g
> It does not really change when I run project-compile.
> It is also fairly normal for my Emacs sessions.
You mean, Emacs has this approximate memory footprint as soon as you
start a fresh session?
Can you start the cpu profiler just before you run project-compile,
and then show the profile (assuming the profiler doesn't cause crashes
on the branch in your build)?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 17:14:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 76505 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Eli Zaretskii <eliz <at> gnu.org> writes:
>> RES=3.4g VIRT=11.2g
>> It does not really change when I run project-compile.
>> It is also fairly normal for my Emacs sessions.
>
> You mean, Emacs has this approximate memory footprint as soon as you
> start a fresh session?
As soon as I open my Org files, the footprint is roughly half of that.
Over time, it tends to increase to the above levels and stay there.
> Can you start the cpu profiler just before you run project-compile,
> and then show the profile (assuming the profiler doesn't cause crashes
> on the branch in your build)?
See the attached.
[project-compile.eld (application/octet-stream, attachment)]
[Message part 3 (text/plain, inline)]
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 17:52:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 76505 <at> debbugs.gnu.org (full text, mbox):
"Ihor Radchenko" <yantar92 <at> posteo.net> writes:
> Pip Cet <pipcet <at> protonmail.com> writes:
>
>>> Any suggestions, ideas?
>>
>> Many! First, may I ask you to save a core file (attach gdb, "gcore",
>> shouldn't destroy the session) somewhere along with the emacs binary and
>> .pdmp file that correspond to it? That way, if we fix your session we
>> can still figure out why it was broken in the first place. (Of course,
>> you should nevershare the core file).
>
> Oh. It is not broken. I am writing this very email from the same
> session. It is just that running compilation makes Emacs hang for a few
> seconds. Same when switching magit branches (and also calling external
> process).
>
>> The next step would be to set garbage-collection-messages to t to see
>> whether it's indeed MPS that's to blame.
>
> Here is what I see alongside with Emacs hanging when I run M-x
> project-compile on Org mode git repo. The hang only happens while make
> is spawning many subprocesses to compile individual .el files. Once it
> proceeds to testing (and running a single emacs subprocess) the hangs stop.
>
> Garbage collecting...
> Generation 0 of a chain has reached capacity: start a minor collection.
> Garbage collecting...
> Generation 0 of a chain has reached capacity: start a minor collection.
> Garbage collecting...
> Opportunism: client predicts plenty of idle time, so start full collection.
This seems like an IGC bug for very unusual MPS parameters, which we
use. (I hesitate to call it an MPS bug, but the documentation could be
clearer here).
The relevant code is ArenaStep in global.c:
Bool ArenaStep(Globals globals, double interval, double multiplier)
{
Bool workWasDone = FALSE;
Clock start, intervalEnd, availableEnd, now;
start = now = ClockNow();
intervalEnd = start + (Clock)(interval * (double)clocks_per_sec);
availableEnd = start + (Clock)(interval * multiplier * (double)clocks_per_sec);
/* loop while there is work to do and time on the clock. */
do {
Trace trace;
if (arena->busyTraces != TraceSetEMPTY) {
trace = ArenaTrace(arena, (TraceId)0);
/* falls through */
} else {
/* No traces are running: consider collecting the world. */
if (PolicyShouldCollectWorld(arena, (double)(availableEnd - now), now,
clocks_per_sec))
{
Res res;
res = TraceStartCollectAll(&trace, arena, TraceStartWhyOPPORTUNISM);
if (res != ResOK)
break;
arena->lastWorldCollect = now;
} else {
/* Not worth collecting the world; consider starting a trace. */
Bool worldCollected;
if (!PolicyStartTrace(&trace, &worldCollected, arena, FALSE))
break;
}
}
TraceAdvance(trace);
if (trace->state == TraceFINISHED)
TraceDestroyFinished(trace);
workWasDone = TRUE;
now = ClockNow();
} while (now < intervalEnd);
...
}
We call this code with multiplier = 0. It looks to me like the lowest
useful value for multiplier is 1.0.
The problem, I think, is that the do { ... } while () loop runs (at
least) twice: the first time, a busy trace finishes, the code falls
through, "now" is reset to the *current* clock, which makes
(availableEnd - now) "negative". But as the types are both unsigned,
the "negative" value will correspond to a large Clock value, which will
make MPS reach the conclusion it's a good time to start a full
collection.
I think the intention was for "multiplier" in the above code always to
be at least 1.0. The documentation isn't quite clear on what
"multiplier" is supposed to be set to, particularly in our case where we
want to discourage full collections. 1.0 seems to be the lowest safe
value, so let's use that?
> Opportunism: client predicts plenty of idle time, so start full collection.
Two full collections would cause a significant slow-down, yes.
If you want to continue testing before this is fixed, please consider
setting igc-step-interval to 0 (the fixnum, not the float) or 0.0 (which
has different behavior because we have a do { ... } while (...) loop).
Does that analysis make sense?
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 18:00:03 GMT)
Full text and
rfc822 format available.
Message #32 received at 76505 <at> debbugs.gnu.org (full text, mbox):
Pip Cet <pipcet <at> protonmail.com> writes:
>> Opportunism: client predicts plenty of idle time, so start full collection.
>
> Two full collections would cause a significant slow-down, yes.
>
> If you want to continue testing before this is fixed, please consider
> setting igc-step-interval to 0 (the fixnum, not the float) or 0.0 (which
> has different behavior because we have a do { ... } while (...) loop).
Setting it to 0 makes hangs somewhat shorter, but they are still there.
Setting it to 0.0 does not change anything.
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 18:05:02 GMT)
Full text and
rfc822 format available.
Message #35 received at 76505 <at> debbugs.gnu.org (full text, mbox):
"Ihor Radchenko" <yantar92 <at> posteo.net> writes:
> Pip Cet <pipcet <at> protonmail.com> writes:
>
>>> Opportunism: client predicts plenty of idle time, so start full collection.
>>
>> Two full collections would cause a significant slow-down, yes.
>>
>> If you want to continue testing before this is fixed, please consider
>> setting igc-step-interval to 0 (the fixnum, not the float) or 0.0 (which
>> has different behavior because we have a do { ... } while (...) loop).
>
> Setting it to 0 makes hangs somewhat shorter, but they are still there.
> Setting it to 0.0 does not change anything.
Do you still see the "opportunism" messages?
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 18:19:01 GMT)
Full text and
rfc822 format available.
Message #38 received at 76505 <at> debbugs.gnu.org (full text, mbox):
Pip Cet <pipcet <at> protonmail.com> writes:
>> Setting it to 0 makes hangs somewhat shorter, but they are still there.
>> Setting it to 0.0 does not change anything.
>
> Do you still see the "opportunism" messages?
No, both for 0 and 0.0.
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 18:38:01 GMT)
Full text and
rfc822 format available.
Message #41 received at 76505 <at> debbugs.gnu.org (full text, mbox):
> From: Ihor Radchenko <yantar92 <at> posteo.net>
> Cc: pipcet <at> protonmail.com, 76505 <at> debbugs.gnu.org
> Date: Sun, 23 Feb 2025 17:13:22 +0000
>
> > Can you start the cpu profiler just before you run project-compile,
> > and then show the profile (assuming the profiler doesn't cause crashes
> > on the branch in your build)?
>
> See the attached.
Looks like you have auto-revert enabled when compiling? Does
disabling that change anything? In general, having auto-revert in too
many buffers while compiling or using Git is likely to flood Emacs
with file-notification events, and is not recommended.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 18:52:01 GMT)
Full text and
rfc822 format available.
Message #44 received at 76505 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> See the attached.
>
> Looks like you have auto-revert enabled when compiling?
Hmm. Yes, I do. Because `magit-auto-revert-mode' (enabled by default).
I explicitly disable global-auto-revert-mode in my config.
> ... Does
> disabling that change anything?
It completely removes the hangs.
> ... In general, having auto-revert in too
> many buffers while compiling or using Git is likely to flood Emacs
> with file-notification events, and is not recommended.
Well. Magit enables auto-revert by default:
(define-globalized-minor-mode magit-auto-revert-mode auto-revert-mode
magit-turn-on-auto-revert-mode-if-desired
:package-version '(magit . "2.4.0")
:link '(info-link "(magit)Automatic Reverting of File-Visiting Buffers")
:group 'magit-auto-revert
:group 'magit-essentials
;; - When `global-auto-revert-mode' is enabled, then this mode is
;; redundant.
;; - In all other cases enable the mode because if buffers are not
;; automatically reverted that would make many very common tasks
;; much more cumbersome.
:init-value (not (or global-auto-revert-mode
noninteractive)))
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 19:12:02 GMT)
Full text and
rfc822 format available.
Message #47 received at 76505 <at> debbugs.gnu.org (full text, mbox):
> From: Ihor Radchenko <yantar92 <at> posteo.net>
> Cc: pipcet <at> protonmail.com, 76505 <at> debbugs.gnu.org
> Date: Sun, 23 Feb 2025 18:50:20 +0000
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> >> See the attached.
> >
> > Looks like you have auto-revert enabled when compiling?
>
> Hmm. Yes, I do. Because `magit-auto-revert-mode' (enabled by default).
> I explicitly disable global-auto-revert-mode in my config.
>
> > ... Does
> > disabling that change anything?
>
> It completely removes the hangs.
Problem solved.
> > ... In general, having auto-revert in too
> > many buffers while compiling or using Git is likely to flood Emacs
> > with file-notification events, and is not recommended.
>
> Well. Magit enables auto-revert by default:
Bad idea, IMO. At least as long as file notifications are used: those
don't scale well. Try setting auto-revert-use-notify to the nil
value, and see if you can the best of both worlds.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 19:19:02 GMT)
Full text and
rfc822 format available.
Message #50 received at 76505 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> Hmm. Yes, I do. Because `magit-auto-revert-mode' (enabled by default).
>> I explicitly disable global-auto-revert-mode in my config.
>>
>> > ... Does
>> > disabling that change anything?
>>
>> It completely removes the hangs.
>
> Problem solved.
Well. Not really. The question why igc branch is _so much slower_
remains. I never experienced such problems on master.
>> > ... In general, having auto-revert in too
>> > many buffers while compiling or using Git is likely to flood Emacs
>> > with file-notification events, and is not recommended.
>>
>> Well. Magit enables auto-revert by default:
>
> Bad idea, IMO. At least as long as file notifications are used: those
> don't scale well. Try setting auto-revert-use-notify to the nil
> value, and see if you can the best of both worlds.
CCing Jonas. I do not want to open an issue on github or discuss magit
here, but want to let Jonas know about this thread and this
recommendation.
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 19:44:02 GMT)
Full text and
rfc822 format available.
Message #53 received at 76505 <at> debbugs.gnu.org (full text, mbox):
> From: Ihor Radchenko <yantar92 <at> posteo.net>
> Cc: pipcet <at> protonmail.com, 76505 <at> debbugs.gnu.org, Jonas Bernoulli
> <jonas <at> bernoul.li>
> Date: Sun, 23 Feb 2025 19:18:15 +0000
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> >> Hmm. Yes, I do. Because `magit-auto-revert-mode' (enabled by default).
> >> I explicitly disable global-auto-revert-mode in my config.
> >>
> >> > ... Does
> >> > disabling that change anything?
> >>
> >> It completely removes the hangs.
> >
> > Problem solved.
>
> Well. Not really. The question why igc branch is _so much slower_
> remains. I never experienced such problems on master.
Good point. So please try disabling file notification, but keep the
auto-revert thing, and let's see if the hangs return or not. That
will allow us to know whether the notifications or the auto-reverting
triggers whatever causes the hangs.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 19:51:02 GMT)
Full text and
rfc822 format available.
Message #56 received at 76505 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> Well. Not really. The question why igc branch is _so much slower_
>> remains. I never experienced such problems on master.
>
> Good point. So please try disabling file notification, but keep the
> auto-revert thing, and let's see if the hangs return or not. That
> will allow us to know whether the notifications or the auto-reverting
> triggers whatever causes the hangs.
Doing (setopt auto-revert-use-notify nil) while keeping
magit-auto-revert-mode enabled also makes the hangs disappear.
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Sun, 23 Feb 2025 20:37:02 GMT)
Full text and
rfc822 format available.
Message #59 received at 76505 <at> debbugs.gnu.org (full text, mbox):
On Sun, Feb 23 2025, Pip Cet wrote:
> The problem, I think, is that the do { ... } while () loop runs (at
> least) twice: the first time, a busy trace finishes, the code falls
> through, "now" is reset to the *current* clock, which makes
> (availableEnd - now) "negative". But as the types are both unsigned,
> the "negative" value will correspond to a large Clock value, which will
> make MPS reach the conclusion it's a good time to start a full
> collection.
>
>
> I think the intention was for "multiplier" in the above code always to
> be at least 1.0.
Good catch! The wrapping on overflow looks problematic.
> The documentation isn't quite clear on what
> "multiplier" is supposed to be set to, particularly in our case where we
> want to discourage full collections. 1.0 seems to be the lowest safe
> value, so let's use that?
Does MPS make a distinction between full collection and "normal"
collections? Are those somehow non-incremental?
It could be that MPS decides to start a full trace, but it advances the
trace only while (now < intervalEnd).
If "full collections" are just like normnal collections, then I see no
particular reason why starting full traces should be discouraged.
(My guess is that the slowness has something to do with intervals, but I
have zero evidence for that.)
Helmut
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Mon, 24 Feb 2025 03:22:02 GMT)
Full text and
rfc822 format available.
Message #62 received at 76505 <at> debbugs.gnu.org (full text, mbox):
> From: Ihor Radchenko <yantar92 <at> posteo.net>
> Cc: pipcet <at> protonmail.com, 76505 <at> debbugs.gnu.org
> Date: Sun, 23 Feb 2025 19:49:43 +0000
>
> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> >> Well. Not really. The question why igc branch is _so much slower_
> >> remains. I never experienced such problems on master.
> >
> > Good point. So please try disabling file notification, but keep the
> > auto-revert thing, and let's see if the hangs return or not. That
> > will allow us to know whether the notifications or the auto-reverting
> > triggers whatever causes the hangs.
>
> Doing (setopt auto-revert-use-notify nil) while keeping
> magit-auto-revert-mode enabled also makes the hangs disappear.
Which means file notifications are somehow the trigger. Maybe it's
the Lisp data they cons? Or maybe the fact that they are reported to
Emacs via pselect mechanism?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Mon, 24 Feb 2025 04:14:02 GMT)
Full text and
rfc822 format available.
Message #65 received at 76505 <at> debbugs.gnu.org (full text, mbox):
Pip Cet <pipcet <at> protonmail.com> writes:
> Does that analysis make sense?
It looks like 1.0 would be a sensible value, indeed. Unless someone has
an idea how to predict idle time, of course :-).
👍
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Mon, 24 Feb 2025 18:37:02 GMT)
Full text and
rfc822 format available.
Message #68 received at 76505 <at> debbugs.gnu.org (full text, mbox):
Ihor Radchenko <yantar92 <at> posteo.net> writes:
> M-: (benchmark-progn (igc-collect))
> Elapsed time: 3.086109s
btw, is 3 seconds collection normal?
I am asking because I see few second hangs that are caused by automatic
garbage collection as well (now, when I enabled garbage-collection-messages)
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Mon, 24 Feb 2025 20:37:01 GMT)
Full text and
rfc822 format available.
Message #71 received at 76505 <at> debbugs.gnu.org (full text, mbox):
"Ihor Radchenko" <yantar92 <at> posteo.net> writes:
> Ihor Radchenko <yantar92 <at> posteo.net> writes:
>
>> M-: (benchmark-progn (igc-collect))
>> Elapsed time: 3.086109s
>
> btw, is 3 seconds collection normal?
For a full collection, it may be, but that's why we don't want to do
those :-)
> I am asking because I see few second hangs that are caused by automatic
> garbage collection as well (now, when I enabled garbage-collection-messages)
That sounds excessive, even for a large session! In this large session,
I'm certainly not seeing hangs of a few seconds, and a full GC also
takes about 3 seconds.
Are you using the standard generation chain?
I suspect the problem is indeed in the file notification code. I recall
I ended up turning off auto-revert because it broke some scenarios too
badly, but it'd be nice to figure out what's going on there and fix it
properly.
Pip
Severity set to 'minor' from 'normal'
Request was from
Stefan Kangas <stefankangas <at> gmail.com>
to
control <at> debbugs.gnu.org.
(Mon, 24 Feb 2025 21:50:04 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Tue, 25 Feb 2025 09:16:02 GMT)
Full text and
rfc822 format available.
Message #76 received at submit <at> debbugs.gnu.org (full text, mbox):
Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text
editors" <bug-gnu-emacs <at> gnu.org> writes:
Hi,
> I suspect the problem is indeed in the file notification code. I recall
> I ended up turning off auto-revert because it broke some scenarios too
> badly, but it'd be nice to figure out what's going on there and fix it
> properly.
There is a new macro 'inhibit-auto-revert'. If there are some critical
operations, which could trigger heavy auto-revert actions, they can by
wrapped by this macro.
> Pip
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Tue, 25 Feb 2025 09:16:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76505; Package
emacs.
(Tue, 25 Feb 2025 18:49:01 GMT)
Full text and
rfc822 format available.
Message #82 received at 76505 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Pip Cet <pipcet <at> protonmail.com> writes:
>>> M-: (benchmark-progn (igc-collect))
>>> Elapsed time: 3.086109s
>>
>> btw, is 3 seconds collection normal?
>
> For a full collection, it may be, but that's why we don't want to do
> those :-)
With the traditional GC, I have <1sec collection times.
See the attached.
In general, most GC durations for Emacs users is less than 3 seconds for
our traditional GC.
See another attached :)
>> I am asking because I see few second hangs that are caused by automatic
>> garbage collection as well (now, when I enabled garbage-collection-messages)
>
> That sounds excessive, even for a large session! In this large session,
> I'm certainly not seeing hangs of a few seconds, and a full GC also
> takes about 3 seconds.
>
> Are you using the standard generation chain?
I have no idea what you are talking about, so I guess whatever it is, it
should be the default.
> I suspect the problem is indeed in the file notification code. I recall
> I ended up turning off auto-revert because it broke some scenarios too
> badly, but it'd be nice to figure out what's going on there and fix it
> properly.
Anything I can help?
I am now running a different session (latest commit; under gdb), but
when I encounter the same problem again, I can try whatever you ask me
to do.
[0-merged.dat.png (image/png, attachment)]
[init-summary-cum-std.png (image/png, attachment)]
[Message part 4 (text/plain, inline)]
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
This bug report was last modified 109 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.