Subject: bug#75672: 31.0.50; scratch/igc memory usage/collection issues

From: alexis purslane <[email-masked]>
Date: Sun, 19 Jan 2025 17:42:15 +0000

-------- Original Message --------
On 1/19/25 12:28 PM, Pip Cet <[email-masked]> wrote:

>  "Alexis Purslane via \"Bug reports for GNU Emacs, the Swiss army knife of text editors\"" <[email-masked]> writes:
>  
>  > I've been using scratch/igc for the past few days (see details below for
>  > the exact version and situation) and have been having some /interesting/
>  > experiences:
>  
>  Thanks for sharing them!  Please try to catch a freezing Emacs and do
>  not kill it; hopefully, we can then use gdb to extract enough
>  information to let us fix the bug.  Instructions below.
>  
>  >     1. It seems slower than Emacs 29.4 with GCMH (gcmh-idle-timer set to
>  
>  I don't know how gcmh works.
>  
>  >     'auto, so it GCs usually after less than a second of idle time, and
>  >     gcmh-max-cons-threshold set to 200MB so that if there's a very
>  >     intense operation going on that's allocating a lot of memory, it
>  >     will actually GC before it gets out of hand and the idle GC pause
>  >     would freeze Emacs, but most of the time it'll almost never GC
>  >     unless I'm idle) when opening a lot of files (for instance, when
>  >     running org-agenda). About twice as slow. Startup is faster, though.
>  
>  Does "twice as slow" mean that long tasks take Emacs twice as long, or
>  that latency appears to have doubled?
>  
>  feature/igc isn't optimized particularly well right now.  IMHO, it's
>  already more usable than "master", but I care about latency, not CPU
>  usage.
>  
>  I don't know why startup is faster.
>  
>  I'm confident that we'll be able to outperform the traditional GC for
>  some users, and come close for all others.  A forced full GC will
>  probably take longer than an alloc.c GC run (it might not: string
>  compaction isn't efficient, for example).
>  
>  >     2. It uses monotonically more and more memory throughout a session,
>  
>  That's bad.  I thought it was my fault, but I've seen the same thing
>  here.
>  
>  >     even when doing things that shouldn't cause new memory to be
>  >     allocated from the OS, eventually a really, really large amount
>  >     (I've seen 2GB) even though the amount of memory it claims it's
>  
>  Is this memory actually used, or is it virtual memory which was never
>  paged in?  One good way to do that is to create a coredump file from gdb
>  attached to Emacs, which should not kill Emacs.
>  
>  >     using when I run memory-report isn't that large. E.g., right now
>  >     it's using 981 MB (just went up from 700 in the last few minutes
>  >     despite only writing in this buffer the entire time), and
>  >     memory-report says:
>  
>  I'll check whether memory-report does anything useful for the IGC build,
>  right now; if it doesn't, we should fix it.
>  
>  However, M-x igc-stats and M-x igc-roots-stats may provide further
>  insight (use the "s" key to get a snapshot.  Sometimes I have to hit "a"
>  first, but that's possibly local breakage).
>  
>  >     Estimated Emacs Memory Usage
>  >
>  >     73 MiB  Total Buffer Memory Usage
>  >     18 MiB  Memory Used By Global Variables
>  >    9.8 MiB  Memory Used By Symbol Plists
>  >    1.1 MiB  Total Image Cache Size
>  >        0 B  Reserved (But Unused) Object Memory
>  >        0 B  Overall Object Memory Usage
>  
>  That looks negligible.
>  
>  > Object Storage
>  >
>  >        0 B  Strings
>  >        0 B  Vectors
>  >        0 B  Floats
>  >        0 B  Conses
>  >        0 B  Symbols
>  >        0 B  Intervals
>  >        0 B  Buffer-Objects
>  
>  So we're not scanning those at all :-)
>  
>  > Largest Variables
>  >
>  >      2 MiB  load-history
>  >    1.5 MiB  ucs-normalize-hangul-translation-alist
>  >      1 MiB  nerd-icons/mdicon-alist
>  >    746 KiB  easy-menu-converted-items-table
>  >    659 KiB  face--new-frame-defaults
>  >    613 KiB  sly-common-lisp-system-indentation
>  >    565 KiB  undo-equiv-table
>  >    498 KiB  gnus-newsrc-hashtb
>  >    495 KiB  gnus-newsrc-alist
>  >    494 KiB  nnimap-current-infos
>  >    413 KiB  uni-confusable-table
>  >    305 KiB  definition-prefixes
>  >    285 KiB  nerd-icons/faicon-alist
>  >    234 KiB  minor-mode-map-alist
>  >    201 KiB  doom-themes-base-faces
>  >    189 KiB  org-entities
>  >    180 KiB  company-keywords-alist
>  >    157 KiB  common-lisp-hyperspec--symbols
>  >    149 KiB  global-map
>  >    149 KiB  help-quick-use-map
>  
>  Negligible, too.
>  
>  >     3. Eventually, Emacs will freeze irrecoverably (i.e., C-g doesn't
>  >     work), probably trying to GC, causing total lossage of my Emacs
>  >     session and necessitating a restart.
>  
>  Ouch.  It'd be very important if you could catch Emacs in this state.
>  
>  Are you using anything that creates many child processes? If that is the
>  problem, it might be possible to recover from it.
>  
>  If you haven't started Emacs in GDB, the next time it freezes, please
>  try running
>  
>  ps aux | grep emacs ==> determine pid of emacs
>  
>  gdb -p <pid of emacs>
>  
>  (you might need sudo because some people think it's more secure to run
>  gdb as root than it is to allow users to trace their own processes)
>  
>  At the gdb prompt (note that "gcore" will produce a large file; please
>  save it, along with the freezing emacs binary (called "emacs" and its
>  pdump file "emacs.pdmp")):
>  
>  source /path/to/emacs/src/.gdbinit
>  bt full
>  gcore
>  
>  > I may have configured it wrong, or it may be an issue particular to my
>  > system, hence the debug info below.
>  
>  I haven't seen uninterruptible freezes here; I have had to use the magic
>  "C-g C-g C-g" triple-quit once in a while, because magit sometimes seems
>  to take very long.  Just to be sure, triple C-g doesn't fix the problem
>  for you?
>  
>  > In GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
>  >  3.24.43, cairo version 1.18.2) of 2025-01-16 built on fedora
>  > Repository revision: 6185a57afed3ef02b2608e6bc476d117b02c8e81
>  > Repository branch: scratch/igc
>  > System Description: Fedora Linux 41.20241229.0 (Silverblue)
>  >
>  > Configured using:
>  >  'configure
>  >  CPPFLAGS=-I/home/alexispurslane/Development/scratch/emacs/mps/mps-artifacts
>  >  LDFLAGS=-L/home/alexispurslane/Development/scratch/emacs/mps/mps-artifacts
>  >  --with-cairo --with-dbus --with-gif --with-gpm=no --with-harfbuzz
>  >  --with-jpeg --with-modules --with-native-compilation=aot --with-png
>  >  --with-rsvg --with-sqlite3 --with-tiff --with-tree-sitter --with-webp
>  >  --with-x-toolkit=gtk3 --with-xinput2 --with-xpm --with-mps=yes
>  >  --with-pgtk --prefix=/var/home/alexispurslane/.local'
>  >
>  > Configured features:
>  > CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG
>  > LIBSELINUX LIBXML2 MODULES MPS NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK
>  > PNG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM
>  > GTK3 ZLIB
>    ^^^^
>  
>  We're working on fixing some GTK memory leaks in bug#75636; it's
>  possible those are partially responsible for growing memory usage.
>  
>  > Features:
>  > (gnus-draft gnus-async shrface embark-org ox-odt rng-loc rng-uri
>  > ...
>  > magit-section cursor-sensor dash compat plz warnings color dns
>    ^^^^^^^^^^^^^
>  
>  I see some magit-related features here, but not the plain "magit"
>  feature.  Is that correct?
>  
>  > Memory information:
>  > ((conses 24 0 0) (symbols 56 0 0) (strings 40 0 0)
>  >  (string-bytes 1 0) (vectors 24 0) (vector-slots 8 0 0)
>  >  (floats 24 0 0) (intervals 64 0 0) (buffers 1000 0))
>  
>  Also something we have to fix.
>  
>  Pip
>  

I didn't realize I could attach GDB to a running emacs process. That's a huge help. I'll do that as soon as I can.