GNU bug report logs - #37006
27.0.50; garbage collection not happening after 26de2d42

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 37006 in the body.
You can then email your comments to 37006 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Joseph Mingrone <jrm <at> ftfl.ca>
To: bug-gnu-emacs <at> gnu.org
Subject: 27.0.50; garbage collection not happening after 26de2d42
Date: Sun, 11 Aug 2019 09:39:58 -0300

After 26de2d42 from 2019-06-20, garbage collection is not happening and
allocated memory continues to grow until

Aug 10 21:39:02 <kern.err> phe kernel: pid 73800 (emacs-27.0.50), uid
1001, was killed: out of swap space

and

Aug 10 21:39:07 <kern.crit> phe kernel: swap_pager_getswapspace(3):
failed

In GNU Emacs 27.0.50 (build 1, amd64-portbld-freebsd12.0, GTK+ Version 3.24.10)
Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
System Description: 12.0-RELEASE-p9

Configured using:
 'configure --disable-build-details --localstatedir=/var --with-x
 --enable-acl --without-cairo --without-dbus --without-gconf --with-gif
 --with-gnutls --without-gsettings --with-x-toolkit=gtk3
 --without-harfbuzz --with-jpeg --with-json
 --with-file-notification=kqueue --with-lcms2 --with-m17n-flt
 --with-imagemagick --with-mailutils --with-modules --with-sound=oss
 --with-libotf --with-png --with-toolkit-scroll-bars --with-rsvg
 --with-threads --with-tiff --with-xft --with-xim --with-xml2 --with-xpm
 --without-xwidgets --x-libraries=/usr/local/lib
 --x-includes=/usr/local/include --prefix=/usr/local
 --mandir=/usr/local/man --disable-silent-rules
 --infodir=/usr/local/share/emacs/info/
 --build=amd64-portbld-freebsd12.0 'CFLAGS=-O2 -pipe
 -fstack-protector-strong -isystem /usr/local/include
 -fno-strict-aliasing ' 'CPPFLAGS=-isystem /usr/local/include' 'LDFLAGS=
 -fstack-protector-strong -L/usr/local/lib ''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GLIB NOTIFY KQUEUE ACL
GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS
GTK3 X11 XDBE XIM MODULES THREADS JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: en_CA.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Message

Minor modes in effect:
  gnus-message-citation-mode: t
  erc-spelling-mode: t
  erc-ring-mode: t
  erc-networks-mode: t
  erc-netsplit-mode: t
  erc-menu-mode: t
  erc-log-mode: t
  erc-list-mode: t
  erc-pcomplete-mode: t
  erc-button-mode: t
  erc-stamp-mode: t
  erc-autojoin-mode: t
  erc-track-mode: t
  erc-track-minor-mode: t
  erc-match-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  mml-mode: t
  shell-dirtrack-mode: t
  flyspell-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  which-key-mode: t
  show-paren-mode: t
  savehist-mode: t
  global-company-mode: t
  company-mode: t
  global-auto-revert-mode: t
  counsel-mode: t
  ivy-mode: t
  cl-old-struct-compat-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  visual-line-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
/home/jrm/.emacs.d/elpa/slime-20190724.1352/slime-autoloads hides /usr/local/share/emacs/27.0.50/site-lisp/slime/slime-autoloads
/home/jrm/.emacs.d/elpa/slime-20190724.1352/slime hides /usr/local/share/emacs/27.0.50/site-lisp/slime/slime
/home/jrm/.emacs.d/elpa/slime-20190724.1352/slime-tests hides /usr/local/share/emacs/27.0.50/site-lisp/slime/slime-tests
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox-org hides /usr/local/share/emacs/27.0.50/lisp/org/ox-org
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-bibtex hides /usr/local/share/emacs/27.0.50/lisp/org/org-bibtex
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-ruby hides /usr/local/share/emacs/27.0.50/lisp/org/ob-ruby
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-table hides /usr/local/share/emacs/27.0.50/lisp/org/ob-table
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-awk hides /usr/local/share/emacs/27.0.50/lisp/org/ob-awk
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-element hides /usr/local/share/emacs/27.0.50/lisp/org/org-element
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-ref hides /usr/local/share/emacs/27.0.50/lisp/org/ob-ref
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-processing hides /usr/local/share/emacs/27.0.50/lisp/org/ob-processing
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-ditaa hides /usr/local/share/emacs/27.0.50/lisp/org/ob-ditaa
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-info hides /usr/local/share/emacs/27.0.50/lisp/org/org-info
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox-md hides /usr/local/share/emacs/27.0.50/lisp/org/ox-md
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-colview hides /usr/local/share/emacs/27.0.50/lisp/org/org-colview
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-mhe hides /usr/local/share/emacs/27.0.50/lisp/org/org-mhe
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-python hides /usr/local/share/emacs/27.0.50/lisp/org/ob-python
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-shell hides /usr/local/share/emacs/27.0.50/lisp/org/ob-shell
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox-ascii hides /usr/local/share/emacs/27.0.50/lisp/org/ox-ascii
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-lisp hides /usr/local/share/emacs/27.0.50/lisp/org/ob-lisp
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-capture hides /usr/local/share/emacs/27.0.50/lisp/org/org-capture
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-js hides /usr/local/share/emacs/27.0.50/lisp/org/ob-js
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-src hides /usr/local/share/emacs/27.0.50/lisp/org/org-src
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-sass hides /usr/local/share/emacs/27.0.50/lisp/org/ob-sass
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-footnote hides /usr/local/share/emacs/27.0.50/lisp/org/org-footnote
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-lob hides /usr/local/share/emacs/27.0.50/lisp/org/ob-lob
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-gnus hides /usr/local/share/emacs/27.0.50/lisp/org/org-gnus
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-picolisp hides /usr/local/share/emacs/27.0.50/lisp/org/ob-picolisp
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-scheme hides /usr/local/share/emacs/27.0.50/lisp/org/ob-scheme
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-mobile hides /usr/local/share/emacs/27.0.50/lisp/org/org-mobile
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-latex hides /usr/local/share/emacs/27.0.50/lisp/org/ob-latex
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-crypt hides /usr/local/share/emacs/27.0.50/lisp/org/org-crypt
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-docview hides /usr/local/share/emacs/27.0.50/lisp/org/org-docview
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob hides /usr/local/share/emacs/27.0.50/lisp/org/ob
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox-icalendar hides /usr/local/share/emacs/27.0.50/lisp/org/ox-icalendar
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-rmail hides /usr/local/share/emacs/27.0.50/lisp/org/org-rmail
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox-publish hides /usr/local/share/emacs/27.0.50/lisp/org/ox-publish
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-sed hides /usr/local/share/emacs/27.0.50/lisp/org/ob-sed
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-datetree hides /usr/local/share/emacs/27.0.50/lisp/org/org-datetree
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-protocol hides /usr/local/share/emacs/27.0.50/lisp/org/org-protocol
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-matlab hides /usr/local/share/emacs/27.0.50/lisp/org/ob-matlab
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-inlinetask hides /usr/local/share/emacs/27.0.50/lisp/org/org-inlinetask
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-shen hides /usr/local/share/emacs/27.0.50/lisp/org/ob-shen
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-core hides /usr/local/share/emacs/27.0.50/lisp/org/ob-core
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-entities hides /usr/local/share/emacs/27.0.50/lisp/org/org-entities
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-ebnf hides /usr/local/share/emacs/27.0.50/lisp/org/ob-ebnf
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-coq hides /usr/local/share/emacs/27.0.50/lisp/org/ob-coq
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-forth hides /usr/local/share/emacs/27.0.50/lisp/org/ob-forth
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox hides /usr/local/share/emacs/27.0.50/lisp/org/ox
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-hledger hides /usr/local/share/emacs/27.0.50/lisp/org/ob-hledger
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-abc hides /usr/local/share/emacs/27.0.50/lisp/org/ob-abc
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org hides /usr/local/share/emacs/27.0.50/lisp/org/org
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-archive hides /usr/local/share/emacs/27.0.50/lisp/org/org-archive
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-J hides /usr/local/share/emacs/27.0.50/lisp/org/ob-J
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-macs hides /usr/local/share/emacs/27.0.50/lisp/org/org-macs
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-version hides /usr/local/share/emacs/27.0.50/lisp/org/org-version
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-exp hides /usr/local/share/emacs/27.0.50/lisp/org/ob-exp
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-ctags hides /usr/local/share/emacs/27.0.50/lisp/org/org-ctags
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-java hides /usr/local/share/emacs/27.0.50/lisp/org/ob-java
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-sql hides /usr/local/share/emacs/27.0.50/lisp/org/ob-sql
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-lint hides /usr/local/share/emacs/27.0.50/lisp/org/org-lint
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-keys hides /usr/local/share/emacs/27.0.50/lisp/org/ob-keys
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-eshell hides /usr/local/share/emacs/27.0.50/lisp/org/org-eshell
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-plantuml hides /usr/local/share/emacs/27.0.50/lisp/org/ob-plantuml
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-ocaml hides /usr/local/share/emacs/27.0.50/lisp/org/ob-ocaml
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-sqlite hides /usr/local/share/emacs/27.0.50/lisp/org/ob-sqlite
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-list hides /usr/local/share/emacs/27.0.50/lisp/org/org-list
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-lua hides /usr/local/share/emacs/27.0.50/lisp/org/ob-lua
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-C hides /usr/local/share/emacs/27.0.50/lisp/org/ob-C
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-habit hides /usr/local/share/emacs/27.0.50/lisp/org/org-habit
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-bbdb hides /usr/local/share/emacs/27.0.50/lisp/org/org-bbdb
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-lilypond hides /usr/local/share/emacs/27.0.50/lisp/org/ob-lilypond
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox-latex hides /usr/local/share/emacs/27.0.50/lisp/org/ox-latex
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-asymptote hides /usr/local/share/emacs/27.0.50/lisp/org/ob-asymptote
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-groovy hides /usr/local/share/emacs/27.0.50/lisp/org/ob-groovy
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-screen hides /usr/local/share/emacs/27.0.50/lisp/org/ob-screen
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-org hides /usr/local/share/emacs/27.0.50/lisp/org/ob-org
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-eww hides /usr/local/share/emacs/27.0.50/lisp/org/org-eww
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-octave hides /usr/local/share/emacs/27.0.50/lisp/org/ob-octave
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-vala hides /usr/local/share/emacs/27.0.50/lisp/org/ob-vala
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-indent hides /usr/local/share/emacs/27.0.50/lisp/org/org-indent
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-macro hides /usr/local/share/emacs/27.0.50/lisp/org/org-macro
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-gnuplot hides /usr/local/share/emacs/27.0.50/lisp/org/ob-gnuplot
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-haskell hides /usr/local/share/emacs/27.0.50/lisp/org/ob-haskell
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-dot hides /usr/local/share/emacs/27.0.50/lisp/org/ob-dot
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox-html hides /usr/local/share/emacs/27.0.50/lisp/org/ox-html
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-calc hides /usr/local/share/emacs/27.0.50/lisp/org/ob-calc
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-fortran hides /usr/local/share/emacs/27.0.50/lisp/org/ob-fortran
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-stan hides /usr/local/share/emacs/27.0.50/lisp/org/ob-stan
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-ledger hides /usr/local/share/emacs/27.0.50/lisp/org/ob-ledger
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-io hides /usr/local/share/emacs/27.0.50/lisp/org/ob-io
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-install hides /usr/local/share/emacs/27.0.50/lisp/org/org-install
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-clock hides /usr/local/share/emacs/27.0.50/lisp/org/org-clock
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-agenda hides /usr/local/share/emacs/27.0.50/lisp/org/org-agenda
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-table hides /usr/local/share/emacs/27.0.50/lisp/org/org-table
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-attach hides /usr/local/share/emacs/27.0.50/lisp/org/org-attach
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-faces hides /usr/local/share/emacs/27.0.50/lisp/org/org-faces
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox-beamer hides /usr/local/share/emacs/27.0.50/lisp/org/ox-beamer
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-comint hides /usr/local/share/emacs/27.0.50/lisp/org/ob-comint
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-mscgen hides /usr/local/share/emacs/27.0.50/lisp/org/ob-mscgen
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-w3m hides /usr/local/share/emacs/27.0.50/lisp/org/org-w3m
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-maxima hides /usr/local/share/emacs/27.0.50/lisp/org/ob-maxima
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-css hides /usr/local/share/emacs/27.0.50/lisp/org/ob-css
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-compat hides /usr/local/share/emacs/27.0.50/lisp/org/org-compat
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-feed hides /usr/local/share/emacs/27.0.50/lisp/org/org-feed
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-irc hides /usr/local/share/emacs/27.0.50/lisp/org/org-irc
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-clojure hides /usr/local/share/emacs/27.0.50/lisp/org/ob-clojure
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox-odt hides /usr/local/share/emacs/27.0.50/lisp/org/ox-odt
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox-man hides /usr/local/share/emacs/27.0.50/lisp/org/ox-man
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-emacs-lisp hides /usr/local/share/emacs/27.0.50/lisp/org/ob-emacs-lisp
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-tangle hides /usr/local/share/emacs/27.0.50/lisp/org/ob-tangle
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-R hides /usr/local/share/emacs/27.0.50/lisp/org/ob-R
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-makefile hides /usr/local/share/emacs/27.0.50/lisp/org/ob-makefile
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-duration hides /usr/local/share/emacs/27.0.50/lisp/org/org-duration
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ox-texinfo hides /usr/local/share/emacs/27.0.50/lisp/org/ox-texinfo
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-id hides /usr/local/share/emacs/27.0.50/lisp/org/org-id
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-eval hides /usr/local/share/emacs/27.0.50/lisp/org/ob-eval
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-timer hides /usr/local/share/emacs/27.0.50/lisp/org/org-timer
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-mouse hides /usr/local/share/emacs/27.0.50/lisp/org/org-mouse
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-pcomplete hides /usr/local/share/emacs/27.0.50/lisp/org/org-pcomplete
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-loaddefs hides /usr/local/share/emacs/27.0.50/lisp/org/org-loaddefs
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/org-plot hides /usr/local/share/emacs/27.0.50/lisp/org/org-plot
/home/jrm/.emacs.d/elpa/org-plus-contrib-20190805/ob-perl hides /usr/local/share/emacs/27.0.50/lisp/org/ob-perl
/home/jrm/.emacs.d/elpa/let-alist-1.0.6/let-alist hides /usr/local/share/emacs/27.0.50/lisp/emacs-lisp/let-alist

Features:
(shadow emacsbug bbdb-message sendmail nnir bbdb-mua bbdb-com crm sort
gnus-cite mail-extr gnus-async gnus-bcklg qp gnus-ml disp-table
ace-window cl-print help-fns radix-tree make-mode tramp-cache tramp-sh
bookmark mm-archive mule-util url-http url-gw url-cache url-auth url
url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util finder-inf gnutls network-stream nsm erc-spelling
erc-ring erc-networks erc-netsplit erc-menu erc-log erc-list
erc-pcomplete erc-button erc-fill erc-stamp erc-join znc warnings
erc-track erc-match erc-tex easy-mmode erc-goodies erc erc-backend
erc-compat pp erc-loaddefs cl hl-line gnus-topic nndraft nnmh nnagent
nnml gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg
gnus-art mm-uu mml2015 mm-view mml-smime smime dig mailcap nntp
gnus-cache gnus-sum gnus-group gnus-undo gnus-start gnus-cloud nnimap
nnmail mail-source utf7 netrc nnoo gnus-spec gnus-int gnus-range message
rmc puny rfc822 mml mml-sec epa derived epg mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader gnus-win
gnus nnheader wid-edit gnus-util rmail rmail-loaddefs rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils text-property-search time-date
tramp tramp-loaddefs trampver tramp-integration files-x tramp-compat
shell pcomplete parse-time ls-lisp format-spec rx server flyspell ispell
undo-tree diff smart-mode-line-dark-theme smart-mode-line rich-minority
s pdf-tools-loaddefs misc hydra lv ace-link avy bbdb bbdb-site timezone
company-oddmuse company-keywords company-etags etags fileloop generator
company-gtags company-dabbrev-code company-dabbrev company-files
company-capf company-cmake company-xcode company-clang company-semantic
company-eclim company-template company-bbdb which-key paren savehist
company edmacro kmacro pcase autorevert filenotify counsel xdg xref
project dired dired-loaddefs compile comint ansi-color swiper cl-extra
help-mode ivy delsel ring colir color ivy-overlay ffap thingatpt
cus-start cus-load benchmark-init benchmark-init-loaddefs tex-site
advice slime-autoloads info package easymenu epg-config url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json subr-x map url-vars seq byte-opt gv bytecomp
byte-compile cconv cl-loaddefs cl-lib tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads kqueue lcms2 dynamic-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 4672710 1249112)
 (symbols 48 33763 2)
 (strings 32 177500 153301)
 (string-bytes 1 6559318)
 (vectors 16 108474)
 (vector-slots 8 4931122 2331820)
 (floats 8 565 3210)
 (intervals 56 12670 13121)
 (buffers 992 90))
<#secure method=pgpmime mode=sign>

Message #8 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Joseph Mingrone <jrm <at> ftfl.ca>
Cc: 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50;
 garbage collection not happening after 26de2d42
Date: Sun, 11 Aug 2019 18:13:07 +0300

> From: Joseph Mingrone <jrm <at> ftfl.ca>
> Date: Sun, 11 Aug 2019 09:39:58 -0300
> 
> After 26de2d42 from 2019-06-20, garbage collection is not happening and
> allocated memory continues to grow until
> 
> Aug 10 21:39:02 <kern.err> phe kernel: pid 73800 (emacs-27.0.50), uid
> 1001, was killed: out of swap space
> 
> and
> 
> Aug 10 21:39:07 <kern.crit> phe kernel: swap_pager_getswapspace(3):
> failed

I see this on GNU/Linux, but not on MS-Windows, FWIW.

Message #13 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: jrm <at> ftfl.ca
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Sun, 11 Aug 2019 18:23:28 +0200

[Message part 1 (text/plain, inline)]

Observed on macOS as well. Reason: free_cons has the condition

 if (INT_ADD_WRAPV (consing_until_gc, sizeof *ptr, &consing_until_gc))

which will return true (overflow) if consing_until_gc is negative, which is kind of defensible since sizeof is unsigned which causes the sum (consing_until_gc + sizeof *ptr) to be a large unsigned number that doesn't fit into consing_until_gc.

Clang 10 defines __GNUC__ to 4 which causes intprops.h to not use __builtin_add_overflow despite that being present and working.

Casting the sizeof should fix it; patch attached.

[0001-Avoid-unsigned-addend-in-overflow-check-bug-37006.patch (application/octet-stream, attachment)]

Message #16 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: jrm <at> ftfl.ca, eggert <at> cs.ucla.edu, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Sun, 11 Aug 2019 20:07:15 +0300

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Sun, 11 Aug 2019 18:23:28 +0200
> Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
>         37006 <at> debbugs.gnu.org
> 
> Observed on macOS as well. Reason: free_cons has the condition
> 
>  if (INT_ADD_WRAPV (consing_until_gc, sizeof *ptr, &consing_until_gc))
> 
> which will return true (overflow) if consing_until_gc is negative, which is kind of defensible since sizeof is unsigned which causes the sum (consing_until_gc + sizeof *ptr) to be a large unsigned number that doesn't fit into consing_until_gc.
> 
> Clang 10 defines __GNUC__ to 4 which causes intprops.h to not use __builtin_add_overflow despite that being present and working.
> 
> Casting the sizeof should fix it; patch attached.

I believe it should be fixed now.

Message #19 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Joseph Mingrone <jrm <at> ftfl.ca>
Cc: mattiase <at> acm.org, eggert <at> cs.ucla.edu, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Mon, 12 Aug 2019 05:31:18 +0300

> From: Joseph Mingrone <jrm <at> ftfl.ca>
> Cc: Mattias Engdegård <mattiase <at> acm.org>,
>   eggert <at> cs.ucla.edu
> Date: Sun, 11 Aug 2019 17:28:11 -0300
> 
> > Thanks, I fixed this slightly differently, in a way that makes it more
> > explicit why we need a non-trivial code there.  Joseph, please see if
> > the latest master fixes the problem.
> 
> > (IMNSHO, this issue makes INT_ADD_WRAPV and friends unsafe; at the
> > very least this caveat should be prominently documented in Gnulib's
> > intprops.h.)
> 
> I have been running 94644d8 for the past hour or so and resident memory for the Emacs process is up over 1300 MB.  Also with `garbage-collection-messages' set to t, I do not see any messages about garbage collection.

Are you saying that the fix didn't solve the problem for you?  I
definitely saw a lot of GC messages after the fix where I didn't
before.  For example, if you visit xdisp.c from the Emacs sources and
page through it with C-v, don't you see a lot of GC messages?

> I should also add that after my initial report, running 26de2d42, I did eventually start seeing garbage collection messages and the memory usage stopped increasing.  Something must have triggered garbage collection to start again.

After a lot of consing, the GC would come back for a while, until it
would be effectively disabled again by some opportune code path.

If you see no GC messages for a long time, attach a debugger and look
at the value of consing_until_gc.  If its value is huge, around
LONG_MAX, the problem is still not completely solved.

Message #22 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Joseph Mingrone <jrm <at> ftfl.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: mattiase <at> acm.org, eggert <at> cs.ucla.edu, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Mon, 12 Aug 2019 11:34:18 -0300

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Joseph Mingrone <jrm <at> ftfl.ca>
>> Cc: Mattias Engdegård <mattiase <at> acm.org>,
>>   eggert <at> cs.ucla.edu
>> Date: Sun, 11 Aug 2019 17:28:11 -0300

>> > Thanks, I fixed this slightly differently, in a way that makes it more
>> > explicit why we need a non-trivial code there.  Joseph, please see if
>> > the latest master fixes the problem.

>> > (IMNSHO, this issue makes INT_ADD_WRAPV and friends unsafe; at the
>> > very least this caveat should be prominently documented in Gnulib's
>> > intprops.h.)

>> I have been running 94644d8 for the past hour or so and resident
>> memory for the Emacs process is up over 1300 MB.  Also with
>> `garbage-collection-messages' set to t, I do not see any messages
>> about garbage collection.

> Are you saying that the fix didn't solve the problem for you?  I
> definitely saw a lot of GC messages after the fix where I didn't
> before.  For example, if you visit xdisp.c from the Emacs sources and
> page through it with C-v, don't you see a lot of GC messages?

>> I should also add that after my initial report, running 26de2d42, I
>> did eventually start seeing garbage collection messages and the
>> memory usage stopped increasing.  Something must have triggered
>> garbage collection to start again.

> After a lot of consing, the GC would come back for a while, until it
> would be effectively disabled again by some opportune code path.

> If you see no GC messages for a long time, attach a debugger and look
> at the value of consing_until_gc.  If its value is huge, around
> LONG_MAX, the problem is still not completely solved.

The fix did not initially work for me.  I tested a bit more.  With

1. emacs -Q
2. (setq garbage-collection-messages t)
3. page through xdisp.c

I saw lots of garbage collection messages.  But, with my init.el there
were no such messages.  My init.el looked like this.

----------------------------------------------------
(setq gc-cons-threshold most-positive-fixnum)

;; contents of init.el here

(setq gc-cons-threshold 800000) ;; default value
----------------------------------------------------

When I removed the surrounding setqs, garbage collection message were
shown again when paging through xdisp.c.

I assume that temporarily setting `gc-cons-threshold' to a large number
to temporarily prevent garbage collection, then setting it back to a
reasonable value should be acceptable.  Help for `gc-cons-threshold'
says

      By binding this temporarily to a large number, you can effectively
      prevent garbage collection during a part of the program.

--
Joseph

[signature.asc (application/pgp-signature, inline)]

Message #25 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Joseph Mingrone <jrm <at> ftfl.ca>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Mon, 12 Aug 2019 19:49:43 +0300

> From: Joseph Mingrone <jrm <at> ftfl.ca>
> Cc: mattiase <at> acm.org,  eggert <at> cs.ucla.edu,  37006 <at> debbugs.gnu.org
> Date: Mon, 12 Aug 2019 11:34:18 -0300
> 
> The fix did not initially work for me.  I tested a bit more.  With
> 
> 1. emacs -Q
> 2. (setq garbage-collection-messages t)
> 3. page through xdisp.c
> 
> I saw lots of garbage collection messages.  But, with my init.el there
> were no such messages.  My init.el looked like this.
> 
> ----------------------------------------------------
> (setq gc-cons-threshold most-positive-fixnum)
> 
> ;; contents of init.el here
> 
> (setq gc-cons-threshold 800000) ;; default value
> ----------------------------------------------------
> 
> When I removed the surrounding setqs, garbage collection message were
> shown again when paging through xdisp.c.
> 
> I assume that temporarily setting `gc-cons-threshold' to a large number
> to temporarily prevent garbage collection, then setting it back to a
> reasonable value should be acceptable.

Yes, of course.  There's a separate bug in the recent GC-related
changes.  Thanks for pointing this out.

Paul, the current method of updating consing_until_gc only in
garbage_collect_1 isn't workable, because it doesn't support the (very
popular nowadays) paradigm of temporarily setting gc-cons-threshold
very high: doing so avoids calling garbage_collect_1, and thus the
change of the threshold back to a lower value is never seen.  We must
have something in maybe_gc to notice the change and recompute the
threshold.  We must also notice the memory-full condition there.

We need to fix this ASAP, please.

I also don't think I understand the details of the threshold
calculations:

  if (!NILP (Vmemory_full))
    consing_until_gc = memory_full_cons_threshold;
  else
    {
      intptr_t threshold = min (max (GC_DEFAULT_THRESHOLD,
				     gc_cons_threshold >> 3),
				OBJECT_CT_MAX);
      if (FLOATP (Vgc_cons_percentage))
	{
	  double tot = (XFLOAT_DATA (Vgc_cons_percentage)
			* total_bytes_of_live_objects ());
	  if (threshold < tot)
	    {
	      if (tot < OBJECT_CT_MAX)
		threshold = tot;
	      else
		threshold = OBJECT_CT_MAX;
	    }
	}
      consing_until_gc = threshold;
    }

First, gc_cons_threshold is an EMACS_INT, so putting its value into
intptr_t is wrong in 32-bit builds --with-wide-int, right?  For the
same reason, using intptr_t for OBJECT_CT_MAX is wrong in such a
build.

And second, why does the code divide gc_cons_threshold by 8?  If the
value of gc_cons_threshold is most-positive-fixnum, that is wrong, I
think.  Did you mean to divide GC_DEFAULT_THRESHOLD instead?

Message #28 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Joseph Mingrone <jrm <at> ftfl.ca>, Paul Eggert <eggert <at> cs.ucla.edu>,
 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Mon, 12 Aug 2019 19:00:37 +0200

12 aug. 2019 kl. 18.49 skrev Eli Zaretskii <eliz <at> gnu.org>:

> [...] We must
> have something in maybe_gc to notice the change and recompute the
> threshold.  We must also notice the memory-full condition there.

What about biasing consing_until_gc by gc_cons_threshold, and change the condition in maybe_gc to (the moral equivalent of)

 if (consing_until_gc < gc_cons_threshold)

? It is practically as cheap as the current test against 0.

Message #31 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: jrm <at> ftfl.ca, eggert <at> cs.ucla.edu, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Tue, 13 Aug 2019 18:37:42 +0300

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Mon, 12 Aug 2019 19:00:37 +0200
> Cc: Joseph Mingrone <jrm <at> ftfl.ca>, Paul Eggert <eggert <at> cs.ucla.edu>,
>         37006 <at> debbugs.gnu.org
> 
> What about biasing consing_until_gc by gc_cons_threshold, and change the condition in maybe_gc to (the moral equivalent of)
> 
>  if (consing_until_gc < gc_cons_threshold)
> 
> ? It is practically as cheap as the current test against 0.

Yes, but the full calculation of the threshold is more complicated
than that.  For starters, how do you handle gc_cons_threshold values
that are smaller than GC_DEFAULT_THRESHOLD / 10 under your proposal?
There are use cases where the value was below that before and is above
now, or the other way around, or was below and stays below.

And that's even before we consider other complications: when
memory-full is non-nil, we should use a different threshold; and what
about user changes to gc-cons-percentage?

Message #34 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jrm <at> ftfl.ca, eggert <at> cs.ucla.edu, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Tue, 13 Aug 2019 18:48:05 +0200

13 aug. 2019 kl. 17.37 skrev Eli Zaretskii <eliz <at> gnu.org>:
> 
> Yes, but the full calculation of the threshold is more complicated
> than that.  For starters, how do you handle gc_cons_threshold values
> that are smaller than GC_DEFAULT_THRESHOLD / 10 under your proposal?
> There are use cases where the value was below that before and is above
> now, or the other way around, or was below and stays below.

If a change to gc_cons_threshold has us end up in garbage_collect too soon, we can just adjust the bias and consing_until_gc and continue; the cost for doing so is small and amortised. Conversely, if the user raises gc_cons_threshold beyond the limit (OBJECT_CT_MAX), the intention is clearly to inhibit GC anyway.

> And that's even before we consider other complications: when
> memory-full is non-nil, we should use a different threshold; and what
> about user changes to gc-cons-percentage?

The check for memory-full was already eliminated from maybe_gc by changing consing_until_gc when that condition occurs, which seems reasonable enough. Regarding changes gc-cons-percentage, the effect will just be delayed to next GC --- is this really harmful?

I could be wrong about all this; I'm a bit confused by the threshold computation, too. However, Paul's consolidation of conditions in the hot and inlined maybe_gc makes eminently sense to me.

Message #37 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: jrm <at> ftfl.ca, eggert <at> cs.ucla.edu, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Tue, 13 Aug 2019 20:04:13 +0300

> From: Mattias Engdegård <mattiase <at> acm.org>
> Date: Tue, 13 Aug 2019 18:48:05 +0200
> Cc: jrm <at> ftfl.ca, eggert <at> cs.ucla.edu, 37006 <at> debbugs.gnu.org
> 
> > Yes, but the full calculation of the threshold is more complicated
> > than that.  For starters, how do you handle gc_cons_threshold values
> > that are smaller than GC_DEFAULT_THRESHOLD / 10 under your proposal?
> > There are use cases where the value was below that before and is above
> > now, or the other way around, or was below and stays below.
> 
> If a change to gc_cons_threshold has us end up in garbage_collect too soon, we can just adjust the bias and consing_until_gc and continue; the cost for doing so is small and amortised. Conversely, if the user raises gc_cons_threshold beyond the limit (OBJECT_CT_MAX), the intention is clearly to inhibit GC anyway.
> 
> > And that's even before we consider other complications: when
> > memory-full is non-nil, we should use a different threshold; and what
> > about user changes to gc-cons-percentage?
> 
> The check for memory-full was already eliminated from maybe_gc by changing consing_until_gc when that condition occurs, which seems reasonable enough. Regarding changes gc-cons-percentage, the effect will just be delayed to next GC --- is this really harmful?

IOW, you think that whatever this changes in user-facing behavior is
unimportant, and we shouldn't be bothered about it.  I happen to
disagree.

> I could be wrong about all this; I'm a bit confused by the threshold computation, too. However, Paul's consolidation of conditions in the hot and inlined maybe_gc makes eminently sense to me.

I cannot say the same thing myself, sorry.  Making a frequent
operation faster definitely makes sense, even if the speedup is minor.
But doing that and as result breaking use cases that worked for years,
or even just changing their effects in a tangible manner? what's our
excuse for that?

Message #40 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>, Joseph Mingrone <jrm <at> ftfl.ca>
Cc: mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Tue, 13 Aug 2019 10:21:51 -0700

[Message part 1 (text/plain, inline)]

Eli Zaretskii wrote:

> We must
> have something in maybe_gc to notice the change and recompute the
> threshold.

I don't see why the threshold needs to be recomputed on each maybe_gc call. I 
suppose we could add a new builtin variable type, so that the threshold could be 
recomputed whenever GC-related builtin variables are changed; that should do the 
trick without slowing down maybe_gc. But is it that important to recalculate the 
GC threshold immediately? Variables like gc-cons-threshold aren't intended for 
fine-grained control over exactly when the GC is called; they're merely advice.

> We must also notice the memory-full condition there.

memory_full already does that, no? It sets consing_until_gc. Or are you thinking 
of some other memory-full condition?

>    if (!NILP (Vmemory_full))
>      consing_until_gc = memory_full_cons_threshold;
>    else
>      {
>        intptr_t threshold = min (max (GC_DEFAULT_THRESHOLD,
> 				     gc_cons_threshold >> 3),
> 				OBJECT_CT_MAX);
>        if (FLOATP (Vgc_cons_percentage))
> 	{
> 	  double tot = (XFLOAT_DATA (Vgc_cons_percentage)
> 			* total_bytes_of_live_objects ());
> 	  if (threshold < tot)
> 	    {
> 	      if (tot < OBJECT_CT_MAX)
> 		threshold = tot;
> 	      else
> 		threshold = OBJECT_CT_MAX;
> 	    }
> 	}
>        consing_until_gc = threshold;
>      }
> 
> First, gc_cons_threshold is an EMACS_INT, so putting its value into
> intptr_t is wrong in 32-bit builds --with-wide-int, right?

That's not a problem, since the above code does min (..., OBJECT_CT_MAX) on the 
result before storing it into intptr_t.

> using intptr_t for OBJECT_CT_MAX is wrong in such a build.

I don't see why it's wrong. The idea is to count the total number of object 
bytes in use. This cannot exceed the number of bytes addressable by pointers 
regardless of the width of EMACS_INT, so intptr_t is appropriate for such counts.
> And second, why does the code divide gc_cons_threshold by 8?  If the
> value of gc_cons_threshold is most-positive-fixnum, that is wrong, I
> think.  Did you mean to divide GC_DEFAULT_THRESHOLD instead?

Right you are; that's a typo. Thanks. I fixed that in master in the attached patch.

[0001-Fix-GC-threshold-typo.patch (text/x-patch, attachment)]

Message #43 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jrm <at> ftfl.ca, eggert <at> cs.ucla.edu, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Tue, 13 Aug 2019 19:29:07 +0200

13 aug. 2019 kl. 19.04 skrev Eli Zaretskii <eliz <at> gnu.org>:
> 
>> Regarding changes gc-cons-percentage, the effect will just be delayed to next GC --- is this really harmful?
> 
> IOW, you think that whatever this changes in user-facing behavior is
> unimportant, and we shouldn't be bothered about it.  I happen to
> disagree.

In contrast to gc-cons-threshold, we never really promised the effect of gc-cons-percentage to be immediate, and it seems unlikely that anyone would use the latter variable in such a way.

Message #46 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Tue, 13 Aug 2019 20:53:38 +0300

> Cc: mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Tue, 13 Aug 2019 10:21:51 -0700
> 
> > We must
> > have something in maybe_gc to notice the change and recompute the
> > threshold.
> 
> I don't see why the threshold needs to be recomputed on each maybe_gc call. I 
> suppose we could add a new builtin variable type, so that the threshold could be 
> recomputed whenever GC-related builtin variables are changed; that should do the 
> trick without slowing down maybe_gc.

I don't think I understand what this proposal means in practice.  Can
you elaborate, or show an example?

> But is it that important to recalculate the GC threshold
> immediately?

How else would you succeed in reacting to the change "soon enough"?
In the use case which triggered this bug report, setting
gc-cons-threshold to a very large number disables GC for an extremely
long time, and all that time the gc-cons-threshold changes made by the
user are not acted upon.

IOW, we could perhaps explain away a delay of seconds in acting upon
the change, but we surely cannot explain away a delay of hours, let
alone days.

> Variables like gc-cons-threshold aren't intended for fine-grained
> control over exactly when the GC is called; they're merely advice.

Yes, but abrupt changes, like to most-positive-fixnum and then back to
much smaller values, should produce at least approximately the
expected behavior.  In particular, changing from most-positive-fixnum
to a value like 800000 should cause a GC "soon enough".  If we don't
test the value inside maybe_gc, what alternative mechanisms would you
suggest to produce such an effect?

> > We must also notice the memory-full condition there.
> 
> memory_full already does that, no? It sets consing_until_gc.

It sets it to a positive value, so no immediate GC will follow.  The
original code was setting the threshold to a very small value, so GC
would happen immediately.  I think the code in memory_full which sets
consing_until_gc should be amended to (a) not raise consing_until_gc
if the current value is already below memory_full_cons_threshold, and
(b) probably even set it to the negative of sizeof (struct cons_block)
so as to cause an immediate GC.

> > First, gc_cons_threshold is an EMACS_INT, so putting its value into
> > intptr_t is wrong in 32-bit builds --with-wide-int, right?
> 
> That's not a problem, since the above code does min (..., OBJECT_CT_MAX) on the 
> result before storing it into intptr_t.

??? But that effectively disallows/ignores values of gc-cons-threshold
above LONG_MAX.  Why would we make such a backward-incompatible change?
When the user sets the value of that variable, the variable should
keep its value, and we should be able to compare the threshold with
the value of the user variable.  If nothing else, arbitrarily throwing
away high-order bits of the value is a sure path towards bugs due to
confusion between these two value ranges.

> > using intptr_t for OBJECT_CT_MAX is wrong in such a build.
> 
> I don't see why it's wrong.

Because OBJECT_CT_MAX should have the value EMACS_INT_MAX.

Message #49 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Tue, 13 Aug 2019 12:32:24 -0700

[Message part 1 (text/plain, inline)]

Eli Zaretskii wrote:

> OBJECT_CT_MAX should have the value EMACS_INT_MAX.

Not if EMACS_INT_MAX < INTPTR_MAX, since object counts might overflow in that 
case. However, I take your point that consing_until_gc can easily be made to 
hold any fixnum value, so I installed the first attached patch. This is to some 
extent overkill, since these variables should not be assumed to have this sort 
of fine-grained control, but the change is tiny so should be fine.

Come to think of it, the limit should be INTMAX_MAX not EMACS_INT_MAX since 
gc-cons-threshold could exceed EMACS_INT_MAX. So I installed the second attached 
patch to do that.

>> I don't see why the threshold needs to be recomputed on each maybe_gc call. I
>> suppose we could add a new builtin variable type, so that the threshold could be
>> recomputed whenever GC-related builtin variables are changed; that should do the
>> trick without slowing down maybe_gc.
> 
> I don't think I understand what this proposal means in practice.  Can
> you elaborate, or show an example?

The idea would be to have a type that is like struct Lisp_Objfwd but with an 
extra member, a function to be called whenever the variable is accessed. (Or 
perhaps two extra members, a getter and a setter.) This could be useful for 
other builtin vars, I suspect.

> How else would you succeed in reacting to the change "soon enough"?

There are other possibilities. We could have a timer, for example.
>>> We must also notice the memory-full condition there.
>>
>> memory_full already does that, no? It sets consing_until_gc.
> 
> It sets it to a positive value, so no immediate GC will follow.  The
> original code was setting the threshold to a very small value, so GC
> would happen immediately.

Are you talking about the change in commit 
2019-07-20T02:40:03Z!eggert <at> cs.ucla.edu 
(26de2d42d0460c5b193456950a568cb04a29dc00)? If so, I'm not quite following, as 
the old code did not GC until consing_since_gc > memory_full_cons_threshold. I 
expect that the idea was to not thrash doing GCs when memory is full.

I think the code in memory_full which sets
> consing_until_gc should be amended to (a) not raise consing_until_gc
> if the current value is already below memory_full_cons_threshold, and
> (b) probably even set it to the negative of sizeof (struct cons_block)
> so as to cause an immediate GC.

Immediate-GC might cause GC thrashing, no? But (a) makes sense so I installed 
the third attached patch.

[0001-Let-consing_until_gc-exceed-INTPTR_MAX.patch (text/x-patch, attachment)]

[0002-Let-consing_until_gc-exceed-EMACS_INT_MAX.patch (text/x-patch, attachment)]

[0003-Don-t-increase-consing_until_gc-when-out-of-memory.patch (text/x-patch, attachment)]

Message #52 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Wed, 14 Aug 2019 19:06:49 +0300

> Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Tue, 13 Aug 2019 12:32:24 -0700
> 
> > OBJECT_CT_MAX should have the value EMACS_INT_MAX.
> 
> Not if EMACS_INT_MAX < INTPTR_MAX, since object counts might overflow in that 
> case. However, I take your point that consing_until_gc can easily be made to 
> hold any fixnum value, so I installed the first attached patch. This is to some 
> extent overkill, since these variables should not be assumed to have this sort 
> of fine-grained control, but the change is tiny so should be fine.

Thanks.

However, I'd rather we don't invent new data types unless really
necessary.  I think we should simply use EMACS_INT (see below), but
even if we end up using intptr_max, let's just use that directly, not
introduce yet another type which we will have to look up every time we
read this code.  And likewise with the corresponding _MAX value.
Using a non-standard data type makes the code harder to read.

> Come to think of it, the limit should be INTMAX_MAX not EMACS_INT_MAX since 
> gc-cons-threshold could exceed EMACS_INT_MAX.

Sorry, I don't think I follow.  gc-cons-threshold is a Lisp integer, a
fixnum, so it cannot exceed EMACS_INT_MAX, I think.

> The idea would be to have a type that is like struct Lisp_Objfwd but with an 
> extra member, a function to be called whenever the variable is accessed. (Or 
> perhaps two extra members, a getter and a setter.) This could be useful for 
> other builtin vars, I suspect.

Ah, okay.  Can we use for this purpose the existing trapped_write
field of Lisp_Symbol that is the base for implementing Lisp watcher
functions?

> > How else would you succeed in reacting to the change "soon enough"?
> 
> There are other possibilities. We could have a timer, for example.

I don't think timers are reliable enough, as they can be deferred for
arbitrarily long time interval by some Lisp that takes a long time to
finish.

> >>> We must also notice the memory-full condition there.
> >>
> >> memory_full already does that, no? It sets consing_until_gc.
> > 
> > It sets it to a positive value, so no immediate GC will follow.  The
> > original code was setting the threshold to a very small value, so GC
> > would happen immediately.
> 
> Are you talking about the change in commit 
> 2019-07-20T02:40:03Z!eggert <at> cs.ucla.edu 
> (26de2d42d0460c5b193456950a568cb04a29dc00)? If so, I'm not quite following, as 
> the old code did not GC until consing_since_gc > memory_full_cons_threshold. I 
> expect that the idea was to not thrash doing GCs when memory is full.

With the old code, whenever memory-full was non-nil, and
consing_since_gc was more than the size of cons_block (about 1KB on my
system), the very next maybe_gc call would actually trigger GC.  With
the new code, no matter how much consing happened before memory-full
became non-nil, we still need to cons 1KB worth of objects before GC
happens.  This 1KB might be critical when we are out of memory.

> Immediate-GC might cause GC thrashing, no?

Not sure how, can you elaborate?

Message #55 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Wed, 14 Aug 2019 18:37:44 -0700

Eli Zaretskii wrote:

> However, I'd rather we don't invent new data types unless really
> necessary.

I did that yesterday, in commit 2019-08-13T19:20:40Z!eggert <at> cs.ucla.edu 
(b80559be212292d44ce14ca5e94505cab4d9a868).

> gc-cons-threshold is a Lisp integer, a
> fixnum, so it cannot exceed EMACS_INT_MAX, I think.

No, (setq gc-cons-threshold (1+ most-positive-fixnum)) works and does the right 
thing. The variable's value can be any intmax_t value. This is useful for 
quantities like GC object byte counts that might not fit into fixnums.

> Can we use for this purpose the existing trapped_write
> field of Lisp_Symbol that is the base for implementing Lisp watcher
> functions?

Don't see why not.

> With the old code, whenever memory-full was non-nil, and
> consing_since_gc was more than the size of cons_block (about 1KB on my
> system), the very next maybe_gc call would actually trigger GC.  With
> the new code, no matter how much consing happened before memory-full
> became non-nil, we still need to cons 1KB worth of objects before GC
> happens.  This 1KB might be critical when we are out of memory.

I don't think the scenario is worth worrying about doing a GC now rather than 
later. But if we go the trapped_write route, this issue won't matter since the 
GC will be done quickly.

>> Immediate-GC might cause GC thrashing, no?
> 
> Not sure how, can you elaborate?

When EMacs is low on memory, if we're not careful Emacs could GC every time 
maybe_gc is called, which will be roughly equivalent to Emacs hanging and doing 
nothing.

Message #58 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Thu, 15 Aug 2019 17:17:43 +0300

> Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Wed, 14 Aug 2019 18:37:44 -0700
> 
> > However, I'd rather we don't invent new data types unless really
> > necessary.
> 
> I did that yesterday, in commit 2019-08-13T19:20:40Z!eggert <at> cs.ucla.edu 
> (b80559be212292d44ce14ca5e94505cab4d9a868).

OK, but I still hope we will switch to EMACS_INT instead.

> > gc-cons-threshold is a Lisp integer, a
> > fixnum, so it cannot exceed EMACS_INT_MAX, I think.
> 
> No, (setq gc-cons-threshold (1+ most-positive-fixnum)) works and does the right 
> thing.

That makes gc-cons-threshold a bignum, right?  I don't see why we
should support such large values of the threshold, we never did
before.  most-positive-fixnum on 32-bit systems is large enough for
every practical purpose.  Also, supporting the full 32 bits (and 64
bits in 64-bit builds) will also allow contradictory situation whereby
gc-cons-threshold is higher than what we say should be used to disable
GC.

> The variable's value can be any intmax_t value. This is useful for 
> quantities like GC object byte counts that might not fit into fixnums.

Why do we need to talk about how many objects are there?  GC threshold
is not about counting objects, it's about deciding when to GC.

> > Can we use for this purpose the existing trapped_write
> > field of Lisp_Symbol that is the base for implementing Lisp watcher
> > functions?
> 
> Don't see why not.

Are you working on that, or should someone else do it?

> > With the old code, whenever memory-full was non-nil, and
> > consing_since_gc was more than the size of cons_block (about 1KB on my
> > system), the very next maybe_gc call would actually trigger GC.  With
> > the new code, no matter how much consing happened before memory-full
> > became non-nil, we still need to cons 1KB worth of objects before GC
> > happens.  This 1KB might be critical when we are out of memory.
> 
> I don't think the scenario is worth worrying about doing a GC now rather than 
> later. But if we go the trapped_write route, this issue won't matter since the 
> GC will be done quickly.

I don't think I understand how trapped writes could help in the case
of memory-full, since that situation happens independently of setting
the value of gc-cons-threshold.

> >> Immediate-GC might cause GC thrashing, no?
> > 
> > Not sure how, can you elaborate?
> 
> When EMacs is low on memory, if we're not careful Emacs could GC every time 
> maybe_gc is called, which will be roughly equivalent to Emacs hanging and doing 
> nothing.

Right, but that's not what I proposed.  I proposed to trigger an
immediate GC only the first time we detect memory-full.  Thereafter,
the threshold will be set to 1KB, which should prevent thrashing.

Message #61 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Thu, 15 Aug 2019 11:51:06 -0700

Eli Zaretskii wrote:

> That makes gc-cons-threshold a bignum, right?

If the user sets it to a bignum, yes. Ordinarily it's not.

> most-positive-fixnum on 32-bit systems is large enough for
> every practical purpose.

It's not that hard for the number of consed bytes to exceed most-positive-fixnum 
on a 32-bit Emacs. Here's a simple test case to illustrate the phenomenon:

(let* ((cons-size (car (cdr (car (garbage-collect)))))
       (long-length (1+ (/ most-positive-fixnum cons-size)))
       (l (make-list long-length nil)))
  (cons most-positive-fixnum (* cons-size (length l))))

This yielded (536870911 . 536870912) on the 32-bit Emacs that I just built. Of 
course a practical application would likely have a bunch of smaller lists, but 
the same basic idea applies. On such a platform, a user who wants to disable GC 
while fiddling with a bunch of large lists will need to set gc-cons-threshold to 
a bignum.

> supporting the full 32 bits (and 64
> bits in 64-bit builds) will also allow contradictory situation whereby
> gc-cons-threshold is higher than what we say should be used to disable
> GC.

Sorry, I'm not following. If setting gc-cons-threshold to a large value 
effectively disables GC, then setting gc-cons-threshold to a larger value should 
do the same thing. This is independent of any particular large value that we 
suggest in the manual.
>> The variable's value can be any intmax_t value. This is useful for
>> quantities like GC object byte counts that might not fit into fixnums.
> 
> Why do we need to talk about how many objects are there?  GC threshold
> is not about counting objects, it's about deciding when to GC.

The GC threshold is part of a related set of integers that count objects and 
bytes, for use in the returned value of garbage-collect among other things. It's 
convenient for it to be as least as wide as those other integers, so that 
calculations involving it do not overflow.

>> Don't see why not.
> 
> Are you working on that, or should someone else do it?

I can add it to my list of things to do. To my mind, getting the timestamp API 
nailed down is more urgent, though, because fiddling with GC heuristics doesn't 
affect the API.

> Right, but that's not what I proposed.  I proposed to trigger an
> immediate GC only the first time we detect memory-full.  Thereafter,
> the threshold will be set to 1KB, which should prevent thrashing.

Isn't it more complicated than that? Emacs can be low on memory, but can then 
get more and not be low on memory, and then be low on memory again later.

Message #64 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Thu, 15 Aug 2019 22:34:27 +0300

> Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Thu, 15 Aug 2019 11:51:06 -0700
> 
> > most-positive-fixnum on 32-bit systems is large enough for
> > every practical purpose.
> 
> It's not that hard for the number of consed bytes to exceed most-positive-fixnum 
> on a 32-bit Emacs. Here's a simple test case to illustrate the phenomenon:
> 
> (let* ((cons-size (car (cdr (car (garbage-collect)))))
>         (long-length (1+ (/ most-positive-fixnum cons-size)))
>         (l (make-list long-length nil)))
>    (cons most-positive-fixnum (* cons-size (length l))))
> 
> This yielded (536870911 . 536870912) on the 32-bit Emacs that I just built. Of 
> course a practical application would likely have a bunch of smaller lists, but 
> the same basic idea applies. On such a platform, a user who wants to disable GC 
> while fiddling with a bunch of large lists will need to set gc-cons-threshold to 
> a bignum.

I don't see why we must complicate our code to support such a use
case.  Users who want to disable GC while consing so much object will
have to learn that they cannot, not on a 32-bit machine.

> > supporting the full 32 bits (and 64
> > bits in 64-bit builds) will also allow contradictory situation whereby
> > gc-cons-threshold is higher than what we say should be used to disable
> > GC.
> 
> Sorry, I'm not following. If setting gc-cons-threshold to a large value 
> effectively disables GC, then setting gc-cons-threshold to a larger value should 
> do the same thing.

most-positive-fixnum is infinity for this purpose, and there should
not be numbers greater than infinity.  It's confusing and hard to
explain.

> > Why do we need to talk about how many objects are there?  GC threshold
> > is not about counting objects, it's about deciding when to GC.
> 
> The GC threshold is part of a related set of integers that count objects and 
> bytes, for use in the returned value of garbage-collect among other things.

No, they are just a means to decide when it's a good time to GC.  Very
large values are used to effectively disable GC, very small values to
cause GC "soon".  That's all, there are no requirements to count
objects or to have any accurate bookeeping of how many objects or
bytes were consed.

> > Are you working on that, or should someone else do it?
> 
> I can add it to my list of things to do. To my mind, getting the timestamp API 
> nailed down is more urgent, though, because fiddling with GC heuristics doesn't 
> affect the API.

Well, meanwhile we've broken a very popular use case, so I think
fixing this is rather urgent.

> > Right, but that's not what I proposed.  I proposed to trigger an
> > immediate GC only the first time we detect memory-full.  Thereafter,
> > the threshold will be set to 1KB, which should prevent thrashing.
> 
> Isn't it more complicated than that? Emacs can be low on memory, but can then 
> get more and not be low on memory, and then be low on memory again later.

Maybe so, but we had such code in Emacs for decades.  I just want to
avoid losing those features.

Message #69 received at 37006-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>, Akater <nuclearspace <at> gmail.com>
Cc: Joseph Mingrone <jrm <at> ftfl.ca>,
 Mattias Engdegård <mattiase <at> acm.org>,
 37006-done <at> debbugs.gnu.org
Subject: Re: Emacs 27 master consumes more memory than 26 and freezes regularly
Date: Tue, 3 Sep 2019 13:11:03 -0700

[Message part 1 (text/plain, inline)]

Paul Eggert wrote:
> Eli Zaretskii wrote:
>> once
>> gc-cons-threshold is set to a large value, it effectively disables GC
>> for the rest of the session, even if after that gc-cons-threshold is
>> reset back
> 
> Yes, that's next on my list of things to look at, after untangling this XDG 
> configuration mess.

Although I haven't finished untangling the XDG configuration mess, I did get 
some time free to attack the gc-cons-threshold issue and installed the attached. 
As this should fix the problem reported in Bug#37006 I'm boldly closing that bug 
report.

[0001-Sync-consing_until_gc-with-gc-cons-threshold.patch (text/x-patch, attachment)]

Message #72 received at 37006-done <at> debbugs.gnu.org (full text, mbox):

From: Joseph Mingrone <jrm <at> ftfl.ca>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 Eli Zaretskii <eliz <at> gnu.org>, Akater <nuclearspace <at> gmail.com>,
 37006-done <at> debbugs.gnu.org
Subject: Re: Emacs 27 master consumes more memory than 26 and freezes regularly
Date: Thu, 05 Sep 2019 13:12:11 -0300

[Message part 1 (text/plain, inline)]

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> Paul Eggert wrote:
>> Eli Zaretskii wrote:
>>> once
>>> gc-cons-threshold is set to a large value, it effectively disables GC
>>> for the rest of the session, even if after that gc-cons-threshold is
>>> reset back
>> Yes, that's next on my list of things to look at, after untangling this XDG 
>> configuration mess.

> Although I haven't finished untangling the XDG configuration mess, I did get some time free to attack the gc-cons-threshold issue and installed the attached. 
> As this should fix the problem reported in Bug#37006 I'm boldly closing that bug report.

This issue still exists for me.  After rebuilding on 5f089ac, I re-added
the (setq gc-cons-threshold most-positive-fixnum) / (setq
gc-cons-threshold 800000) surrounding init.el (assuming the issue was
fixed) then when Emacs' memory usage was over 2 GB, I turned on
`garbage-collection-messages', opened some large files, and could see
that garbage collection was not happening.

I just removed the surrounding (setq gc-cons-threshold
most-positive-fixnum) / (setq gc-cons-threshold 800000) and I see
garbage collection is happening for now.  I will report back if this
does not remain so.  In the last week or two running on a commit that
was some time after a recent fix (sorry, don't have the exact commit)
garbage collection was happening and the memory usage was not exploding.
The `gc-cons-threshold' hack in init.el was commented out.

[signature.asc (application/pgp-signature, inline)]

Message #77 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Joseph Mingrone <jrm <at> ftfl.ca>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 Eli Zaretskii <eliz <at> gnu.org>, 37006 <at> debbugs.gnu.org,
 Akater <nuclearspace <at> gmail.com>
Subject: Re: Emacs 27 master consumes more memory than 26 and freezes regularly
Date: Thu, 5 Sep 2019 10:01:26 -0700

On 9/5/19 9:12 AM, Joseph Mingrone wrote:
>> As this should fix the problem reported in Bug#37006 I'm boldly closing that bug report.
> This issue still exists for me.  After rebuilding on 5f089ac, I re-added
> the (setq gc-cons-threshold most-positive-fixnum) / (setq
> gc-cons-threshold 800000) surrounding init.el (assuming the issue was
> fixed) then when Emacs' memory usage was over 2 GB, I turned on
> `garbage-collection-messages', opened some large files, and could see
> that garbage collection was not happening.

OK, reopening the bug report. I take it that all I need to do is to 
modify init.el as mentioned above, and then edit some large files. I'll 
take a look at that.

Message #80 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Joseph Mingrone <jrm <at> ftfl.ca>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 Eli Zaretskii <eliz <at> gnu.org>, 37006 <at> debbugs.gnu.org,
 Akater <nuclearspace <at> gmail.com>
Subject: Re: Emacs 27 master consumes more memory than 26 and freezes regularly
Date: Thu, 05 Sep 2019 14:06:03 -0300

[Message part 1 (text/plain, inline)]

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> On 9/5/19 9:12 AM, Joseph Mingrone wrote:
>>> As this should fix the problem reported in Bug#37006 I'm boldly closing that bug report.
>> This issue still exists for me.  After rebuilding on 5f089ac, I re-added
>> the (setq gc-cons-threshold most-positive-fixnum) / (setq
>> gc-cons-threshold 800000) surrounding init.el (assuming the issue was
>> fixed) then when Emacs' memory usage was over 2 GB, I turned on
>> `garbage-collection-messages', opened some large files, and could see
>> that garbage collection was not happening.

> OK, reopening the bug report. I take it that all I need to do is to modify init.el as mentioned above, and then edit some large files. I'll 
> take a look at that.

That's all I had to do (on a FreeBSD 12.0 system).  I can give you an
account a FreeBSD system if that is helpful and saves you some time.

[signature.asc (application/pgp-signature, inline)]

Message #83 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Joseph Mingrone <jrm <at> ftfl.ca>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 Eli Zaretskii <eliz <at> gnu.org>, 37006 <at> debbugs.gnu.org,
 Akater <nuclearspace <at> gmail.com>
Subject: Re: Emacs 27 master consumes more memory than 26 and freezes regularly
Date: Thu, 5 Sep 2019 13:30:18 -0700

[Message part 1 (text/plain, inline)]

I reproduced the problem on Fedora 30 x86-64 and installed the attached 
fix. Please give it a try.

[0001-Fix-bugs-when-recalculating-consing_until_gc.patch (text/x-patch, attachment)]

Message #86 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Joseph Mingrone <jrm <at> ftfl.ca>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 Eli Zaretskii <eliz <at> gnu.org>, 37006 <at> debbugs.gnu.org,
 Akater <nuclearspace <at> gmail.com>
Subject: Re: Emacs 27 master consumes more memory than 26 and freezes regularly
Date: Thu, 05 Sep 2019 18:28:53 -0300

[Message part 1 (text/plain, inline)]

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> I reproduced the problem on Fedora 30 x86-64 and installed the attached fix. Please give it a try.

Hello Paul,

I see lots of garbage collection messages after a few minutes, so it
seems to be working.  If garbage collection stops after Emacs has been
running longer, I'll report back.

Thank you,

Joseph

[signature.asc (application/pgp-signature, inline)]

Message #89 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Joseph Mingrone <jrm <at> ftfl.ca>
Cc: Mattias Engdegård <mattiase <at> acm.org>,
 Eli Zaretskii <eliz <at> gnu.org>, 37006 <at> debbugs.gnu.org,
 Akater <nuclearspace <at> gmail.com>
Subject: Re: Emacs 27 master consumes more memory than 26 and freezes regularly
Date: Thu, 5 Sep 2019 14:34:56 -0700

On 9/5/19 2:28 PM, Joseph Mingrone wrote:

> I see lots of garbage collection messages after a few minutes, so it
> seems to be working.  If garbage collection stops after Emacs has been
> running longer, I'll report back.

Thanks for checking. I'll close the bug report (again :-). If the bug 
reappears I can reopen it again.

Message #94 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org,
 nuclearspace <at> gmail.com
Subject: Re: Emacs 27 master consumes more memory than 26 and freezes regularly
Date: Fri, 06 Sep 2019 10:03:45 +0300

> Cc: Eli Zaretskii <eliz <at> gnu.org>, Akater <nuclearspace <at> gmail.com>,
>  37006 <at> debbugs.gnu.org, Mattias Engdegård <mattiase <at> acm.org>
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Thu, 5 Sep 2019 14:34:56 -0700
> 
> On 9/5/19 2:28 PM, Joseph Mingrone wrote:
> 
> > I see lots of garbage collection messages after a few minutes, so it
> > seems to be working.  If garbage collection stops after Emacs has been
> > running longer, I'll report back.
> 
> Thanks for checking. I'll close the bug report (again :-). If the bug 
> reappears I can reopen it again.

Thanks for fixing the bug.  I wonder if it would make sense to have a
simple test for this particular use case, as this seems to be quite a
popular technique these days in users' init files.  The test would do
the sequence of GC settings like the one which triggered this bug, and
then verify that GC happens after it when expected.

WDYT?

Message #97 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Sat, 14 Sep 2019 00:51:18 -0700

[Message part 1 (text/plain, inline)]

On 8/15/19 7:17 AM, Eli Zaretskii wrote:
> OK, but I still hope we will switch to EMACS_INT instead.

I installed the attached patch into master, and it changes the relevant 
threshold variables to EMACS_INT, as part of a somewhat-larger cleanup. This 
should make a practical difference only on 32-bit Emacs without --with-wide-int.

[0001-Improve-gc-cons-percentage-calculation.patch (text/x-patch, attachment)]

Message #100 received at 37006 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
Subject: Re: bug#37006: 27.0.50; garbage collection not happening after
 26de2d42
Date: Sat, 14 Sep 2019 11:30:39 +0300

> Cc: jrm <at> ftfl.ca, mattiase <at> acm.org, 37006 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Sat, 14 Sep 2019 00:51:18 -0700
> 
> On 8/15/19 7:17 AM, Eli Zaretskii wrote:
> > OK, but I still hope we will switch to EMACS_INT instead.
> 
> I installed the attached patch into master, and it changes the relevant 
> threshold variables to EMACS_INT, as part of a somewhat-larger cleanup.

Thanks.

> This should make a practical difference only on 32-bit Emacs without
> --with-wide-int.

Right.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.