GNU bug report logs -
#76538
31.0.50; 31.0.50; 31.0.50; feature/igc: using magit-section-cycle-global (S-TAB) and magit-section-toggle (TAB) in some random ways blocks GNU Emacs.
Previous Next
To reply to this bug, email your comments to 76538 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Tue, 25 Feb 2025 03:42:01 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
João Moreira <joaomoreira <at> gmx.se>:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org.
(Tue, 25 Feb 2025 03:42:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
HAPPENED
Using magit-section-cycle-global (S-TAB) and magit-section-toggle (TAB) in some random ways blocks GNU Emacs while using feature/igc.
EXPECTED
Magit does not block GNU Emacs.
REPRODUCE
1. On a non-customized GNU Emacs feature/igc 48909543bdc7cfb605034398f2757e8d01004aca, eval
```emacs-lisp
;;;; PACKAGES
(require 'package)
(require 'package-vc)
(require 'use-package)
(defmacro my-package-install (package &optional feature &rest body)
"Set up PACKAGE from an Elisp archive, with rest BODY; require
FEATURE instead of PACKAGE if given.
PACKAGE is a quoted symbol, FEATURE is a quoted symbol, while BODY
consists of balanced expressions.
Try to install the package if it is missing.
(declare (indent 1))
`(progn
;; replace when not with unless?
(unless (package-installed-p ,package)
(unless package-archive-contents
(package-refresh-contents))
(package-install ,package))))
;; Set archives.
(dolist (package-archive '(("melpa" . "https://melpa.org/packages/"))
package-archives)
(unless (assoc-default (car package-archive) package-archives)
(add-to-list 'package-archives package-archive t)))
;; Initialize.
(package-initialize)
;;;; INSTALL
(my-package-install 'magit)
;;;; CONFIG
(use-package magit
:custom
(magit-git-executable (executable-find "git"))
(magit-perl-executable (executable-find "perl")))
```
2. `git clone https://github.com/joaoymoreira/joao.edu.uni.pi22019208901.real-review.git`
3. `cd real-review && rm -rf .git`
4. `git init`
3. Open magit, press S to stage files.
4. On Staged changes, press magit-section-cycle-global (S-TAB) and magit-section-toggle (TAB) randomly until it blocks GNU Emacs in showing white screen (all frames).
SYSTEM INFORMATION
In GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
version 1.18.2, Xaw3d scroll bars)
Repository revision: 48909543bdc7cfb605034398f2757e8d01004aca
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101014
System Description: NixOS 25.05 (Warbler)
Configured using:
'configure
--prefix=/nix/store/ji21vqsmpbnqsrq0l2rfvxaxpvpr0b7q-emacs-igc-20250217.0
--disable-build-details --with-modules --with-x-toolkit=lucid
--with-cairo --with-xft --with-compress-install
--with-toolkit-scroll-bars --with-native-compilation
--without-imagemagick --with-mailutils --without-small-ja-dic
--with-tree-sitter --with-xinput2 --without-xwidgets --with-dbus
--with-selinux --with-mps=yes'
Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES MPS NATIVE_COMP
NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XAW3D XDBE XIM XINPUT2 XPM
LUCID ZLIB
Important settings:
value of $EMACSLOADPATH:
value of $EMACSNATIVELOADPATH:
value of $LC_CTYPE: pt_BR.UTF-8
value of $LC_TIME: en_DK.UTF-8
value of $LANG: en_US.UTF-8
locale-coding-system: utf-8-unix
Major mode: Fundamental
Minor modes in effect:
global-git-commit-mode: t
magit-auto-revert-mode: t
override-global-mode: t
tooltip-mode: t
global-eldoc-mode: t
show-paren-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
blink-cursor-mode: t
minibuffer-regexp-mode: t
buffer-read-only: t
line-number-mode: t
indent-tabs-mode: t
transient-mark-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
Load-path shadows:
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-30 hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-30
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-29 hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-29
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-28 hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-28
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-27 hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-27
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-26 hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-26
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-25 hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-25
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-autoloads hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-autoloads
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-pkg hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-pkg
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-macs hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat-macs
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/dash-20240510.1327/dash-autoloads hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/dash-20240510.1327/dash-autoloads
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/dash-20240510.1327/dash hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/dash-20240510.1327/dash
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/dash-20240510.1327/dash-pkg hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/dash-20240510.1327/dash-pkg
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/seq-2.24/seq hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/seq-2.24/seq
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/seq-2.24/seq-autoloads hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/seq-2.24/seq-autoloads
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/seq-2.24/seq-25 hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/seq-2.24/seq-25
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/seq-2.24/seq-pkg hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/seq-2.24/seq-pkg
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/seq-2.24/seq-24 hides /nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/seq-2.24/seq-24
/nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/site-start hides /nix/store/ji21vqsmpbnqsrq0l2rfvxaxpvpr0b7q-emacs-igc-20250217.0/share/emacs/site-lisp/site-start
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/compat-30.0.2.0/compat hides /nix/store/ji21vqsmpbnqsrq0l2rfvxaxpvpr0b7q-emacs-igc-20250217.0/share/emacs/31.0.50/lisp/emacs-lisp/compat
/nix/store/nbi6gy4n0w8h13fna2fqp7a29cdn0km9-emacs-packages-deps/share/emacs/site-lisp/elpa/let-alist-1.0.6/let-alist hides /nix/store/ji21vqsmpbnqsrq0l2rfvxaxpvpr0b7q-emacs-igc-20250217.0/share/emacs/31.0.50/lisp/emacs-lisp/let-alist
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/seq-2.24/seq hides /nix/store/ji21vqsmpbnqsrq0l2rfvxaxpvpr0b7q-emacs-igc-20250217.0/share/emacs/31.0.50/lisp/emacs-lisp/seq
/etc/profiles/per-user/joao/share/emacs/site-lisp/elpa/cl-generic-0.3/cl-generic hides /nix/store/ji21vqsmpbnqsrq0l2rfvxaxpvpr0b7q-emacs-igc-20250217.0/share/emacs/31.0.50/lisp/emacs-lisp/cl-generic
Features:
(shadow sort mail-extr emacsbug cus-edit cus-start cus-load wid-edit
ediff ediff-merg ediff-mult ediff-wind ediff-diff ediff-help ediff-init
ediff-util let-alist magit-bookmark bookmark magit-submodule magit-blame
magit-stash magit-reflog magit-bisect magit-push magit-pull magit-fetch
magit-clone magit-remote magit-commit magit-sequence magit-notes
magit-worktree magit-tag magit-merge magit-branch magit-reset
magit-files magit-refs magit-status magit magit-repos magit-apply
magit-wip magit-log magit-diff smerge-mode diff git-commit magit-core
magit-autorevert autorevert filenotify magit-margin magit-transient
log-edit pcvs-util add-log magit-process with-editor magit-mode elp
transient pp edmacro kmacro magit-git magit-base which-func imenu vc-git
diff-mode track-changes files-x magit-section crm magit-autoloads
benchmark format-spec cursor-sensor llama comp comp-cstr cl-extra
help-mode magit-section-autoloads llama-autoloads pcase warnings shell
pcomplete server compat compat-30 with-editor-autoloads loaddefs-gen
radix-tree tar-mode arc-mode archive-mode mm-archive message sendmail
yank-media dired dired-loaddefs rfc822 mml mml-sec epa derived gnus-util
time-date mailabbrev gmm-utils mailheader mm-decode mm-bodies mm-encode
mail-utils gnutls network-stream url-cache url-http url-auth mail-parse
rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr url-gw nsm puny
compile text-property-search comint ansi-osc ansi-color ring comp-run
comp-common rx epg rfc6068 epg-config finder-inf use-package
use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core package-vc vc
vc-dispatcher lisp-mnt emacsql-sqlite-autoloads emojify-logos-autoloads
emojify-autoloads ht-autoloads info dash-autoloads jinx-autoloads
pdf-tools-autoloads tablist-autoloads vterm-autoloads package browse-url
xdg url url-proxy url-privacy url-expand url-methods url-history
url-cookie generate-lisp-file url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs icons
password-cache json subr-x map byte-opt gv bytecomp byte-compile
url-vars cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo x-toolkit
xinput2 x multi-tty move-toolbar make-network-process tty-child-frames
native-compile mps emacs)
Memory information:
((conses 24 0 0) (symbols 56 0 0) (strings 40 0 0) (string-bytes 1 0)
(vectors 24 0) (vector-slots 8 0 0) (floats 24 0 0)
(intervals 64 0 0) (buffers 1000 0))
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Tue, 25 Feb 2025 16:18:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> Date: Mon, 24 Feb 2025 21:52:45 -0300
> From: João Moreira via "Bug reports for GNU Emacs,
> the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
>
> Using magit-section-cycle-global (S-TAB) and magit-section-toggle (TAB) in some random ways blocks GNU Emacs while using feature/igc.
Isn't this because of the auto-revert mode that Magit turns on by
default? See bug#76505 for more details, and a possible workaround.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Tue, 25 Feb 2025 16:32:02 GMT)
Full text and
rfc822 format available.
Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):
João Moreira via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org> writes:
> HAPPENED
>
> Using magit-section-cycle-global (S-TAB) and magit-section-toggle (TAB) in some random ways blocks GNU Emacs while using feature/igc.
Thanks for the report! I tried reproducing it, and indeed Emacs hangs
for several minutes in back_to_previous_visible_line_start. However,
after that, it seems to become usable again. I tried reproducing this
again and failed, though.
This is a bug, and it seems to be much worse with --with-mps=yes, but
the behavior with --with-mps=no is hardly ideal either.
The main difference appears to be the charpos<->bytepos cache in the
weak marker vector, which grows to 0x8000 entries. That seems
excessive, but IIRC, what really matters is the order in which they're
tried.
> EXPECTED
>
> Magit does not block GNU Emacs.
>
> REPRODUCE
>
> 1. On a non-customized GNU Emacs feature/igc 48909543bdc7cfb605034398f2757e8d01004aca, eval
> ```emacs-lisp
> ;;;; PACKAGES
>
> (require 'package)
> (require 'package-vc)
> (require 'use-package)
>
> (defmacro my-package-install (package &optional feature &rest body)
> "Set up PACKAGE from an Elisp archive, with rest BODY; require
> FEATURE instead of PACKAGE if given.
>
> PACKAGE is a quoted symbol, FEATURE is a quoted symbol, while BODY
> consists of balanced expressions.
>
> Try to install the package if it is missing.
I had to add a closing quote here.
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Tue, 25 Feb 2025 16:32:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Tue, 25 Feb 2025 17:01:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text
editors" <bug-gnu-emacs <at> gnu.org> writes:
> The main difference appears to be the charpos<->bytepos cache in the
> weak marker vector, which grows to 0x8000 entries. That seems
> excessive, but IIRC, what really matters is the order in which they're
> tried.
FYI, I regularly observe bytepos_to_charpos in my perf traces. So, the
impact is rather significant. Especially when buffer has many markers.
I think we discussed this problem as improved the situation somewhat in
https://yhetil.org/emacs-devel/87v81u85hv.fsf <at> localhost/
Maybe there is something more that can be done?
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Tue, 25 Feb 2025 17:28:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> Cc: for <at> debbugs.gnu.org, Bug <at> debbugs.gnu.org,
> =?UTF-8?Q?Jo=C3=A3o <at> debbugs.gnu.org, eller.helmut <at> gmail.com,
> GNU <at> debbugs.gnu.org, via <at> debbugs.gnu.org, reports <at> debbugs.gnu.org,
> joaomoreira <at> gmx.se, 76538 <at> debbugs.gnu.org, Emacs <at> debbugs.gnu.org
> From: Ihor Radchenko <yantar92 <at> posteo.net>
> Date: Tue, 25 Feb 2025 16:59:29 +0000
>
> Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text
> editors" <bug-gnu-emacs <at> gnu.org> writes:
>
> > The main difference appears to be the charpos<->bytepos cache in the
> > weak marker vector, which grows to 0x8000 entries. That seems
> > excessive, but IIRC, what really matters is the order in which they're
> > tried.
>
> FYI, I regularly observe bytepos_to_charpos in my perf traces. So, the
> impact is rather significant. Especially when buffer has many markers.
> I think we discussed this problem as improved the situation somewhat in
> https://yhetil.org/emacs-devel/87v81u85hv.fsf <at> localhost/
>
> Maybe there is something more that can be done?
Maybe, but why is this so much worse with MPS?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Tue, 25 Feb 2025 17:56:01 GMT)
Full text and
rfc822 format available.
Message #23 received at 76538 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Tue, Feb 25 2025, Pip Cet wrote:
> Thanks for the report! I tried reproducing it, and indeed Emacs hangs
> for several minutes in back_to_previous_visible_line_start. However,
> after that, it seems to become usable again. I tried reproducing this
> again and failed, though.
I think I can reproduce this with the attached script and Magit
4.2.0. from .emacs.d.
I'm not sure if the non-MPS version is much better in this case. Magit
seems to create an impressive number of markers.
Helmut
[magit-test.sh (text/x-sh, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Tue, 25 Feb 2025 17:57:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Tue, 25 Feb 2025 20:14:01 GMT)
Full text and
rfc822 format available.
Message #29 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Helmut Eller <eller.helmut <at> gmail.com> writes:
> I think I can reproduce this with the attached script and Magit
> 4.2.0. from .emacs.d.
Confirmed with Magit 4.3.0 from ELPA
> I'm not sure if the non-MPS version is much better in this case. Magit
> seems to create an impressive number of markers.
Better for me.
The reproducer involves 20+Mb magit buffer with a lot of text hidden.
I toyed around with gdb a bit, and it appears to me that the problem is
not with the number of markers. Rather an opposite. Marker cache does
not help here initially and we need to scan all the way through 20Mb of
bytes.
In my testing, I looked into the total number of markers in the buffer,
and it appears that the number is always fairly small.
May someone more familiar with the code check if markers created in
buf_charpos_to_bytepos might be created and immediately GCed? That's my
impression (maybe totally wrong).
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Wed, 26 Feb 2025 13:25:02 GMT)
Full text and
rfc822 format available.
Message #32 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> Cc: Pip Cet <pipcet <at> protonmail.com>, for <at> debbugs.gnu.org, Bug <at> debbugs.gnu.org,
> =?UTF-8?Q?Jo=C3=A3o <at> debbugs.gnu.org, GNU <at> debbugs.gnu.org, via <at> debbugs.gnu.org,
> reports <at> debbugs.gnu.org, joaomoreira <at> gmx.se, 76538 <at> debbugs.gnu.org,
> Emacs <at> debbugs.gnu.org
> From: Ihor Radchenko <yantar92 <at> posteo.net>
> Date: Tue, 25 Feb 2025 20:12:21 +0000
>
> The reproducer involves 20+Mb magit buffer with a lot of text hidden.
> I toyed around with gdb a bit, and it appears to me that the problem is
> not with the number of markers. Rather an opposite. Marker cache does
> not help here initially and we need to scan all the way through 20Mb of
> bytes.
??? AFAIR, we perform binary search there, no?
Can you show the details of what you saw in GDB?
> In my testing, I looked into the total number of markers in the buffer,
> and it appears that the number is always fairly small.
How small?
> May someone more familiar with the code check if markers created in
> buf_charpos_to_bytepos might be created and immediately GCed? That's my
> impression (maybe totally wrong).
Why is the immediate GC a problem? And why do you think there is an
immediate GC? And finally, are we still talking about the igc branch
or about master (or both)?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Wed, 26 Feb 2025 17:41:03 GMT)
Full text and
rfc822 format available.
Message #35 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> The reproducer involves 20+Mb magit buffer with a lot of text hidden.
>> I toyed around with gdb a bit, and it appears to me that the problem is
>> not with the number of markers. Rather an opposite. Marker cache does
>> not help here initially and we need to scan all the way through 20Mb of
>> bytes.
>
> ??? AFAIR, we perform binary search there, no?
No. AFAIU, the search goes like the following:
1. Go through all the buffer markers and find the nearest marker to
charpos/bytepos
[ This is O(num_of_markers) on master and O(length_of_marker_array)
on feature/igc ]
2. Starting from the nearest marker, scan bytes symbol by symbol until
we find the requested charpos/bytepos
[ O(distance to nearest marker) ]
3. While scanning, every 5000 bytes, create a new marker to cache the
charpos and bytepos.
> Can you show the details of what you saw in GDB?
Here is what I did:
1. I modified the marker scan in buf_charpos_to_bytepos to record the
number of markers in buffer (just stored it in a C variable)
2. I followed the reproducer (C-p from point-max) and paused Emacs from
gdb while Emacs was hanging, looking at the backtrace and checking
the number of markers
3. I observed ~8-11k markers in the buffer. Notably, the number of
markers often decreased while runnign buf_charpos_to_bytepos,
indicating that the marker list is cleaned up along the way.
>> In my testing, I looked into the total number of markers in the buffer,
>> and it appears that the number is always fairly small.
>
> How small?
8-11k. Probably, not that small after all in terms of absolute numbers.
The reason why it is small in my mind is that limiting the loop over
markers to 50 iterations max did not speed things up.
>> May someone more familiar with the code check if markers created in
>> buf_charpos_to_bytepos might be created and immediately GCed? That's my
>> impression (maybe totally wrong).
>
> Why is the immediate GC a problem? And why do you think there is an
> immediate GC? And finally, are we still talking about the igc branch
> or about master (or both)?
I compared with Emacs 30. Emacs 30 takes a few seconds for C-p to work
and then does not hang (presumably, marker cache is populated). On
feature/igc, I did not manage to wait enough for C-p to compelte
(minutes).
Why do I think that there is an immediate GC? It is a guess (possibly
incorrect). My guess is originating from the fact that
bug_charpos_to_bytepos is calling build_marker directly to cache buffer
positions. However, the returned Lisp object is not used and thus
probably not referenced by any of the roots (it does get stored in the
buffer's marker list - but that's a weak array). So, there is a chance
that it can be collected the next time IGC decides to run the
collection. Given that we produce a large number of markers in
buf_charpos_to_bytepos, that "next time" might be very soon.
But I may be misunderstanding something important about how IGC code
works.
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Wed, 26 Feb 2025 19:59:01 GMT)
Full text and
rfc822 format available.
Message #38 received at 76538 <at> debbugs.gnu.org (full text, mbox):
>> ??? AFAIR, we perform binary search there, no?
>
> No. AFAIU, the search goes like the following:
>
> 1. Go through all the buffer markers and find the nearest marker to
> charpos/bytepos
> [ This is O(num_of_markers) on master and O(length_of_marker_array)
> on feature/igc ]
> 2. Starting from the nearest marker, scan bytes symbol by symbol until
> we find the requested charpos/bytepos
> [ O(distance to nearest marker) ]
> 3. While scanning, every 5000 bytes, create a new marker to cache the
> charpos and bytepos.
FWIW, I have a branch `scratch/markers-as-gap-array` where I replace the
linked list of markers with an "array with a gap" where markers are
sorted, so we use a binary search. It might help in your case.
It was pretty close to be "ready to merge" at some point, except that
I had trouble convincing myself that it was a clear win: in my tests, it
made `save-excursion` slower (the current naïve data-structure is ideal
for most "trivial" uses of `save-excursion`: we create a marker,
push it onto the head of the list, do a bit of unrelated work in the
body and then come back and just pop the marker that's still at the head
of the list) and the cases where it makes the code faster are hard to
come by, other than artificial micro-benchmarks.
>> How small?
> 8-11k.
I'd expect around 20MB / 5kB = 4k markers, so that seems about right.
> Probably, not that small after all in terms of absolute numbers.
Definitely not small if you scan it linearly/naïvely.
> The reason why it is small in my mind is that limiting the loop over
> markers to 50 iterations max did not speed things up.
We already limit the max number of iterations by increasing
progressively the distance considered "close enough", so maybe the
problem is simply that we have to choose between spending too much time
in the linear scan of the list of markers or we have to spend too much
time in the linear scan of the chars.
If that's the case, my branch should help significantly.
> Why do I think that there is an immediate GC? It is a guess (possibly
> incorrect). My guess is originating from the fact that
> bug_charpos_to_bytepos is calling build_marker directly to cache buffer
> positions. However, the returned Lisp object is not used and thus
> probably not referenced by any of the roots (it does get stored in the
> buffer's marker list - but that's a weak array).
Indeed those cache-markers are referenced only weakly so they currently
disappear at every GC. I can't remember if I kept this behavior in my
branch, but I remember considering some alternative where we keep every
other weak marker.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Wed, 26 Feb 2025 20:11:02 GMT)
Full text and
rfc822 format available.
Message #41 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
> FWIW, I have a branch `scratch/markers-as-gap-array` where I replace the
> linked list of markers with an "array with a gap" where markers are
> sorted, so we use a binary search. It might help in your case.
Maybe not on feature/igc (this bug is about igc). It made changes to the
way markers are stored in buffer and your branch might need to be
adapted to work with the branch.
> It was pretty close to be "ready to merge" at some point, except that
> I had trouble convincing myself that it was a clear win: in my tests, it
> made `save-excursion` slower (the current naïve data-structure is ideal
> for most "trivial" uses of `save-excursion`: we create a marker,
> push it onto the head of the list, do a bit of unrelated work in the
> body and then come back and just pop the marker that's still at the head
> of the list) and the cases where it makes the code faster are hard to
> come by, other than artificial micro-benchmarks.
FYI, feature/igc is also using an array for markers. So, your approach
might not be as bad here.
> Indeed those cache-markers are referenced only weakly so they currently
> disappear at every GC. I can't remember if I kept this behavior in my
> branch, but I remember considering some alternative where we keep every
> other weak marker.
It may or may not be the case on feature/igc. I think it remains to be
the case, and, unlike master, GC on feature/igc may happen at arbitrary
time rather than at specific code points. So, it may clear the cache in
the middle of the search (AFAIU).
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Wed, 26 Feb 2025 21:28:02 GMT)
Full text and
rfc822 format available.
Message #44 received at 76538 <at> debbugs.gnu.org (full text, mbox):
[ See the end for a possible explanation and solution to the slowdown
we see on the igc branch. ]
>> FWIW, I have a branch `scratch/markers-as-gap-array` where I replace the
>> linked list of markers with an "array with a gap" where markers are
>> sorted, so we use a binary search. It might help in your case.
>
> Maybe not on feature/igc (this bug is about igc). It made changes to the
> way markers are stored in buffer and your branch might need to be
> adapted to work with the branch.
Definitely.
Not sure how to proceed, tho.
I kind of like the idea of keeping "separate" development on
separate branches. But indeed, since the igc branch already changed the
implementation of markers, maybe I should rebase my branch on top of igc?
>> It was pretty close to be "ready to merge" at some point, except that
>> I had trouble convincing myself that it was a clear win: in my tests, it
>> made `save-excursion` slower (the current naïve data-structure is ideal
>> for most "trivial" uses of `save-excursion`: we create a marker,
>> push it onto the head of the list, do a bit of unrelated work in the
>> body and then come back and just pop the marker that's still at the head
>> of the list) and the cases where it makes the code faster are hard to
>> come by, other than artificial micro-benchmarks.
>
> FYI, feature/igc is also using an array for markers. So, your approach
> might not be as bad here.
In the igc branch, both adding a marker to a buffer and removing it from
a buffer are very fast O(1) operations (even better than the code on
`master`, for the case of removal), whereas with my sorted-array-with-gap
I need O(log N) time for each of those. 🙁
I can probably improve the performance a bit by taking advantage of
locality, but even if I can reduce the average number of iterations
through the loop, there is still the fact that the current code doesn't
need to iterate at all.
>> Indeed those cache-markers are referenced only weakly so they currently
>> disappear at every GC. I can't remember if I kept this behavior in my
>> branch, but I remember considering some alternative where we keep every
>> other weak marker.
> It may or may not be the case on feature/igc. I think it remains to be
> the case, and, unlike master, GC on feature/igc may happen at arbitrary
> time rather than at specific code points. So, it may clear the cache in
> the middle of the search (AFAIU).
With my sorted-array-with-gap we could be much more sophisticated about
the handling of cache-markers:
- The data-structure can much more easily handle a large number of
markers (even during things like insertion/deletion the updates are
significantly cheaper), so there is less pressure the get rid of
cache markers.
- Because the array is sorted we can easily estimate the usefulness
of a given cache-marker (by looking at its distance to the next marker).
- We could keep a running cost/benefit analysis for each cache marker,
which gets incremented when we need to update the position
and decremented when we use the marker for byte<->char conversion.
- In the current system, once we have "too many markers" (and that
happens pretty quickly) the byte<->char conversion code may fail to
find the nearest marker, simply because it's too far from the top, so
we end up creating yet more cache-markers. If we're not careful to
throw away the cache-markers from the end of the linked-list, this
could quickly lead to absurdly large numbers of (cache-)markers.
BTW, now that I look at the code, maybe the problem you're bumping into
is that on the igc branch, (cache-)markers are added to the rear of the
array (as a first approximation), whereas on `master` they're added to
the front of the linked list.
In `buf_charpos_to_bytepos` we scan the array/list starting from
the front, so once the array gets large we never reach the end of the
array where we would find the recently-created nearby cache-marker, so
instead of using that nearby (cache-)marker we end up creating a new one
and paying (again) for a costly long scan.
Maybe if we change the `DO_MARKERS` iteration to scan from the end,
we'd recover a behavior much closer to what we see on `master`.
Gerd?
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Wed, 26 Feb 2025 21:36:02 GMT)
Full text and
rfc822 format available.
Message #47 received at 76538 <at> debbugs.gnu.org (full text, mbox):
"Stefan Monnier" <monnier <at> iro.umontreal.ca> writes:
> [ See the end for a possible explanation and solution to the slowdown
> we see on the igc branch. ]
>
>>> FWIW, I have a branch `scratch/markers-as-gap-array` where I replace the
>>> linked list of markers with an "array with a gap" where markers are
>>> sorted, so we use a binary search. It might help in your case.
>>
>> Maybe not on feature/igc (this bug is about igc). It made changes to the
>> way markers are stored in buffer and your branch might need to be
>> adapted to work with the branch.
>
> Definitely.
>
> Not sure how to proceed, tho.
> I kind of like the idea of keeping "separate" development on
> separate branches. But indeed, since the igc branch already changed the
> implementation of markers, maybe I should rebase my branch on top of igc?
That'd be great. It seems we definitely need a better implementation
here :-)
>>> Indeed those cache-markers are referenced only weakly so they currently
>>> disappear at every GC. I can't remember if I kept this behavior in my
>>> branch, but I remember considering some alternative where we keep every
>>> other weak marker.
>> It may or may not be the case on feature/igc. I think it remains to be
>> the case, and, unlike master, GC on feature/igc may happen at arbitrary
>> time rather than at specific code points. So, it may clear the cache in
>> the middle of the search (AFAIU).
>
> With my sorted-array-with-gap we could be much more sophisticated about
> the handling of cache-markers:
>
> - The data-structure can much more easily handle a large number of
> markers (even during things like insertion/deletion the updates are
> significantly cheaper), so there is less pressure the get rid of
> cache markers.
> - Because the array is sorted we can easily estimate the usefulness
> of a given cache-marker (by looking at its distance to the next marker).
> - We could keep a running cost/benefit analysis for each cache marker,
> which gets incremented when we need to update the position
> and decremented when we use the marker for byte<->char conversion.
> - In the current system, once we have "too many markers" (and that
> happens pretty quickly) the byte<->char conversion code may fail to
> find the nearest marker, simply because it's too far from the top, so
> we end up creating yet more cache-markers. If we're not careful to
> throw away the cache-markers from the end of the linked-list, this
> could quickly lead to absurdly large numbers of (cache-)markers.
>
> BTW, now that I look at the code, maybe the problem you're bumping into
> is that on the igc branch, (cache-)markers are added to the rear of the
> array (as a first approximation), whereas on `master` they're added to
> the front of the linked list.
>
> In `buf_charpos_to_bytepos` we scan the array/list starting from
> the front, so once the array gets large we never reach the end of the
> array where we would find the recently-created nearby cache-marker, so
> instead of using that nearby (cache-)marker we end up creating a new one
> and paying (again) for a costly long scan.
That seems to match what's happening here, with the order of scanning to
blame, but...
> Maybe if we change the `DO_MARKERS` iteration to scan from the end,
> we'd recover a behavior much closer to what we see on `master`.
...I tried doing that, and it didn't work. Maybe I messed up somehow.
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Wed, 26 Feb 2025 22:41:02 GMT)
Full text and
rfc822 format available.
Message #50 received at 76538 <at> debbugs.gnu.org (full text, mbox):
>> BTW, now that I look at the code, maybe the problem you're bumping into
>> is that on the igc branch, (cache-)markers are added to the rear of the
>> array (as a first approximation), whereas on `master` they're added to
>> the front of the linked list.
>>
>> In `buf_charpos_to_bytepos` we scan the array/list starting from
>> the front, so once the array gets large we never reach the end of the
>> array where we would find the recently-created nearby cache-marker, so
>> instead of using that nearby (cache-)marker we end up creating a new one
>> and paying (again) for a costly long scan.
>
> That seems to match what's happening here, with the order of scanning to
> blame, but...
>
>> Maybe if we change the `DO_MARKERS` iteration to scan from the end,
>> we'd recover a behavior much closer to what we see on `master`.
>
> ...I tried doing that, and it didn't work. Maybe I messed up somehow.
Hmm... yeah, maybe you messed up somehow, or maybe it's because of the
"as a first approximation":
Gerd's array of markers keeps a free-list so new markers are not really
always added to the end, but rather they're added "randomly" in the
reverse order where markers were removed from the buffer (until reaching
the end of this, at which point it's added to the end of the array).
If there are enough markers in the array, anywhere that's not very-near
the rear (or the front, depending on the order in which we scan) will
simply never be consulted during char<->byte conversion. 🙁
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Wed, 26 Feb 2025 22:54:02 GMT)
Full text and
rfc822 format available.
Message #53 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> Gerd's array of markers keeps a free-list so new markers are not really
> always added to the end, but rather they're added "randomly" in the
> reverse order where markers were removed from the buffer (until reaching
> the end of this, at which point it's added to the end of the array).
> If there are enough markers in the array, anywhere that's not very-near
> the rear (or the front, depending on the order in which we scan) will
> simply never be consulted during char<->byte conversion. 🙁
Maybe we can confirm this behavior:
- in the `buf_*pos_to_*pos` code, in the `DO_MARKERS` loop, we can keep
track of the range of indices of the markers we consult.
- then when we exit unsuccessfully and create a new cache-marker, we can
compare the index of that cache-marker with the indices we consulted.
If the index is not within the range of indices we consulted, then this
cache-marker is most likely going to be useless (because, if nothing
else is changed, next time we need to convert at this position we simply
won't find it).
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Wed, 26 Feb 2025 23:20:02 GMT)
Full text and
rfc822 format available.
Message #56 received at 76538 <at> debbugs.gnu.org (full text, mbox):
On Wed, Feb 26 2025, Stefan Monnier wrote:
[...]
> Maybe if we change the `DO_MARKERS` iteration to scan from the end,
> we'd recover a behavior much closer to what we see on `master`.
> Gerd?
Déjà vu:
https://mail.gnu.org/archive/html/emacs-devel/2024-08/msg00171.html
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Thu, 27 Feb 2025 04:34:02 GMT)
Full text and
rfc822 format available.
Message #59 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
> Maybe if we change the `DO_MARKERS` iteration to scan from the end,
> we'd recover a behavior much closer to what we see on `master`.
> Gerd?
I don't know, but it might well be that one order is as bad as the
other, depending on the order in which markers are added to and removed
from the marker array. And I have no intuition what a marker array ends
up looking like in a typical use case, alas.
Hm. Can we record the insertion order somehow? I guess that's what the
heuristic is using?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Thu, 27 Feb 2025 04:37:02 GMT)
Full text and
rfc822 format available.
Message #62 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Helmut Eller <eller.helmut <at> gmail.com> writes:
> On Wed, Feb 26 2025, Stefan Monnier wrote:
>
> [...]
>> Maybe if we change the `DO_MARKERS` iteration to scan from the end,
>> we'd recover a behavior much closer to what we see on `master`.
>> Gerd?
>
> Déjà vu:
>
> https://mail.gnu.org/archive/html/emacs-devel/2024-08/msg00171.html
:-)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Thu, 27 Feb 2025 18:21:01 GMT)
Full text and
rfc822 format available.
Message #65 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>
>> Maybe if we change the `DO_MARKERS` iteration to scan from the end,
>> we'd recover a behavior much closer to what we see on `master`.
>> Gerd?
>
> I don't know, but it might well be that one order is as bad as the
> other, depending on the order in which markers are added to and removed
> from the marker array. And I have no intuition what a marker array ends
> up looking like in a typical use case, alas.
>
> Hm. Can we record the insertion order somehow? I guess that's what the
> heuristic is using?
FYI, I tried various tweaks to that loop while debugging and concluded
that DO_MARKERS loop is not the main bottleneck in this particular bug
report.
I am wondering if anyone else is able to reproduce the problem locally.
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Thu, 27 Feb 2025 19:34:02 GMT)
Full text and
rfc822 format available.
Message #68 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Ihor Radchenko <yantar92 <at> posteo.net> writes:
> Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
>
>> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>>
>>> Maybe if we change the `DO_MARKERS` iteration to scan from the end,
>>> we'd recover a behavior much closer to what we see on `master`.
>>> Gerd?
>>
>> I don't know, but it might well be that one order is as bad as the
>> other, depending on the order in which markers are added to and removed
>> from the marker array. And I have no intuition what a marker array ends
>> up looking like in a typical use case, alas.
>>
>> Hm. Can we record the insertion order somehow? I guess that's what the
>> heuristic is using?
>
> FYI, I tried various tweaks to that loop while debugging and concluded
> that DO_MARKERS loop is not the main bottleneck in this particular bug
> report.
>
> I am wondering if anyone else is able to reproduce the problem locally.
I haven't tried the OP's recipe, but I know that Magit can get really
slow, things taking minutes. An example would be a large merge of
several hundred files, and with conflicts. The status buffer becomes
as good as unusable then. I don't think this started with igc.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Thu, 27 Feb 2025 19:50:01 GMT)
Full text and
rfc822 format available.
Message #71 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
>> I am wondering if anyone else is able to reproduce the problem locally.
>
> I haven't tried the OP's recipe, but I know that Magit can get really
> slow, things taking minutes. An example would be a large merge of
> several hundred files, and with conflicts. The status buffer becomes
> as good as unusable then. I don't think this started with igc.
In this particular case, igc is hanging forever while master just hangs
for a few seconds and then goes back to be responsive.
So, something _is_ going on in igc.
--
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Thu, 27 Feb 2025 20:55:02 GMT)
Full text and
rfc822 format available.
Message #74 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> FYI, I tried various tweaks to that loop while debugging and concluded
> that DO_MARKERS loop is not the main bottleneck in this particular bug
> report.
The DO_MARKERS is probably not the main bottleneck nowadays, because the
code explicitly reduces the "estimated benefit" of remaining in the loop
at each iteration and exits once that gets too small.
But that just pushes the problem to the subsequent bytes-scan.
The main problem is simply that we need to make sure some nearby
(cache-)marker is found near the "beginning" of the set of markers we
consider. Once this set becomes large the ordering between the markers
becomes super-important. The code on `master` behaves a bit like an
"move to front" policy which works fairly well for the usual locality
properties of buffer navigation, whereas the code on the `igc` branch
has a tendency of putting new entries near the end where they risk not
being found at all, making the cache ineffective.
> I am wondering if anyone else is able to reproduce the problem locally.
I haven't looked at the recipe yet, TBH.
But I strongly suspect that you're seeing something similar to what
happens in the `bytechar-100k-random` benchmark in Helmut Eller's email.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Fri, 28 Feb 2025 05:10:05 GMT)
Full text and
rfc822 format available.
Message #77 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>> FYI, I tried various tweaks to that loop while debugging and concluded
>> that DO_MARKERS loop is not the main bottleneck in this particular bug
>> report.
>
> The DO_MARKERS is probably not the main bottleneck nowadays, because the
> code explicitly reduces the "estimated benefit" of remaining in the loop
> at each iteration and exits once that gets too small.
>
> But that just pushes the problem to the subsequent bytes-scan.
>
> The main problem is simply that we need to make sure some nearby
> (cache-)marker is found near the "beginning" of the set of markers we
> consider. Once this set becomes large the ordering between the markers
> becomes super-important. The code on `master` behaves a bit like an
> "move to front" policy which works fairly well for the usual locality
> properties of buffer navigation, whereas the code on the `igc` branch
> has a tendency of putting new entries near the end where they risk not
> being found at all, making the cache ineffective.
Right, the order is important because the DO_MARKERS loop relies on the
assumption that cache-markers are found "early" in the loop. Or that's
how I understand it.
From just reading the code again, and taking into account what Pip
said about the number of markers being created, I think what happens
in igc is something like this, in buf_charpos_to_bytepos.
Let's say we have initially our array of markers [m1, m2, m3, ...]. We
are looping over the array with DO_MARKERS. With each round, we decrease
our expectations what a good enough marker looks like. M1 would have to
be relatively close to the position we are looking for, m2 not so close,
m3 may be even farther away, and so on.
Okay. Let's say we don't find our position, and add a cache-marker.
The array now looks like [m1, m2, m3, ..., c1, ...].
Next time in the function, we loop again. We compare with m1, then m2,
m3 as before. But there is nothing saying that we'll reach c1. We add
c2 and now have [m1, m2, m3, ..., c2, c1, ...].
And so on, and so on. And we can't really say where c1 and c2 are added
in the array. That depends on the free-list, which depends on how
markers are added/removed or weakly fixed (fix_marker_vector).
Do we know how often fix_marker_vector runs?
In any case, I think I would try to change DO_MARKERS to start at the
end of the marker vector. If fix_marker_vector runs often enough one
could also "straighten" the free-list there, maybe.
(Sorry, I can't bring me to dive deep into this, ATM.)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Fri, 28 Feb 2025 05:20:09 GMT)
Full text and
rfc822 format available.
Message #80 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
> In any case, I think I would try to change DO_MARKERS to start at the
> end of the marker vector. If fix_marker_vector runs often enough one
> could also "straighten" the free-list there, maybe.
Or one could build a doubly-linked list of markers in most-recently
created order. But that would mean 3 array slots per marker in the
marker vector. Don't know.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Fri, 28 Feb 2025 08:21:02 GMT)
Full text and
rfc822 format available.
Message #83 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> From: Gerd Möllmann <gerd.moellmann <at> gmail.com>
> Cc: Ihor Radchenko <yantar92 <at> posteo.net>, Eli Zaretskii <eliz <at> gnu.org>,
> eller.helmut <at> gmail.com, pipcet <at> protonmail.com, joaomoreira <at> gmx.se,
> 76538 <at> debbugs.gnu.org
> Date: Fri, 28 Feb 2025 06:19:34 +0100
>
> Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
>
> > In any case, I think I would try to change DO_MARKERS to start at the
> > end of the marker vector. If fix_marker_vector runs often enough one
> > could also "straighten" the free-list there, maybe.
>
> Or one could build a doubly-linked list of markers in most-recently
> created order. But that would mean 3 array slots per marker in the
> marker vector. Don't know.
How about not relying on buffer markers too much for this job? And in
particular, not adding any ephemeral markers when we don't find a
close-enough existing marker? We could instead build our own data
structure, and make it so we could use efficient search methods.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Fri, 28 Feb 2025 15:02:02 GMT)
Full text and
rfc822 format available.
Message #86 received at 76538 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
> Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
>
>> In any case, I think I would try to change DO_MARKERS to start at the
>> end of the marker vector. If fix_marker_vector runs often enough one
>> could also "straighten" the free-list there, maybe.
>
> Or one could build a doubly-linked list of markers in most-recently
> created order. But that would mean 3 array slots per marker in the
> marker vector. Don't know.
Which would be something like the attached. Can someone try this with
the original recipe? (It's a quick sketch, and blah blah.)
[0001-Iterating-over-markers-in-most-recently-added-order.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Fri, 28 Feb 2025 15:23:02 GMT)
Full text and
rfc822 format available.
Message #89 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> How about not relying on buffer markers too much for this job? And in
> particular, not adding any ephemeral markers when we don't find a
> close-enough existing marker? We could instead build our own data
> structure, and make it so we could use efficient search methods.
FWIW, my sorted-array-with-gap is kind of trying that, after making the
choice that the cache is eagerly updated during insertion/deletion.
But admittedly, we could keep the markers as they are and use the
sorted-array-with-gap only for the cache. This way `save-excursion`
would remain as efficient as before. At the cost of extra code,
of course.
Another attractive choice would be a cache that is flushed during
insertion/deletions (like the cache in `syntax-ppss`). In that case it
can be a simple growable array with a constant lookup time since we just
look at `cache[POS / N]` where N is some arbitrary factor we choose
based on a tradeoff of size of the cache vs time to scan the bytes from
the cache's info to the actual POS.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Fri, 28 Feb 2025 16:08:01 GMT)
Full text and
rfc822 format available.
Message #92 received at 76538 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Friday, February 28th, 2025 at 08:19, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > From: Gerd Möllmann gerd.moellmann <at> gmail.com
>
> > Cc: Ihor Radchenko yantar92 <at> posteo.net, Eli Zaretskii eliz <at> gnu.org,
> > eller.helmut <at> gmail.com, pipcet <at> protonmail.com, joaomoreira <at> gmx.se,
> > 76538 <at> debbugs.gnu.org
> > Date: Fri, 28 Feb 2025 06:19:34 +0100
> >
> > Gerd Möllmann gerd.moellmann <at> gmail.com writes:
> >
> > > In any case, I think I would try to change DO_MARKERS to start at the
> > > end of the marker vector. If fix_marker_vector runs often enough one
> > > could also "straighten" the free-list there, maybe.
> >
> > Or one could build a doubly-linked list of markers in most-recently
> > created order. But that would mean 3 array slots per marker in the
> > marker vector. Don't know.
>
> How about not relying on buffer markers too much for this job? And in
> particular, not adding any ephemeral markers when we don't find a
> close-enough existing marker? We could instead build our own data
> structure, and make it so we could use efficient search methods.
I can't really contribute much to the general discussion, but, if I understand correctly, in this specific case the problem is that magit causes redisplay to skip over a large invisible region, finding each newline in it, calculating its charpos, then advancing to the next one because the newline is invisible.
So I've tried applying the attached patch, and that seems to avoid the specific problem here: we simply skip to the beginning of the invisible region, which means only one charpos<->bytepos lookup for the entire region, and it doesn't matter too much that it's a slow one.
Of course there are still problems caused by the heuristic nature of the charpos<->bytepos caching mechanism, which simply doesn't have very good worst-time complexity, but I think the right solution there is to continue working on storing buffer text in a number of chunks (and saving the correct bytepos/charpos change for each chunk). I have some code for that and I'd rather continue working on it for a while before giving up and spending time on different implementations of the marker array. That would give us predictable and acceptable worst-case performance, I think, and we wouldn't need to go out hunting for use cases that come close to the worst case.
Pip
[invisible.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Sat, 01 Mar 2025 07:18:01 GMT)
Full text and
rfc822 format available.
Message #95 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
> Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
>
>> Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
>>
>>> In any case, I think I would try to change DO_MARKERS to start at the
>>> end of the marker vector. If fix_marker_vector runs often enough one
>>> could also "straighten" the free-list there, maybe.
>>
>> Or one could build a doubly-linked list of markers in most-recently
>> created order. But that would mean 3 array slots per marker in the
>> marker vector. Don't know.
>
> Which would be something like the attached. Can someone try this with
> the original recipe? (It's a quick sketch, and blah blah.)
And I think I'll keep that. It makes an overall nicer implementation of
DO_MARKERS, keeps add and remove O(1), and provides iteration in
most-recently added order, which should be the same as without igc.
I like it :-). Until something better comes along, of course.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Sat, 01 Mar 2025 13:35:01 GMT)
Full text and
rfc822 format available.
Message #98 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
> Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
>
>> Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
>>
>>> Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
>>>
>>>> In any case, I think I would try to change DO_MARKERS to start at the
>>>> end of the marker vector. If fix_marker_vector runs often enough one
>>>> could also "straighten" the free-list there, maybe.
>>>
>>> Or one could build a doubly-linked list of markers in most-recently
>>> created order. But that would mean 3 array slots per marker in the
>>> marker vector. Don't know.
>>
>> Which would be something like the attached. Can someone try this with
>> the original recipe? (It's a quick sketch, and blah blah.)
Performance seems slightly better, but not as good as disabling the
bytepos<->charpos cache entirely and skipping the DO_MARKERS loop.
OTOH, it's not worse! And keeping the markers in an effectively
doubly-linked list allows us to sort them whatever way we want.
> And I think I'll keep that. It makes an overall nicer implementation of
> DO_MARKERS, keeps add and remove O(1), and provides iteration in
> most-recently added order, which should be the same as without igc.
I think we might want to sort markers at some point: if we start out
with the median marker, then the 25-percentile one, then the
75-percentile one, etc., we should arrive at an acceptable one fairly
early without changing the scanning code at all!
(In more detail, I think we should start with any marker, then choose
the one with the greatest distance to all previously-chosen ones next,
recursively. I don't remember whether there's a particularly efficient
algorithm to do that in the general case, but in the linear case, it's
pretty simple, just build a btree and emit it in depth-first order. Fun
fact: do the same with a visual color metric and you get a usable color
scheme!).
> I like it :-). Until something better comes along, of course.
I still think I have "something better" in my head, but it's not so much
in code form yet, and work on that is quite slow right now, for various
reasons, so having a no-worse-than-master solution is important!
So I'd be happy to see this merged! If it improves performance to be
acceptable, that's most likely sufficient, as far as this problem goes,
for merging feature/igc (which I still believe in), even though the
xdisp.c change (invisible.patch) I proposed (and analogous ones) should
improve things a lot for magit even on master.
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Sat, 01 Mar 2025 14:13:02 GMT)
Full text and
rfc822 format available.
Message #101 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Pip Cet <pipcet <at> protonmail.com> writes:
>>> Which would be something like the attached. Can someone try this with
>>> the original recipe? (It's a quick sketch, and blah blah.)
>
> Performance seems slightly better, but not as good as disabling the
> bytepos<->charpos cache entirely and skipping the DO_MARKERS loop.
>
> OTOH, it's not worse! And keeping the markers in an effectively
> doubly-linked list allows us to sort them whatever way we want.
That's good to hear, thanks for checking!
>> And I think I'll keep that. It makes an overall nicer implementation of
>> DO_MARKERS, keeps add and remove O(1), and provides iteration in
>> most-recently added order, which should be the same as without igc.
>
> I think we might want to sort markers at some point: if we start out
> with the median marker, then the 25-percentile one, then the
> 75-percentile one, etc., we should arrive at an acceptable one fairly
> early without changing the scanning code at all!
>
> (In more detail, I think we should start with any marker, then choose
> the one with the greatest distance to all previously-chosen ones next,
> recursively. I don't remember whether there's a particularly efficient
> algorithm to do that in the general case, but in the linear case, it's
> pretty simple, just build a btree and emit it in depth-first order. Fun
> fact: do the same with a visual color metric and you get a usable color
> scheme!).
Or what Stefan or Eli said, I think, something independent of the buffer
markers. I'd like to see something like that.
>> I like it :-). Until something better comes along, of course.
>
> I still think I have "something better" in my head, but it's not so much
> in code form yet, and work on that is quite slow right now, for various
> reasons, so having a no-worse-than-master solution is important!
>
> So I'd be happy to see this merged! If it improves performance to be
> acceptable, that's most likely sufficient, as far as this problem goes,
> for merging feature/igc (which I still believe in), even though the
> xdisp.c change (invisible.patch) I proposed (and analogous ones) should
> improve things a lot for magit even on master.
Ok, I'll add it then.
(And I'd like to have the other stuff for Magit :-)!)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Sat, 01 Mar 2025 14:34:02 GMT)
Full text and
rfc822 format available.
Message #104 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
> Ok, I'll add it then.
Pushed.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Sat, 01 Mar 2025 17:00:02 GMT)
Full text and
rfc822 format available.
Message #107 received at 76538 <at> debbugs.gnu.org (full text, mbox):
On Fri, Feb 28 2025, Gerd Möllmann wrote:
> +static void
> +unchain (struct Lisp_Vector *v, int slot)
> +{
> + IGC_MA_MARKER (v, slot) = IGC_MA_FREE_LIST (v);
> + IGC_MA_FREE_LIST (v) = make_fixnum (slot);
> +
> + int prev = XFIXNUM (IGC_MA_PREV (v, slot));
> + if (prev >= 0)
> + IGC_MA_NEXT (v, prev) = IGC_MA_NEXT (v, slot);
> + else
> + IGC_MA_HEAD (v) = IGC_MA_NEXT (v, slot);
> +
> + int next = XFIXNUM (IGC_MA_NEXT (v, slot));
> + if (next >= 0)
> + IGC_MA_PREV (v, next) = IGC_MA_PREV (v, slot);
> +}
Doing this in the collector looks quite risky. It seems to assume that
the mutator is not in the middle of reading from or writing to any of
the involved slots.
Helmut
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 05:59:02 GMT)
Full text and
rfc822 format available.
Message #110 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Helmut Eller <eller.helmut <at> gmail.com> writes:
> On Fri, Feb 28 2025, Gerd Möllmann wrote:
>
>> +static void
>> +unchain (struct Lisp_Vector *v, int slot)
>> +{
>> + IGC_MA_MARKER (v, slot) = IGC_MA_FREE_LIST (v);
>> + IGC_MA_FREE_LIST (v) = make_fixnum (slot);
>> +
>> + int prev = XFIXNUM (IGC_MA_PREV (v, slot));
>> + if (prev >= 0)
>> + IGC_MA_NEXT (v, prev) = IGC_MA_NEXT (v, slot);
>> + else
>> + IGC_MA_HEAD (v) = IGC_MA_NEXT (v, slot);
>> +
>> + int next = XFIXNUM (IGC_MA_NEXT (v, slot));
>> + if (next >= 0)
>> + IGC_MA_PREV (v, next) = IGC_MA_PREV (v, slot);
>> +}
>
> Doing this in the collector looks quite risky. It seems to assume that
> the mutator is not in the middle of reading from or writing to any of
> the involved slots.
>
Do you have a concrete scenario in mind?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 08:41:01 GMT)
Full text and
rfc822 format available.
Message #113 received at 76538 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Mon, Mar 03 2025, Gerd Möllmann wrote:
> Do you have a concrete scenario in mind?
To increase the probability of the problem, let's call igc_collect from
igc_add_marker, like so:
diff --git a/src/igc.c b/src/igc.c
index ec714cad435..9bf0a0eca91 100644
--- a/src/igc.c
+++ b/src/igc.c
@@ -4595,7 +4595,9 @@ igc_add_marker (struct buffer *b, struct Lisp_Marker *m)
IGC_MA_FREE_LIST (xv) = IGC_MA_MARKER (xv, slot);
IGC_MA_MARKER (xv, slot) = make_lisp_ptr (m, Lisp_Vectorlike);
- IGC_MA_NEXT (xv, slot) = IGC_MA_HEAD (xv);
+ Lisp_Object head = IGC_MA_HEAD (xv);
+ igc_collect ();
+ IGC_MA_NEXT (xv, slot) = head;
IGC_MA_PREV (xv, slot) = make_fixnum (-1);
IGC_MA_HEAD (xv) = make_fixnum (slot);
ptrdiff_t next = XFIXNUM (IGC_MA_NEXT (xv, slot));
Then, run the code below with
./src/emacs -Q -batch -l /tmp/m.el -f test
For me, Emacs seems to enter an endless loop after creating 36 markers.
Helmut
[m.el (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 09:17:01 GMT)
Full text and
rfc822 format available.
Message #116 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Helmut Eller <eller.helmut <at> gmail.com> writes:
> On Mon, Mar 03 2025, Gerd Möllmann wrote:
>
>> Do you have a concrete scenario in mind?
>
> To increase the probability of the problem, let's call igc_collect from
> igc_add_marker, like so:
>
> diff --git a/src/igc.c b/src/igc.c
> index ec714cad435..9bf0a0eca91 100644
> --- a/src/igc.c
> +++ b/src/igc.c
> @@ -4595,7 +4595,9 @@ igc_add_marker (struct buffer *b, struct Lisp_Marker *m)
>
> IGC_MA_FREE_LIST (xv) = IGC_MA_MARKER (xv, slot);
> IGC_MA_MARKER (xv, slot) = make_lisp_ptr (m, Lisp_Vectorlike);
> - IGC_MA_NEXT (xv, slot) = IGC_MA_HEAD (xv);
> + Lisp_Object head = IGC_MA_HEAD (xv);
> + igc_collect ();
> + IGC_MA_NEXT (xv, slot) = head;
> IGC_MA_PREV (xv, slot) = make_fixnum (-1);
> IGC_MA_HEAD (xv) = make_fixnum (slot);
> ptrdiff_t next = XFIXNUM (IGC_MA_NEXT (xv, slot));
>
> Then, run the code below with
>
> ./src/emacs -Q -batch -l /tmp/m.el -f test
>
> For me, Emacs seems to enter an endless loop after creating 36 markers.
>
> Helmut
That's not what I meant, sorry, let me try to ask differently. Without
modifying the code, which situation would you call risky? Is it a signal
that interrupts the mutator, at the place above for instance? And then
lets MPS do what? Or is it a concern for the case that MPS becomes
concurrent?
Sorry if I'm a bit dense today. It's Rosenmontag, after all :-).
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 09:30:03 GMT)
Full text and
rfc822 format available.
Message #119 received at 76538 <at> debbugs.gnu.org (full text, mbox):
On Mon, Mar 03 2025, Gerd Möllmann wrote:
> That's not what I meant, sorry, let me try to ask differently. Without
> modifying the code, which situation would you call risky? Is it a signal
> that interrupts the mutator, at the place above for instance? And then
> lets MPS do what? Or is it a concern for the case that MPS becomes
> concurrent?
Yes, concurrency. A concurrent mutator thread could trigger GC at any
moment
Helmut
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 09:38:01 GMT)
Full text and
rfc822 format available.
Message #122 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Helmut Eller <eller.helmut <at> gmail.com> writes:
> On Mon, Mar 03 2025, Gerd Möllmann wrote:
>
>> That's not what I meant, sorry, let me try to ask differently. Without
>> modifying the code, which situation would you call risky? Is it a signal
>> that interrupts the mutator, at the place above for instance? And then
>> lets MPS do what? Or is it a concern for the case that MPS becomes
>> concurrent?
>
> Yes, concurrency. A concurrent mutator thread could trigger GC at any
> moment
>
> Helmut
Thanks, and I agree of course. Something for future generations to
solve, from my POV at least. Should it ever happen :-).
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 10:43:01 GMT)
Full text and
rfc822 format available.
Message #125 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
> Helmut Eller <eller.helmut <at> gmail.com> writes:
>
>> On Mon, Mar 03 2025, Gerd Möllmann wrote:
>>
>>> That's not what I meant, sorry, let me try to ask differently. Without
>>> modifying the code, which situation would you call risky? Is it a signal
>>> that interrupts the mutator, at the place above for instance? And then
>>> lets MPS do what? Or is it a concern for the case that MPS becomes
>>> concurrent?
>>
>> Yes, concurrency. A concurrent mutator thread could trigger GC at any
>> moment
>>
>> Helmut
>
> Thanks, and I agree of course. Something for future generations to
> solve, from my POV at least. Should it ever happen :-).
Here's my current patch series for finalizing markers. Still needs
testing to see whether it actually avoids the problem (or will once we
stop the world for finalizers, as we should), but it doesn't crash
immediately.
BTW, I'm currently running with a background thread eagerly triggering
GC, which allowed me to find the missing bt.function trace, but hasn't
uncovered other bugs so far. So that is concurrent GC, even if it's
unlikely to be a performance win right now :-)
I have no idea why I hit the assert which required the second patch. It
seems to me like a bug in the xdisp.c code, but it's xdisp.c, so who
knows?
Even though we now emulate the alloc.c code quite precisely, I think
feature/igc will accumulate many more markers than alloc.c, since weak
objects are collected quite rarely by MPS, IME. Maybe we need to mark
"automatic" markers which are never exposed to Lisp and splat them in
DO_MARKERS if there are too many of them?
A more convoluted approach would be to alternate between considering
markers and calculating the charpos for the "best" known marker: do one
marker, then one character, repeat. That sort of thing is good for
theoretical complexity but rarely useful in practice...
From 31f3a7f25efe3840d42535f65147c40a7298357b Mon Sep 17 00:00:00 2001
From: Pip Cet <pipcet <at> protonmail.com>
Subject: [PATCH 1/2] [MPS] finalize markers rather than unchaining them in the
collector
* src/buffer.h (marker_it_next): Skip splatted markers.
* src/igc.c (fix_marker_vector): Don't call 'unchain'.
(finalize_marker): New function, calling 'unchain'.
(finalize_vector): Finalize markers.
(maybe_finalize): Adjust
---
src/buffer.h | 8 ++++++--
src/igc.c | 25 +++++++++++++++++++------
2 files changed, 25 insertions(+), 8 deletions(-)
diff --git a/src/buffer.h b/src/buffer.h
index ffe6770ba7b..5a460b9dcee 100644
--- a/src/buffer.h
+++ b/src/buffer.h
@@ -775,8 +775,12 @@ marker_it_valid (struct marker_it *it)
INLINE void
marker_it_next (struct marker_it *it)
{
- it->slot = XFIXNUM (IGC_MA_NEXT (it->v, it->slot));
- it->obj = it->slot < 0 ? Qnil : IGC_MA_MARKER (it->v, it->slot);
+ do
+ {
+ it->slot = XFIXNUM (IGC_MA_NEXT (it->v, it->slot));
+ it->obj = it->slot < 0 ? Qnil : IGC_MA_MARKER (it->v, it->slot);
+ }
+ while (!MARKERP (it->obj) && it->slot > 0);
}
INLINE struct Lisp_Marker *
diff --git a/src/igc.c b/src/igc.c
index ec714cad435..54c56309502 100644
--- a/src/igc.c
+++ b/src/igc.c
@@ -2165,10 +2165,6 @@ fix_marker_vector (mps_ss_t ss, struct Lisp_Vector *v)
Lisp_Object old = IGC_MA_MARKER (v, slot);
IGC_FIX12_OBJ (ss, &IGC_MA_MARKER (v, slot));
-
- /* FIXME/igc: this is right for marker vectors only. */
- if (NILP (IGC_MA_MARKER (v, slot)) && !NILP (old))
- unchain (v, slot);
}
}
MPS_SCAN_END (ss);
@@ -3567,6 +3563,20 @@ finalize_finalizer (struct Lisp_Finalizer *f)
}
}
+static void
+finalize_marker (struct Lisp_Marker *m)
+{
+ if (m->buffer && BUF_MARKERS (m->buffer))
+ {
+ struct Lisp_Vector *v = XVECTOR (BUF_MARKERS (m->buffer));
+ if (!BASE_EQ (IGC_MA_MARKER (v, m->slot), make_lisp_ptr (m, Lisp_Vectorlike)) &&
+ !NILP (IGC_MA_MARKER (v, m->slot)))
+ emacs_abort ();
+ unchain (v, m->slot);
+ fprintf (stderr, "unchained %zd\n", (size_t) m->slot);
+ }
+}
+
/* Turn an existing pseudovector into a PVEC_FREE, keeping its size. */
static void
@@ -3641,6 +3651,10 @@ finalize_vector (mps_addr_t v)
finalize_finalizer (v);
break;
+ case PVEC_MARKER:
+ finalize_marker (v);
+ break;
+
#ifndef IN_MY_FORK
case PVEC_OBARRAY:
#endif
@@ -3669,7 +3683,6 @@ finalize_vector (mps_addr_t v)
case PVEC_XWIDGET:
case PVEC_XWIDGET_VIEW:
case PVEC_TERMINAL:
- case PVEC_MARKER:
case PVEC_MODULE_GLOBAL_REFERENCE:
igc_assert (!"finalization not implemented");
break;
@@ -3745,6 +3758,7 @@ maybe_finalize (mps_addr_t client, enum pvec_type tag)
case PVEC_NATIVE_COMP_UNIT:
case PVEC_SUBR:
case PVEC_FINALIZER:
+ case PVEC_MARKER:
{
mps_res_t res = mps_finalize (global_igc->arena, &ref);
IGC_CHECK_RES (res);
@@ -3759,7 +3773,6 @@ maybe_finalize (mps_addr_t client, enum pvec_type tag)
case PVEC_WEAK_HASH_TABLE:
case PVEC_NORMAL_VECTOR:
case PVEC_FREE:
- case PVEC_MARKER:
case PVEC_OVERLAY:
case PVEC_SYMBOL_WITH_POS:
case PVEC_MISC_PTR:
--
2.48.1
From 9148e4c387294b06a471c25d07279d207b1d45c9 Mon Sep 17 00:00:00 2001
From: Pip Cet <pipcet <at> protonmail.com>
Subject: [PATCH 2/2] XXX why is this change necessary?
---
src/xdisp.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/src/xdisp.c b/src/xdisp.c
index 577d5b1b401..2c349bb9adf 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -6714,9 +6714,15 @@ handle_composition_prop (struct it *it)
string = it->string;
s = SDATA (string) + pos_byte;
if (STRING_MULTIBYTE (string))
- it->c = STRING_CHAR (s);
+ {
+ it->c = STRING_CHAR (s);
+ it->len = CHAR_BYTES (it->c);
+ }
else
- it->c = *s;
+ {
+ it->c = *s;
+ it->len = 1;
+ }
}
else
{
@@ -6724,6 +6730,7 @@ handle_composition_prop (struct it *it)
pos_byte = IT_BYTEPOS (*it);
string = Qnil;
it->c = FETCH_CHAR (pos_byte);
+ it->len = it->multibyte_p ? CHAR_BYTES (it->c) : 1;
}
/* If there's a valid composition and point is not inside of the
--
2.48.1
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 11:34:02 GMT)
Full text and
rfc822 format available.
Message #128 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Pip Cet <pipcet <at> protonmail.com> writes:
> Here's my current patch series for finalizing markers. Still needs
> testing to see whether it actually avoids the problem (or will once we
> stop the world for finalizers, as we should), but it doesn't crash
> immediately.
>
> BTW, I'm currently running with a background thread eagerly triggering
> GC, which allowed me to find the missing bt.function trace, but hasn't
> uncovered other bugs so far. So that is concurrent GC, even if it's
> unlikely to be a performance win right now :-)
:-).
> I have no idea why I hit the assert which required the second patch. It
> seems to me like a bug in the xdisp.c code, but it's xdisp.c, so who
> knows?
Yeah, that doesn't look right. Normally, string_char_and_length should
have been used to get characters from a C string.
> Even though we now emulate the alloc.c code quite precisely, I think
> feature/igc will accumulate many more markers than alloc.c, since weak
> objects are collected quite rarely by MPS, IME. Maybe we need to mark
> "automatic" markers which are never exposed to Lisp and splat them in
> DO_MARKERS if there are too many of them?
>
> A more convoluted approach would be to alternate between considering
> markers and calculating the charpos for the "best" known marker: do one
> marker, then one character, repeat. That sort of thing is good for
> theoretical complexity but rarely useful in practice...
I'd like to see something not using markers at all, TBH.
Otherwise, LGTM. Only thing I saw is this:
> +static void
> +finalize_marker (struct Lisp_Marker *m)
> +{
> + if (m->buffer && BUF_MARKERS (m->buffer))
^^^^^^^^^^^
NILP
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 14:02:02 GMT)
Full text and
rfc822 format available.
Message #131 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
> Pip Cet <pipcet <at> protonmail.com> writes:
>
>> Here's my current patch series for finalizing markers. Still needs
>> testing to see whether it actually avoids the problem (or will once we
>> stop the world for finalizers, as we should), but it doesn't crash
>> immediately.
>>
>> BTW, I'm currently running with a background thread eagerly triggering
>> GC, which allowed me to find the missing bt.function trace, but hasn't
>> uncovered other bugs so far. So that is concurrent GC, even if it's
>> unlikely to be a performance win right now :-)
>
> :-).
>
>> I have no idea why I hit the assert which required the second patch. It
>> seems to me like a bug in the xdisp.c code, but it's xdisp.c, so who
>> knows?
>
> Yeah, that doesn't look right. Normally, string_char_and_length should
> have been used to get characters from a C string.
>
>> Even though we now emulate the alloc.c code quite precisely, I think
>> feature/igc will accumulate many more markers than alloc.c, since weak
>> objects are collected quite rarely by MPS, IME. Maybe we need to mark
>> "automatic" markers which are never exposed to Lisp and splat them in
>> DO_MARKERS if there are too many of them?
>>
>> A more convoluted approach would be to alternate between considering
>> markers and calculating the charpos for the "best" known marker: do one
>> marker, then one character, repeat. That sort of thing is good for
>> theoretical complexity but rarely useful in practice...
>
> I'd like to see something not using markers at all, TBH.
>
> Otherwise, LGTM. Only thing I saw is this:
>
>> +static void
>> +finalize_marker (struct Lisp_Marker *m)
>> +{
>> + if (m->buffer && BUF_MARKERS (m->buffer))
> ^^^^^^^^^^^
> NILP
Thanks! The fprintf should also go, but testing revealed a somewhat
more difficult problem: when garbage_collection_messages is true, the
maybe_process_messages call in maybe_finalize can try to print to a
buffer, which is bad because we're in the middle of tearing down one. I
think we should remove that call, to be honest, and put it in maybe_quit
instead, since some messages may cause us to print messages or run Lisp.
Alternatively, we could use pending_funcalls or a similar mechanism.
WDYT?
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 14:14:03 GMT)
Full text and
rfc822 format available.
Message #134 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> Date: Mon, 03 Mar 2025 10:42:25 +0000
> From: Pip Cet <pipcet <at> protonmail.com>
> Cc: Helmut Eller <eller.helmut <at> gmail.com>, Stefan Monnier <monnier <at> iro.umontreal.ca>, Ihor Radchenko <yantar92 <at> posteo.net>, Eli Zaretskii <eliz <at> gnu.org>, joaomoreira <at> gmx.se, 76538 <at> debbugs.gnu.org
>
> I have no idea why I hit the assert which required the second patch.
Which assertion was that? Is there any way of triggering that on the
master branch, or on the igc branch without first applying the patch?
> It seems to me like a bug in the xdisp.c code, but it's xdisp.c, so
> who knows?
I suspect it->len is not used when the character is composed, because
the actual character in the buffer text is not relevant in that case,
as the results of the composition determine how many bytes to skip to
get to the next "display element".
> - it->c = STRING_CHAR (s);
> + {
> + it->c = STRING_CHAR (s);
> + it->len = CHAR_BYTES (it->c);
> + }
This is sub-optimal; please use string_char_and_length instead.
> @@ -6724,6 +6730,7 @@ handle_composition_prop (struct it *it)
> pos_byte = IT_BYTEPOS (*it);
> string = Qnil;
> it->c = FETCH_CHAR (pos_byte);
> + it->len = it->multibyte_p ? CHAR_BYTES (it->c) : 1;
Likewise here.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 14:25:02 GMT)
Full text and
rfc822 format available.
Message #137 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> Even though we now emulate the alloc.c code quite precisely, I think
> feature/igc will accumulate many more markers than alloc.c, since weak
> objects are collected quite rarely by MPS, IME.
One of the benefits of using a better markers cache would be to reduce
reliance on the GC, indeed. E.g. my array-with-gap will tend not to
accumulate cache-markers indefinitely because it always finds the
closest (cache-)marker before deciding whether we need to add
another one.
[ Of course, if the GC never triggers, cache-markers can still
accumulate needlessly, e.g. if you regularly `erase-buffer`, which
causes all the (cache-)markers to be moved to point-min, or in
a comint buffer where you regularly cut the beginning to keep the
buffer from growing indefinitely. ]
> Maybe we need to mark "automatic" markers which are never exposed to
> Lisp and splat them in DO_MARKERS if there are too many of them?
Getting rid of the GC's involvement in flushing the cache would be good.
Doing it at every DO_MARKERS might be requires care if we don't want it
to be too costly.
[ Also, with the current un-ordered setup, it could be hard to know
which ones to throw out. Again, a better cache would help in this
regard. ]
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 14:32:02 GMT)
Full text and
rfc822 format available.
Message #140 received at 76538 <at> debbugs.gnu.org (full text, mbox):
Pip Cet <pipcet <at> protonmail.com> writes:
> Thanks! The fprintf should also go, but testing revealed a somewhat
> more difficult problem: when garbage_collection_messages is true, the
> maybe_process_messages call in maybe_finalize can try to print to a
> buffer, which is bad because we're in the middle of tearing down one. I
> think we should remove that call, to be honest, and put it in maybe_quit
> instead, since some messages may cause us to print messages or run Lisp.
>
> Alternatively, we could use pending_funcalls or a similar mechanism.
>
> WDYT?
Sounds okay. The simpler the better, I'd say.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 14:48:01 GMT)
Full text and
rfc822 format available.
Message #143 received at 76538 <at> debbugs.gnu.org (full text, mbox):
"Eli Zaretskii" <eliz <at> gnu.org> writes:
>> Date: Mon, 03 Mar 2025 10:42:25 +0000
>> From: Pip Cet <pipcet <at> protonmail.com>
>> Cc: Helmut Eller <eller.helmut <at> gmail.com>, Stefan Monnier
>> <monnier <at> iro.umontreal.ca>, Ihor Radchenko <yantar92 <at> posteo.net>,
>> Eli Zaretskii <eliz <at> gnu.org>, joaomoreira <at> gmx.se,
>> 76538 <at> debbugs.gnu.org
>>
>> I have no idea why I hit the assert which required the second patch.
>
> Which assertion was that? Is there any way of triggering that on the
> master branch, or on the igc branch without first applying the patch?
Not on the master branch itself, I have to change the "5000" to "50000"
in the marker code, which I don't think should cause this assertion
violation. The reproducer script is the magit.bash script Helmut
provided, this patch, and hitting C-p C-p C-n C-n when the magit window
opens.
diff --git a/src/marker.c b/src/marker.c
index 4ab68ec7bbe..61f7af3f904 100644
--- a/src/marker.c
+++ b/src/marker.c
@@ -97,6 +97,7 @@ #define CONSIDER(CHARPOS, BYTEPOS) \
{ \
ptrdiff_t this_charpos = (CHARPOS); \
bool changed = false; \
+ considered++; \
\
if (this_charpos == charpos) \
{ \
@@ -194,6 +195,7 @@ buf_charpos_to_bytepos (struct buffer *b, ptrdiff_t charpos)
two best approximations is all single-byte,
we interpolate the result immediately. */
+ ptrdiff_t considered = 0;
CONSIDER (BUF_PT (b), BUF_PT_BYTE (b));
CONSIDER (BUF_GPT (b), BUF_GPT_BYTE (b));
CONSIDER (BUF_BEGV (b), BUF_BEGV_BYTE (b));
@@ -219,7 +221,10 @@ buf_charpos_to_bytepos (struct buffer *b, ptrdiff_t charpos)
eassert (best_below <= charpos && charpos <= best_above);
if (charpos - best_below < best_above - charpos)
{
- bool record = charpos - best_below > 5000;
+ bool record = charpos - best_below > 50000;
+
+ if (record)
+ fprintf (stderr, "recording (1) starting at %ld %ld %ld\n", best_below, charpos, considered);
while (best_below < charpos)
{
@@ -244,7 +249,10 @@ buf_charpos_to_bytepos (struct buffer *b, ptrdiff_t charpos)
}
else
{
- bool record = best_above - charpos > 5000;
+ bool record = best_above - charpos > 50000;
+
+ if (record)
+ fprintf (stderr, "recording (2) starting at %ld %ld %ld\n", charpos, best_above, considered);
while (best_above > charpos)
{
@@ -278,6 +286,7 @@ #define CONSIDER(BYTEPOS, CHARPOS) \
{ \
ptrdiff_t this_bytepos = (BYTEPOS); \
int changed = false; \
+ considered++; \
\
if (this_bytepos == bytepos) \
{ \
@@ -342,6 +351,7 @@ buf_bytepos_to_charpos (struct buffer *b, ptrdiff_t bytepos)
best_below = BEG;
best_below_byte = BEG_BYTE;
+ ptrdiff_t considered = 0;
CONSIDER (BUF_PT_BYTE (b), BUF_PT (b));
CONSIDER (BUF_GPT_BYTE (b), BUF_GPT (b));
CONSIDER (BUF_BEGV_BYTE (b), BUF_BEGV (b));
@@ -366,7 +376,10 @@ buf_bytepos_to_charpos (struct buffer *b, ptrdiff_t bytepos)
if (bytepos - best_below_byte < best_above_byte - bytepos)
{
- bool record = bytepos - best_below_byte > 5000;
+ bool record = bytepos - best_below_byte > 50000;
+
+ if (record)
+ fprintf (stderr, "recording (3) starting at %ld %ld %ld\n", bytepos, best_below_byte, considered);
while (best_below_byte < bytepos)
{
@@ -393,8 +406,10 @@ buf_bytepos_to_charpos (struct buffer *b, ptrdiff_t bytepos)
}
else
{
- bool record = best_above_byte - bytepos > 5000;
+ bool record = best_above_byte - bytepos > 50000;
+ if (record)
+ fprintf (stderr, "recording (4) starting at %ld %ld %ld\n", bytepos, best_above_byte, considered);
while (best_above_byte > bytepos)
{
best_above--;
it looks like this:
it = {window = 0x555557caa01d, w = 0x555557caa018, f = 0x555555e74658, method = GET_FROM_BUFFER, stop_charpos = 27353858, prev_stop = 347705, base_level_stop = 0, end_charpos = 27353858, medium_narrowing_begv = 27244404, medium_narrowing_zv = 27353858, large_narrowing_begv = 27103730, large_narrowing_zv = 27353858, s = 0x0, string_nchars = 0, multibyte_p = true, tab_line_p = false, header_line_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, from_disp_prop_p = false, ellipsis_p = false, avoid_cursor_p = false, dp = 0x0, dpvec = 0x0, dpend = 0x0, dpvec_char_len = 0, dpvec_face_id = 0, saved_face_id = 0, ctl_chars = {0x0 <repeats 16 times>}, start = {pos = {charpos = 347705, bytepos = 348476}, overlay_string_index = -1, string_pos = {charpos = -1, bytepos = -1}, dpvec_index = -1}, current = {pos = {charpos = 27353857, bytepos = 27355172}, overlay_string_index = -1, string_pos = {charpos = -1, bytepos = -1}, dpvec_index = -1}, n_overlay_strings = 0, overlay_strings_charpos = 27353857, overlay_strings = {0x0 <repeats 16 times>}, string_overlays = {0x0 <repeats 16 times>}, string = 0x0, from_overlay = 0x0, stack = {{string = 0x0, string_nchars = 0, end_charpos = 0, stop_charpos = 0, prev_stop = 0, base_level_stop = 0, cmp_it = {stop_pos = 0, id = 0, ch = 0, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, face_id = 0, u = {image = {object = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, image_id = 0}, stretch = {object = 0x0}, xwidget = {object = 0x0}}, position = {charpos = 0, bytepos = 0}, current = {pos = {charpos = 0, bytepos = 0}, overlay_string_index = 0, string_pos = {charpos = 0, bytepos = 0}, dpvec_index = 0}, from_overlay = 0x0, area = LEFT_MARGIN_AREA, method = GET_FROM_BUFFER, paragraph_embedding = NEUTRAL_DIR, multibyte_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, display_ellipsis_p = false, avoid_cursor_p = false, bidi_p = false, from_disp_prop_p = false, line_wrap = TRUNCATE, voffset = 0, space_width = 0x0, font_height = 0x0}, {string = 0x0, string_nchars = 0, end_charpos = 0, stop_charpos = 0, prev_stop = 0, base_level_stop = 0, cmp_it = {stop_pos = 0, id = 0, ch = 0, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, face_id = 0, u = {image = {object = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, image_id = 0}, stretch = {object = 0x0}, xwidget = {object = 0x0}}, position = {charpos = 0, bytepos = 0}, current = {pos = {charpos = 0, bytepos = 0}, overlay_string_index = 0, string_pos = {charpos = 0, bytepos = 0}, dpvec_index = 0}, from_overlay = 0x0, area = LEFT_MARGIN_AREA, method = GET_FROM_BUFFER, paragraph_embedding = NEUTRAL_DIR, multibyte_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, display_ellipsis_p = false, avoid_cursor_p = false, bidi_p = false, from_disp_prop_p = false, line_wrap = TRUNCATE, voffset = 0, space_width = 0x0, font_height = 0x0}, {string = 0x0, string_nchars = 0, end_charpos = 0, stop_charpos = 0, prev_stop = 0, base_level_stop = 0, cmp_it = {stop_pos = 0, id = 0, ch = 0, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, face_id = 0, u = {image = {object = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, image_id = 0}, stretch = {object = 0x0}, xwidget = {object = 0x0}}, position = {charpos = 0, bytepos = 0}, current = {pos = {charpos = 0, bytepos = 0}, overlay_string_index = 0, string_pos = {charpos = 0, bytepos = 0}, dpvec_index = 0}, from_overlay = 0x0, area = LEFT_MARGIN_AREA, method = GET_FROM_BUFFER, paragraph_embedding = NEUTRAL_DIR, multibyte_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, display_ellipsis_p = false, avoid_cursor_p = false, bidi_p = false, from_disp_prop_p = false, line_wrap = TRUNCATE, voffset = 0, space_width = 0x0, font_height = 0x0}, {string = 0x0, string_nchars = 0, end_charpos = 0, stop_charpos = 0, prev_stop = 0, base_level_stop = 0, cmp_it = {stop_pos = 0, id = 0, ch = 0, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, face_id = 0, u = {image = {object = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, image_id = 0}, stretch = {object = 0x0}, xwidget = {object = 0x0}}, position = {charpos = 0, bytepos = 0}, current = {pos = {charpos = 0, bytepos = 0}, overlay_string_index = 0, string_pos = {charpos = 0, bytepos = 0}, dpvec_index = 0}, from_overlay = 0x0, area = LEFT_MARGIN_AREA, method = GET_FROM_BUFFER, paragraph_embedding = NEUTRAL_DIR, multibyte_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, display_ellipsis_p = false, avoid_cursor_p = false, bidi_p = false, from_disp_prop_p = false, line_wrap = TRUNCATE, voffset = 0, space_width = 0x0, font_height = 0x0}, {string = 0x0, string_nchars = 0, end_charpos = 0, stop_charpos = 0, prev_stop = 0, base_level_stop = 0, cmp_it = {stop_pos = 0, id = 0, ch = 0, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, face_id = 0, u = {image = {object = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, image_id = 0}, stretch = {object = 0x0}, xwidget = {object = 0x0}}, position = {charpos = 0, bytepos = 0}, current = {pos = {charpos = 0, bytepos = 0}, overlay_string_index = 0, string_pos = {charpos = 0, bytepos = 0}, dpvec_index = 0}, from_overlay = 0x0, area = LEFT_MARGIN_AREA, method = GET_FROM_BUFFER, paragraph_embedding = NEUTRAL_DIR, multibyte_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, display_ellipsis_p = false, avoid_cursor_p = false, bidi_p = false, from_disp_prop_p = false, line_wrap = TRUNCATE, voffset = 0, space_width = 0x0, font_height = 0x0}}, sp = 0, selective = 0, what = IT_CHARACTER, face_id = 0, selective_display_ellipsis_p = true, ctl_arrow_p = true, face_box_p = false, start_of_box_run_p = false, end_of_box_run_p = false, overlay_strings_at_end_processed_p = false, ignore_overlay_strings_at_pos_p = false, glyph_not_available_p = false, starts_in_middle_of_char_p = false, face_before_selective_p = false, constrain_row_ascent_descent_p = false, line_number_produced_p = false, align_visually_p = false, line_wrap = TRUNCATE, base_face_id = 0, c = 10, len = 0, cmp_it = {stop_pos = 27353857, id = -1, ch = -2, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x7fffffffb8f0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, char_to_display = 0, glyphless_method = GLYPHLESS_DISPLAY_THIN_SPACE, image_id = 0, xwidget = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, space_width = 0x0, voffset = 0, tab_width = 8, font_height = 0x0, object = 0x555557cdf2bd, position = {charpos = 27353857, bytepos = 27355172}, truncation_pixel_width = 9, continuation_pixel_width = 0, first_visible_x = 0, last_visible_x = 1881, last_visible_y = 1836, extra_line_spacing = 0, max_extra_line_spacing = 0, override_ascent = -1, override_descent = 0, override_boff = 0, glyph_row = 0x0, area = TEXT_AREA, nglyphs = 1, pixel_width = 0, ascent = 0, descent = 0, max_ascent = 0, max_descent = 0, phys_ascent = 0, phys_descent = 0, max_phys_ascent = 0, max_phys_descent = 0, current_x = 0, wrap_prefix_width = 0, continuation_lines_width = 0, eol_pos = {charpos = 0, bytepos = 0}, current_y = 396, first_vpos = 0, vpos = 22, hpos = 0, lnum = 0, lnum_bytepos = 0, lnum_width = 0, lnum_pixel_width = 0, pt_lnum = 0, stretch_adjust = 0, left_user_fringe_bitmap = 0, right_user_fringe_bitmap = 0, left_user_fringe_face_id = 0, right_user_fringe_face_id = 0, bidi_p = true, bidi_it = {bytepos = 27355172, charpos = 27353857, ch = 10, nchars = 1, ch_len = 1, type = NEUTRAL_B, type_after_wn = NEUTRAL_B, orig_type = NEUTRAL_B, resolved_level = 0 '\000', isolate_level = 0 '\000', invalid_levels = 0, invalid_isolates = 0, prev = {charpos = 0, type = UNKNOWN_BT, orig_type = UNKNOWN_BT}, last_strong = {charpos = 0, type = UNKNOWN_BT, orig_type = UNKNOWN_BT}, next_for_neutral = {charpos = -1, type = UNKNOWN_BT, orig_type = UNKNOWN_BT}, prev_for_neutral = {charpos = -1, type = UNKNOWN_BT, orig_type = UNKNOWN_BT}, next_for_ws = {charpos = 0, type = UNKNOWN_BT, orig_type = UNKNOWN_BT}, bracket_pairing_pos = -1, bracket_enclosed_type = UNKNOWN_BT, next_en_pos = 0, next_en_type = UNKNOWN_BT, sos = L2R, scan_dir = 0, disp_pos = -1, disp_prop = 0, stack_idx = 0, level_stack = {{next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'} <repeats 128 times>}, string = {lstring = 0x0, s = 0x0, schars = 0, bufpos = 0, from_disp_str = false, unibyte = false}, w = 0x555557caa018, paragraph_dir = NEUTRAL_DIR, separator_limit = -1, first_elt = false, new_paragraph = true, frame_window_p = true}, paragraph_embedding = NEUTRAL_DIR, min_width_property = 0x0, min_width_start = 0}
Should this combination ever be valid?
it->what = IT_CHARACTER
it->c = 10
it->len = 0
>> It seems to me like a bug in the xdisp.c code, but it's xdisp.c, so
>> who knows?
>
> I suspect it->len is not used when the character is composed, because
The character isn't composed, is it? It's a plain newline.
>> - it->c = STRING_CHAR (s);
>> + {
>> + it->c = STRING_CHAR (s);
>> + it->len = CHAR_BYTES (it->c);
>> + }
>
> This is sub-optimal; please use string_char_and_length instead.
Will do, if we need to fix this at all.
>> @@ -6724,6 +6730,7 @@ handle_composition_prop (struct it *it)
>> pos_byte = IT_BYTEPOS (*it);
>> string = Qnil;
>> it->c = FETCH_CHAR (pos_byte);
>> + it->len = it->multibyte_p ? CHAR_BYTES (it->c) : 1;
>
> Likewise here.
I was looking for fetch_char_and_length, but I guess
string_char_and_length (BYTE_POS_ADDR (pos_byte), &it->len) works, too?
I'll try reducing the patch to just the 5000 -> 50000 thing next.
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 15:34:04 GMT)
Full text and
rfc822 format available.
Message #146 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> Date: Mon, 03 Mar 2025 14:46:55 +0000
> From: Pip Cet <pipcet <at> protonmail.com>
> Cc: gerd.moellmann <at> gmail.com, eller.helmut <at> gmail.com, monnier <at> iro.umontreal.ca, yantar92 <at> posteo.net, joaomoreira <at> gmx.se, 76538 <at> debbugs.gnu.org
>
> Should this combination ever be valid?
>
> it->what = IT_CHARACTER
> it->c = 10
> it->len = 0
>
> >> It seems to me like a bug in the xdisp.c code, but it's xdisp.c, so
> >> who knows?
> >
> > I suspect it->len is not used when the character is composed, because
>
> The character isn't composed, is it? It's a plain newline.
AFAIR, the value of it->len for a newline is never used, because we
are at the end of a line, and will then jump to the next visible line
anyway (or stop iteration altogether).
Which is not to say I'm against the changes, I'm just trying to
explain why it never caused us any assertion violations till now.
> >> - it->c = STRING_CHAR (s);
> >> + {
> >> + it->c = STRING_CHAR (s);
> >> + it->len = CHAR_BYTES (it->c);
> >> + }
> >
> > This is sub-optimal; please use string_char_and_length instead.
>
> Will do, if we need to fix this at all.
>
> >> @@ -6724,6 +6730,7 @@ handle_composition_prop (struct it *it)
> >> pos_byte = IT_BYTEPOS (*it);
> >> string = Qnil;
> >> it->c = FETCH_CHAR (pos_byte);
> >> + it->len = it->multibyte_p ? CHAR_BYTES (it->c) : 1;
> >
> > Likewise here.
>
> I was looking for fetch_char_and_length, but I guess
> string_char_and_length (BYTE_POS_ADDR (pos_byte), &it->len) works, too?
Yes.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 15:55:01 GMT)
Full text and
rfc822 format available.
Message #149 received at 76538 <at> debbugs.gnu.org (full text, mbox):
"Eli Zaretskii" <eliz <at> gnu.org> writes:
>> Date: Mon, 03 Mar 2025 14:46:55 +0000
>> From: Pip Cet <pipcet <at> protonmail.com>
>> Cc: gerd.moellmann <at> gmail.com, eller.helmut <at> gmail.com, monnier <at> iro.umontreal.ca, yantar92 <at> posteo.net, joaomoreira <at> gmx.se, 76538 <at> debbugs.gnu.org
>>
>> Should this combination ever be valid?
>>
>> it->what = IT_CHARACTER
>> it->c = 10
>> it->len = 0
>>
>> >> It seems to me like a bug in the xdisp.c code, but it's xdisp.c, so
>> >> who knows?
>> >
>> > I suspect it->len is not used when the character is composed, because
>>
>> The character isn't composed, is it? It's a plain newline.
>
> AFAIR, the value of it->len for a newline is never used, because we
> are at the end of a line, and will then jump to the next visible line
> anyway (or stop iteration altogether).
The way I read set_iterator_to_next:
set_iterator_to_next (struct it *it, bool reseat_p)
{
if (max_redisplay_ticks > 0)
update_redisplay_ticks (1, it->w);
switch (it->method)
{
case GET_FROM_BUFFER:
/* The current display element of IT is a character from
current_buffer. Advance in the buffer, and maybe skip over
invisible lines that are so because of selective display. */
if (ITERATOR_AT_END_OF_LINE_P (it) && reseat_p)
reseat_at_next_visible_line_start (it, false);
...
else
{
eassert (it->len != 0);
it depends on the value of reseat_p, which is false in my case, where
it's called from here:
if (it->what == IT_CHARACTER
&& it->c == '\n'
&& CHARPOS (it->position) == IT_CHARPOS (*it))
{
if (it->bidi_p && bidi_it_prev)
*bidi_it_prev = it->bidi_it;
set_iterator_to_next (it, false);
it->c = 0;
return true;
}
> Which is not to say I'm against the changes, I'm just trying to
> explain why it never caused us any assertion violations till now.
Thank you very much, I appreciate that! I too wonder how creating fewer
cache markers somehow causes us to run into a problem that the master
branch (with one marker per 5000 chars) doesn't, and we should try to
understand that before considering whether to skip over long Qinvisible
sections in back_to_previous_line_start etc, IMHO (it makes magit more
usable but I'm not really comfortable suggesting changes to that code
given my current (lack of) understanding).
Pip
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 16:44:02 GMT)
Full text and
rfc822 format available.
Message #152 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> Date: Mon, 03 Mar 2025 15:54:02 +0000
> From: Pip Cet <pipcet <at> protonmail.com>
> Cc: gerd.moellmann <at> gmail.com, eller.helmut <at> gmail.com, monnier <at> iro.umontreal.ca, yantar92 <at> posteo.net, joaomoreira <at> gmx.se, 76538 <at> debbugs.gnu.org
>
> "Eli Zaretskii" <eliz <at> gnu.org> writes:
>
> > AFAIR, the value of it->len for a newline is never used, because we
> > are at the end of a line, and will then jump to the next visible line
> > anyway (or stop iteration altogether).
>
> The way I read set_iterator_to_next:
>
> set_iterator_to_next (struct it *it, bool reseat_p)
> {
>
> if (max_redisplay_ticks > 0)
> update_redisplay_ticks (1, it->w);
>
> switch (it->method)
> {
> case GET_FROM_BUFFER:
> /* The current display element of IT is a character from
> current_buffer. Advance in the buffer, and maybe skip over
> invisible lines that are so because of selective display. */
> if (ITERATOR_AT_END_OF_LINE_P (it) && reseat_p)
> reseat_at_next_visible_line_start (it, false);
> ...
> else
> {
> eassert (it->len != 0);
>
> it depends on the value of reseat_p, which is false in my case, where
> it's called from here:
>
> if (it->what == IT_CHARACTER
> && it->c == '\n'
> && CHARPOS (it->position) == IT_CHARPOS (*it))
> {
> if (it->bidi_p && bidi_it_prev)
> *bidi_it_prev = it->bidi_it;
> set_iterator_to_next (it, false);
> it->c = 0;
> return true;
> }
I meant this place in display_line, which is where we handle display
iteration that reaches EOL:
if (ITERATOR_AT_END_OF_LINE_P (it))
{
int used_before = row->used[TEXT_AREA];
[...]
/* Consume the line end. This skips over invisible lines. */
set_iterator_to_next (it, true); <<<<<<<<<<<<<<<<<<<<
it->continuation_lines_width = 0;
break;
}
As for the call with 'false' as 2nd arg: First, when it->bidi_p is
true, we don't use it->len for moving to the next display element.
And second, this place (and another similar to it) are in functions
that are used to jump far away, so they are always followed by a call
to 'reseat'.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 17:08:02 GMT)
Full text and
rfc822 format available.
Message #155 received at 76538 <at> debbugs.gnu.org (full text, mbox):
"Eli Zaretskii" <eliz <at> gnu.org> writes:
>> Date: Mon, 03 Mar 2025 15:54:02 +0000
>> From: Pip Cet <pipcet <at> protonmail.com>
>> Cc: gerd.moellmann <at> gmail.com, eller.helmut <at> gmail.com, monnier <at> iro.umontreal.ca, yantar92 <at> posteo.net, joaomoreira <at> gmx.se, 76538 <at> debbugs.gnu.org
>>
>> "Eli Zaretskii" <eliz <at> gnu.org> writes:
>>
>> > AFAIR, the value of it->len for a newline is never used, because we
>> > are at the end of a line, and will then jump to the next visible line
>> > anyway (or stop iteration altogether).
>>
>> The way I read set_iterator_to_next:
>>
>> set_iterator_to_next (struct it *it, bool reseat_p)
>> {
>>
>> if (max_redisplay_ticks > 0)
>> update_redisplay_ticks (1, it->w);
>>
>> switch (it->method)
>> {
>> case GET_FROM_BUFFER:
>> /* The current display element of IT is a character from
>> current_buffer. Advance in the buffer, and maybe skip over
>> invisible lines that are so because of selective display. */
>> if (ITERATOR_AT_END_OF_LINE_P (it) && reseat_p)
>> reseat_at_next_visible_line_start (it, false);
>> ...
>> else
>> {
>> eassert (it->len != 0);
>>
>> it depends on the value of reseat_p, which is false in my case, where
>> it's called from here:
>>
>> if (it->what == IT_CHARACTER
>> && it->c == '\n'
>> && CHARPOS (it->position) == IT_CHARPOS (*it))
>> {
>> if (it->bidi_p && bidi_it_prev)
>> *bidi_it_prev = it->bidi_it;
>> set_iterator_to_next (it, false);
>> it->c = 0;
>> return true;
>> }
>
> I meant this place in display_line, which is where we handle display
> iteration that reaches EOL:
>
> if (ITERATOR_AT_END_OF_LINE_P (it))
> {
> int used_before = row->used[TEXT_AREA];
> [...]
> /* Consume the line end. This skips over invisible lines. */
> set_iterator_to_next (it, true); <<<<<<<<<<<<<<<<<<<<
> it->continuation_lines_width = 0;
> break;
> }
>
> As for the call with 'false' as 2nd arg: First, when it->bidi_p is
> true, we don't use it->len for moving to the next display element.
True. Moving the eassert inside the if might be the minimal change
there.
> And second, this place (and another similar to it) are in functions
> that are used to jump far away, so they are always followed by a call
> to 'reseat'.
(gdb) bt
#0 0x00007ffff5b38e3c in ??? () at /lib64/libc.so.6
#1 0x00007ffff5ae2da6 in raise () at /lib64/libc.so.6
#2 0x000055555578ac0f in terminate_due_to_signal (sig=6, backtrace_limit=2147483647) at emacs.c:462
#3 0x0000555555842adb in die (msg=0x5555559ab358 "it->len != 0", file=0x5555559aa5d7 "xdisp.c", line=8823) at alloc.c:7444
#4 0x00005555555f7b8c in set_iterator_to_next (it=0x7fffffffb8e0, reseat_p=false) at xdisp.c:8823
#5 0x00005555555f4040 in forward_to_next_line_start (it=0x7fffffffb8e0, skipped_p=0x7fffffffae9e, bidi_it_prev=0x0) at xdisp.c:7541
#6 0x00005555555f4e57 in reseat_at_next_visible_line_start (it=0x7fffffffb8e0, on_newline_p=false) at xdisp.c:7781
#7 0x0000555555611609 in redisplay_internal () at xdisp.c:17408
#8 0x000055555560f596 in redisplay () at xdisp.c:16672
#9 0x0000555555797f4e in read_char (commandflag=1, map=0x555557f08453, prev_event=0x0, used_mouse_menu=0x7fffffffd025, end_time=0x0) at keyboard.c:2672
#10 0x00005555557acd81 in read_key_sequence
(keybuf=0x7fffffffd2a0, prompt=0x0, dont_downcase_last=false, can_return_switch_frame=true, fix_current_buffer=true, prevent_redisplay=false, disable_text_conversion_p=false) at keyboard.c:10757
#11 0x0000555555793fb4 in command_loop_1 () at keyboard.c:1424
#12 0x000055555587bb35 in internal_condition_case (bfun=0x555555793b66 <command_loop_1>, handlers=0x90, hfun=0x555555792f64 <cmd_error>) at eval.c:1602
#13 0x0000555555793710 in command_loop_2 (handlers=0x90) at keyboard.c:1163
#14 0x000055555587af29 in internal_catch (tag=0x12420, func=0x5555557936e2 <command_loop_2>, arg=0x90) at eval.c:1282
#15 0x000055555579369e in command_loop () at keyboard.c:1141
#16 0x00005555557929e6 in recursive_edit_1 () at keyboard.c:749
#17 0x0000555555792c1a in Frecursive_edit () at keyboard.c:832
#18 0x000055555578e1d8 in main (argc=4, argv=0x7fffffffd778) at
emacs.c:2562
(gdb) bt full
#0 0x00007ffff5b38e3c in ??? () at /lib64/libc.so.6
#1 0x00007ffff5ae2da6 in raise () at /lib64/libc.so.6
#2 0x000055555578ac0f in terminate_due_to_signal (sig=6, backtrace_limit=2147483647) at emacs.c:462
#3 0x0000555555842adb in die (msg=0x5555559ab358 "it->len != 0", file=0x5555559aa5d7 "xdisp.c", line=8823) at alloc.c:7444
#4 0x00005555555f7b8c in set_iterator_to_next (it=0x7fffffffb8e0, reseat_p=false) at xdisp.c:8823
#5 0x00005555555f4040 in forward_to_next_line_start (it=0x7fffffffb8e0, skipped_p=0x7fffffffae9e, bidi_it_prev=0x0) at xdisp.c:7541
old_selective = 0
newline_found_p = false
n = 0
MAX_NEWLINE_DISTANCE = 500
#6 0x00005555555f4e57 in reseat_at_next_visible_line_start (it=0x7fffffffb8e0, on_newline_p=false) at xdisp.c:7781
skipped_p = false
bidi_it_prev = {bytepos = 35802240, charpos = 35802240, ch = 314, nchars = 16, ch_len = 140737488334608, type = 1435070560, type_after_wn = 21845, orig_type = 4294946544, resolved_level = -1 '\377', isolate_level = 127 '\177', invalid_levels = 38442363915953017, invalid_isolates = 93824998311872, prev = {charpos = 79200, type = 79200, orig_type = UNKNOWN_BT}, last_strong = {charpos = 79200, type = 1433343524, orig_type = 21845}, next_for_neutral = {charpos = 33216, type = 4294946624, orig_type = 32767}, prev_for_neutral = {charpos = 93824993945891, type = 33216, orig_type = UNKNOWN_BT}, next_for_ws = {charpos = 93825034033728, type = UNKNOWN_BT, orig_type = UNKNOWN_BT}, bracket_pairing_pos = 33216, bracket_enclosed_type = 4294946672, next_en_pos = 93825034033725, next_en_type = 4294946704, sos = (L2R | R2L | unknown: 0x7ffc), scan_dir = 1433347019, disp_pos = 0, disp_prop = 1473452605, stack_idx = 21845, level_stack = {{next_for_neutral_pos = 140737488334768, next_for_neutral_type = 0, last_strong_type = 7, prev_for_neutral_type = 0, level = 211 '\323', flags = 87 'W'}, {next_for_neutral_pos = 93824998232672, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 6, prev_for_neutral_type = 6, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 93824993924644, next_for_neutral_type = 5, last_strong_type = 7, prev_for_neutral_type = 0, level = 211 '\323', flags = 87 'W'}, {next_for_neutral_pos = 140737488334832, next_for_neutral_type = 2, last_strong_type = 5, prev_for_neutral_type = 0, level = 111 'o', flags = 85 'U'}, {next_for_neutral_pos = 79200, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 7, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 93824998232672, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 93824993924644, next_for_neutral_type = 0, last_strong_type = 2, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488334960, next_for_neutral_type = 1, last_strong_type = 3, prev_for_neutral_type = 0, level = 111 'o', flags = 85 'U'}, {next_for_neutral_pos = 140737488335088, next_for_neutral_type = 0, last_strong_type = 2, prev_for_neutral_type = 3, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 140737488336208, next_for_neutral_type = 0, last_strong_type = 6, prev_for_neutral_type = 3, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 93825001920248, next_for_neutral_type = 0, last_strong_type = 5, prev_for_neutral_type = 3, level = 229 '\345', flags = 87 'W'}, {next_for_neutral_pos = 87337062896, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 65424, next_for_neutral_type = 0, last_strong_type = 6, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 93824993924644, next_for_neutral_type = 0, last_strong_type = 2, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488335280, next_for_neutral_type = 0, last_strong_type = 2, prev_for_neutral_type = 6, level = 111 'o', flags = 85 'U'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 2, prev_for_neutral_type = 3, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 140737488336208, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 2, level = 34 '"', flags = 2 '\002'}, {next_for_neutral_pos = 93825001920248, next_for_neutral_type = 0, last_strong_type = 5, prev_for_neutral_type = 3, level = 229 '\345', flags = 87 'W'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 1 '\001'}, {next_for_neutral_pos = 93825006600960, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 35802240, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 33216, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 5, level = 1 '\001', flags = 0 '\000'}, {next_for_neutral_pos = 79200, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 5, level = 1 '\001', flags = 0 '\000'}, {next_for_neutral_pos = 93825035534240, next_for_neutral_type = 1, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488335184, next_for_neutral_type = 5, last_s--Type <RET> for more, q to quit, c to continue without paging--
trong_type = 5, prev_for_neutral_type = 2, level = 148 '\224', flags = 85 'U'}, {next_for_neutral_pos = 140737488335536, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 7, level = 231 '\347', flags = 87 'W'}, {next_for_neutral_pos = 93825035346400, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 6, level = 233 '\351', flags = 87 'W'}, {next_for_neutral_pos = 140737488335328, next_for_neutral_type = 2, last_strong_type = 7, prev_for_neutral_type = 4, level = 148 '\224', flags = 85 'U'}, {next_for_neutral_pos = 140737488335536, next_for_neutral_type = 0, last_strong_type = 6, prev_for_neutral_type = 6, level = 242 '\362', flags = 87 'W'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 7, level = 231 '\347', flags = 87 'W'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 93825035346400, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 93825033678435, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 93825028217232, next_for_neutral_type = 0, last_strong_type = 6, prev_for_neutral_type = 4, level = 122 'z', flags = 87 'W'}, {next_for_neutral_pos = 93824998232672, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 6, prev_for_neutral_type = 7, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 93824998232672, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 2, prev_for_neutral_type = 0, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 93824995578802, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488335456, next_for_neutral_type = 2, last_strong_type = 0, prev_for_neutral_type = 7, level = 136 '\210', flags = 85 'U'}, {next_for_neutral_pos = 93825033678435, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 6, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 93824998232672, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 1, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 93824996179882, next_for_neutral_type = 1, last_strong_type = 0, prev_for_neutral_type = 0, level = 133 '\205', flags = 85 'U'}, {next_for_neutral_pos = 93825035534240, next_for_neutral_type = 1, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488335520, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 3, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488335520, next_for_neutral_type = 0, last_strong_type = 5, prev_for_neutral_type = 5, level = 93 ']', flags = 85 'U'}, {next_for_neutral_pos = 224, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 1, level = 87 'W', flags = 87 'W'}, {next_for_neutral_pos = 140737488335568, next_for_neutral_type = 0, last_strong_type = 3, prev_for_neutral_type = 1, level = 93 ']', flags = 85 'U'}, {next_for_neutral_pos = 140737488335568, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 3, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 15264, next_for_neutral_type = 0, last_strong_type = 6, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488336464, next_for_neutral_type = 0, last_strong_type = 7, prev_for_neutral_type = 1, level = 95 '_', flags = 85 'U'}, {next_for_neutral_pos = 27353857, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 3, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 4294967552, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 3, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 4, last_strong_type = 2, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488335808, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488335760, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 93825047111837, next_for_neutral_type = 1, last_strong_type = 7, prev_for_neutral_type = 0, level = 5 '\005', flags = 0 '\000'}, {next_for_neutral_pos = 27353857, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 48, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 2, prev_for_neutral_type = 6, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 93824998232672, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 4, level = 161 '\241', flags = 1 '\001'}, {next_for_neutral_pos = 27353858, next_for_neutral_type = 1, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140733193388033, next_for_neutral_type = 2, last_strong_type = 0, prev_for_neutral_type = 7, level = 136 '\210', flags = 85 'U'}, {next_for_neutral_pos = 93825033678435, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 2, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 93824998232672, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 93824996179882, next_for_neutral_type = 1, last_strong_type = 0, prev_for_neutral_type = 0, level = 133 '\205', flags = 85 'U'}, {next_for_neutral_pos = 140737488335952, next_for_neutral_type = 0, last_strong_type = 5, prev_for_neutral_type = 6, level = 145 '\221', flags = 85 'U'}--Type <RET> for more, q to quit, c to continue without paging--
, {next_for_neutral_pos = 27353857, next_for_neutral_type = 1, last_strong_type = 0, prev_for_neutral_type = 4, level = 161 '\241', flags = 1 '\001'}, {next_for_neutral_pos = 46752, next_for_neutral_type = 3, last_strong_type = 2, prev_for_neutral_type = 1, level = 240 '\360', flags = 87 'W'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488335984, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 2, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488335984, next_for_neutral_type = 2, last_strong_type = 4, prev_for_neutral_type = 7, level = 145 '\221', flags = 85 'U'}, {next_for_neutral_pos = 27353857, next_for_neutral_type = 3, last_strong_type = 2, prev_for_neutral_type = 1, level = 240 '\360', flags = 87 'W'}, {next_for_neutral_pos = 93825035534240, next_for_neutral_type = 1, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 93824998232672, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488336088, next_for_neutral_type = 3, last_strong_type = 6, prev_for_neutral_type = 0, level = 40 '(', flags = 218 '\332'}, {next_for_neutral_pos = 93824996249801, next_for_neutral_type = 2, last_strong_type = 1, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488336144, next_for_neutral_type = 4, last_strong_type = 5, prev_for_neutral_type = 0, level = 146 '\222', flags = 85 'U'}, {next_for_neutral_pos = 140737488336184, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 5, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 93824996249801, next_for_neutral_type = 1, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 27355173, next_for_neutral_type = 2, last_strong_type = 0, prev_for_neutral_type = 4, level = 161 '\241', flags = 1 '\001'}, {next_for_neutral_pos = 140736853489715, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 4, level = 193 '\301', flags = 115 's'}, {next_for_neutral_pos = 93824996249801, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488336368, next_for_neutral_type = 6, last_strong_type = 6, prev_for_neutral_type = 4, level = 146 '\222', flags = 85 'U'}, {next_for_neutral_pos = 93459923358642, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 27353858, next_for_neutral_type = 5, last_strong_type = 4, prev_for_neutral_type = 0, level = 161 '\241', flags = 1 '\001'}, {next_for_neutral_pos = 27353857, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 6, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 93824998232672, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 2147483648010, next_for_neutral_type = 1, last_strong_type = 0, prev_for_neutral_type = 4, level = 161 '\241', flags = 1 '\001'}, {next_for_neutral_pos = 93824996179882, next_for_neutral_type = 1, last_strong_type = 0, prev_for_neutral_type = 0, level = 133 '\205', flags = 85 'U'}, {next_for_neutral_pos = 140737488336336, next_for_neutral_type = 0, last_strong_type = 5, prev_for_neutral_type = 6, level = 145 '\221', flags = 85 'U'}, {next_for_neutral_pos = 140737488336416, next_for_neutral_type = 5, last_strong_type = 7, prev_for_neutral_type = 6, level = 232 '\350', flags = 87 'W'}, {next_for_neutral_pos = 24192, next_for_neutral_type = 3, last_strong_type = 2, prev_for_neutral_type = 1, level = 240 '\360', flags = 87 'W'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 93824998256864, next_for_neutral_type = 0, last_strong_type = 6, prev_for_neutral_type = 2, level = 121 'y', flags = 88 'X'}, {next_for_neutral_pos = 140737488336368, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 4, level = 193 '\301', flags = 115 's'}, {next_for_neutral_pos = 93825044950192, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488336592, next_for_neutral_type = 5, last_strong_type = 2, prev_for_neutral_type = 3, level = 94 '^', flags = 85 'U'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 3, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 140733210146464, next_for_neutral_type = 5, last_strong_type = 4, prev_for_neutral_type = 5, level = 161 '\241', flags = 1 '\001'}, {next_for_neutral_pos = 109415430, next_for_neutral_type = 5, last_strong_type = 7, prev_for_neutral_type = 6, level = 232 '\350', flags = 87 'W'}, {next_for_neutral_pos = 109415830, next_for_neutral_type = 1, last_strong_type = 0, prev_for_neutral_type = 4, level = 161 '\241', flags = 1 '\001'}, {next_for_neutral_pos = 27355172, next_for_neutral_type = 7, last_strong_type = 7, prev_for_neutral_type = 7, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 27353858, next_for_neutral_type = 0, last_strong_type = 6, prev_for_neutral_type = 4, level = 167 '\247', flags = 85 'U'}, {next_for_neutral_pos = 27353858, next_for_neutral_type = 5, last_strong_type = 4, prev_for_neutral_type = 5, level = 161 '\241', flags = 1 '\001'}, {next_for_neutral_pos = 27353857, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 3, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 3, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 140737488336592, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 4, level = 193 '\301', flags = 115 's'}, {next_for_neutral_pos = 48, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 140737488336640, next_for_neutral_type = 5, last_strong_type = 6, prev_for_neutral_type = 1, level = 94 '^', flags = 85 'U'}, {next_for_neutral_pos = 140737488336640, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 3, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 16777216, ne--Type <RET> for more, q to quit, c to continue without paging--
xt_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 3, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 140737488336704, next_for_neutral_type = 4, last_strong_type = 3, prev_for_neutral_type = 2, level = 95 '_', flags = 85 'U'}, {next_for_neutral_pos = 347705, next_for_neutral_type = 4, last_strong_type = 7, prev_for_neutral_type = 4, level = 5 '\005', flags = 0 '\000'}, {next_for_neutral_pos = 139646566646552, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 3, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 348476, next_for_neutral_type = 0, last_strong_type = 4, prev_for_neutral_type = 3, level = 255 '\377', flags = 255 '\377'}, {next_for_neutral_pos = 140737488336832, next_for_neutral_type = 0, last_strong_type = 3, prev_for_neutral_type = 3, level = 94 '^', flags = 85 'U'}, {next_for_neutral_pos = 1474854845, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'}, {next_for_neutral_pos = 348476, next_for_neutral_type = 1, last_strong_type = 7, prev_for_neutral_type = 0, level = 5 '\005', flags = 0 '\000'}}, string = {lstring = 0x555557e528e8, s = 0x7fffffffb8e0 "\355(\345WUU", schars = 140737488336832, bufpos = 8890582302720, from_disp_str = true, unibyte = false}, w = 0x0, paragraph_dir = NEUTRAL_DIR, separator_limit = 0, first_elt = false, new_paragraph = false, frame_window_p = false}
newline_found_p = false
#7 0x0000555555611609 in redisplay_internal () at xdisp.c:17408
it = {window = 0x555557e528ed, w = 0x555557e528e8, f = 0x555555e916f8, method = GET_FROM_BUFFER, stop_charpos = 27353858, prev_stop = 347705, base_level_stop = 0, end_charpos = 27353858, medium_narrowing_begv = 27255690, medium_narrowing_zv = 27353858, large_narrowing_begv = 27103730, large_narrowing_zv = 27353858, s = 0x0, string_nchars = 0, multibyte_p = true, tab_line_p = false, header_line_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, from_disp_prop_p = false, ellipsis_p = false, avoid_cursor_p = false, dp = 0x0, dpvec = 0x0, dpend = 0x0, dpvec_char_len = 0, dpvec_face_id = 0, saved_face_id = 0, ctl_chars = {0x0 <repeats 16 times>}, start = {pos = {charpos = 347705, bytepos = 348476}, overlay_string_index = -1, string_pos = {charpos = -1, bytepos = -1}, dpvec_index = -1}, current = {pos = {charpos = 27353857, bytepos = 27355172}, overlay_string_index = -1, string_pos = {charpos = -1, bytepos = -1}, dpvec_index = -1}, n_overlay_strings = 0, overlay_strings_charpos = 27353857, overlay_strings = {0x0 <repeats 16 times>}, string_overlays = {0x0 <repeats 16 times>}, string = 0x0, from_overlay = 0x0, stack = {{string = 0x0, string_nchars = 0, end_charpos = 0, stop_charpos = 0, prev_stop = 0, base_level_stop = 0, cmp_it = {stop_pos = 0, id = 0, ch = 0, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, face_id = 0, u = {image = {object = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, image_id = 0}, stretch = {object = 0x0}, xwidget = {object = 0x0}}, position = {charpos = 0, bytepos = 0}, current = {pos = {charpos = 0, bytepos = 0}, overlay_string_index = 0, string_pos = {charpos = 0, bytepos = 0}, dpvec_index = 0}, from_overlay = 0x0, area = LEFT_MARGIN_AREA, method = GET_FROM_BUFFER, paragraph_embedding = NEUTRAL_DIR, multibyte_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, display_ellipsis_p = false, avoid_cursor_p = false, bidi_p = false, from_disp_prop_p = false, line_wrap = TRUNCATE, voffset = 0, space_width = 0x0, font_height = 0x0}, {string = 0x0, string_nchars = 0, end_charpos = 0, stop_charpos = 0, prev_stop = 0, base_level_stop = 0, cmp_it = {stop_pos = 0, id = 0, ch = 0, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, face_id = 0, u = {image = {object = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, image_id = 0}, stretch = {object = 0x0}, xwidget = {object = 0x0}}, position = {charpos = 0, bytepos = 0}, current = {pos = {charpos = 0, bytepos = 0}, overlay_string_index = 0, string_pos = {charpos = 0, bytepos = 0}, dpvec_index = 0}, from_overlay = 0x0, area = LEFT_MARGIN_AREA, method = GET_FROM_BUFFER, paragraph_embedding = NEUTRAL_DIR, multibyte_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, display_ellipsis_p = false, avoid_cursor_p = false, bidi_p = false, from_disp_prop_p = false, line_wrap = TRUNCATE, voffset = 0, space_width = 0x0, font_height = 0x0}, {string = 0x0, string_nchars = 0, end_charpos = 0, stop_charpos = 0, prev_stop = 0, base_level_stop = 0, cmp_it = {stop_pos = 0, id = 0, ch = 0, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, face_id = 0, u = {image = {object = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, image_id = 0}, stretch = {object = 0x0}, xwidget = {object = 0x0}}, position = {charpos = 0, bytepos = 0}, current = {pos = {charpos = 0, bytepos = 0}, overlay_string_index = 0, string_pos = {charpos = 0, bytepos = 0}, dpvec_index = 0}, from_overlay = 0x0, area = LEFT_MARGIN_AREA, method = GET_FROM_BUFFER, paragraph_embedding = NEUTRAL_DIR, multibyte_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, display_ellipsis_p = false, avoid_cursor_p = false, bidi_p = false, from_disp_prop_p = false, line_wrap = TRUNCATE, voffset = 0, space_width = 0x0, font_height = 0x0}, {string = 0x0, string_nchars = 0, end_charpos = 0, stop_charpos = 0, prev_stop = 0, base_level_stop = 0, cmp_it = {stop_pos = 0, id = 0, ch = 0, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, face_id = 0, u = {image = {object = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, image_id = 0}, stretch = {object = 0x0}, xwidget = {object = 0x0}}, position = {charpos = 0, bytepos = 0}, current = {pos = {charpos = 0, bytepos = 0}, overlay_string_index = 0, string_pos = {charpos = 0, bytepos = 0}, dpvec_index = 0}, from_overlay = 0x0, area = LEFT_MARGIN_AREA, method = GET_FROM_BUFFER, paragraph_embedding = NEUTRAL_DIR, multibyte_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, display_ellipsis_p = false, avoid_cursor_p = false, bidi_p = false, from_disp_prop_p = false, line_--Type <RET> for more, q to quit, c to continue without paging--
wrap = TRUNCATE, voffset = 0, space_width = 0x0, font_height = 0x0}, {string = 0x0, string_nchars = 0, end_charpos = 0, stop_charpos = 0, prev_stop = 0, base_level_stop = 0, cmp_it = {stop_pos = 0, id = 0, ch = 0, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, face_id = 0, u = {image = {object = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, image_id = 0}, stretch = {object = 0x0}, xwidget = {object = 0x0}}, position = {charpos = 0, bytepos = 0}, current = {pos = {charpos = 0, bytepos = 0}, overlay_string_index = 0, string_pos = {charpos = 0, bytepos = 0}, dpvec_index = 0}, from_overlay = 0x0, area = LEFT_MARGIN_AREA, method = GET_FROM_BUFFER, paragraph_embedding = NEUTRAL_DIR, multibyte_p = false, string_from_display_prop_p = false, string_from_prefix_prop_p = false, display_ellipsis_p = false, avoid_cursor_p = false, bidi_p = false, from_disp_prop_p = false, line_wrap = TRUNCATE, voffset = 0, space_width = 0x0, font_height = 0x0}}, sp = 0, selective = 0, what = IT_CHARACTER, face_id = 0, selective_display_ellipsis_p = true, ctl_arrow_p = true, face_box_p = false, start_of_box_run_p = false, end_of_box_run_p = false, overlay_strings_at_end_processed_p = false, ignore_overlay_strings_at_pos_p = false, glyph_not_available_p = false, starts_in_middle_of_char_p = false, face_before_selective_p = false, constrain_row_ascent_descent_p = false, line_number_produced_p = false, align_visually_p = false, line_wrap = TRUNCATE, base_face_id = 0, c = 10, len = 0, cmp_it = {stop_pos = 27353857, id = -1, ch = -2, rule_idx = 0, lookback = 0, nglyphs = 0, reversed_p = false, parent_it = 0x7fffffffb8e0, charpos = 0, nchars = 0, nbytes = 0, from = 0, to = 0, width = 0}, char_to_display = 0, glyphless_method = GLYPHLESS_DISPLAY_THIN_SPACE, image_id = 0, xwidget = 0x0, slice = {x = 0x0, y = 0x0, width = 0x0, height = 0x0}, space_width = 0x0, voffset = 0, tab_width = 8, font_height = 0x0, object = 0x555557e87fbd, position = {charpos = 27353857, bytepos = 27355172}, truncation_pixel_width = 9, continuation_pixel_width = 0, first_visible_x = 0, last_visible_x = 1881, last_visible_y = 2070, extra_line_spacing = 0, max_extra_line_spacing = 0, override_ascent = -1, override_descent = 0, override_boff = 0, glyph_row = 0x0, area = TEXT_AREA, nglyphs = 1, pixel_width = 0, ascent = 0, descent = 0, max_ascent = 0, max_descent = 0, phys_ascent = 0, phys_descent = 0, max_phys_ascent = 0, max_phys_descent = 0, current_x = 0, wrap_prefix_width = 0, continuation_lines_width = 0, eol_pos = {charpos = 0, bytepos = 0}, current_y = 396, first_vpos = 0, vpos = 22, hpos = 0, lnum = 0, lnum_bytepos = 0, lnum_width = 0, lnum_pixel_width = 0, pt_lnum = 0, stretch_adjust = 0, left_user_fringe_bitmap = 0, right_user_fringe_bitmap = 0, left_user_fringe_face_id = 0, right_user_fringe_face_id = 0, bidi_p = true, bidi_it = {bytepos = 27355172, charpos = 27353857, ch = 10, nchars = 1, ch_len = 1, type = NEUTRAL_B, type_after_wn = NEUTRAL_B, orig_type = NEUTRAL_B, resolved_level = 0 '\000', isolate_level = 0 '\000', invalid_levels = 0, invalid_isolates = 0, prev = {charpos = 0, type = UNKNOWN_BT, orig_type = UNKNOWN_BT}, last_strong = {charpos = 0, type = UNKNOWN_BT, orig_type = UNKNOWN_BT}, next_for_neutral = {charpos = -1, type = UNKNOWN_BT, orig_type = UNKNOWN_BT}, prev_for_neutral = {charpos = -1, type = UNKNOWN_BT, orig_type = UNKNOWN_BT}, next_for_ws = {charpos = 0, type = UNKNOWN_BT, orig_type = UNKNOWN_BT}, bracket_pairing_pos = -1, bracket_enclosed_type = UNKNOWN_BT, next_en_pos = 0, next_en_type = UNKNOWN_BT, sos = L2R, scan_dir = 0, disp_pos = -1, disp_prop = 0, stack_idx = 0, level_stack = {{next_for_neutral_pos = 0, next_for_neutral_type = 0, last_strong_type = 0, prev_for_neutral_type = 0, level = 0 '\000', flags = 0 '\000'} <repeats 128 times>}, string = {lstring = 0x0, s = 0x0, schars = 0, bufpos = 0, from_disp_str = false, unibyte = false}, w = 0x555557e528e8, paragraph_dir = NEUTRAL_DIR, separator_limit = -1, first_elt = false, new_paragraph = true, frame_window_p = true}, paragraph_embedding = NEUTRAL_DIR, min_width_property = 0x0, min_width_start = 0}
row = 0x30
w = 0x555557e528e8
sw = 0x555557e528e8
fr = 0x555555e916f8
must_finish = true
match_p = true
tlbufpos = {charpos = 347705, bytepos = 348476}
tlendpos = {charpos = 0, bytepos = 0}
number_of_visible_frames = 1
sf = 0x555555e916f8
polling_stopped_here = false
tail = 0x0
frame = 0x555555e916fd
hscroll_retries = 0
garbaged_frame_retries = 0
consider_all_windows_p = false
update_miniwindow_p = true
--Type <RET> for more, q to quit, c to continue without paging--
count = {bytes = 128}
previous_frame = 0x7fffffffb870
current_matrices_cleared = false
new_count = -17792
#8 0x000055555560f596 in redisplay () at xdisp.c:16672
#9 0x0000555555797f4e in read_char (commandflag=1, map=0x555557f08453, prev_event=0x0, used_mouse_menu=0x7fffffffd025, end_time=0x0) at keyboard.c:2672
echo_current = false
c = 0x0
local_getcjmp = {{__jmpbuf = {140737488342752, 93824994751784, 93824998232672, 0, 0, 140737488342640, 93824995519404, 1465334272}, __mask_was_saved = -12576, __saved_mask = {__val = {93824995565511, 93825035961443, 128, 93825035961459, 0, 140737488342752, 93824995270759, 93824998232672, 0, 0, 15774553331185344768, 93824994728855, 0, 140737488342944, 93824994753775, 0}}}}
save_jump = {{__jmpbuf = {110770, -4294967296, 93825015696904, 93825035961427, 140737488342848, 93824995251736, 93825035961443, 93825035961427}, __mask_was_saved = -12448, __saved_mask = {__val = {93825035961424, 93825035961443, 93825035961427, 93824998280960, 48288, 48288, 48288, 93824994728855, 48288, 140737488343008, 93824994735190, 139642271682304, 93825035961427, 93824998232672, 0, 0}}}}
tem = 0x555555854874 <XBUFFER+35>
save = 0x7fffffffcdc0
previous_echo_area_message = 0x0
also_record = 0x0
reread = false
recorded = false
polling_stopped_here = false
orig_kboard = 0x555555cf5e60
jmpcount = {bytes = 93824995370437}
c_volatile = 0x0
#10 0x00005555557acd81 in read_key_sequence
(keybuf=0x7fffffffd2a0, prompt=0x0, dont_downcase_last=false, can_return_switch_frame=true, fix_current_buffer=true, prevent_redisplay=false, disable_text_conversion_p=false) at keyboard.c:10757
interrupted_kboard = 0x555555cf5e60
interrupted_frame = 0x555555e916f8
key = 0x55555593027b <find_automatic_composition+814>
used_mouse_menu = false
echo_local_start = 0
last_real_key_start = 0
keys_local_start = 0
new_binding = 0x7fffffffd1d8
count = {bytes = 96}
t = 0
echo_start = 0
keys_start = 0
current_binding = 0x555557f08453
first_unbound = 31
mock_input = 0
used_mouse_menu_history = {false <repeats 30 times>}
fkey = {parent = 0x7ffff49f7873, map = 0x7ffff49f7873, start = 0, end = 0}
--Type <RET> for more, q to quit, c to continue without paging--
keytran = {parent = 0x7ffff2313473, map = 0x7ffff2313473, start = 0, end = 0}
indec = {parent = 0x7ffff49f7863, map = 0x7ffff49f7863, start = 0, end = 0}
shift_translated = false
delayed_switch_frame = 0x0
original_uppercase = 0x0
original_uppercase_position = -1
disabled_conversion = false
starting_buffer = 0x555557e87fb8
fake_prefixed_keys = 0x0
first_event = 0x0
second_event = 0x0
#11 0x0000555555793fb4 in command_loop_1 () at keyboard.c:1424
cmd = 0x1ce16c0
keybuf = {0x74a0, 0x0, 0x0, 0x7fffffffd2d0, 0x555555876bac <builtin_lisp_symbol+48>, 0x575739e0, 0x7fffffffd340, 0x555555881fc7 <unbind_to+217>, 0x7ffff49f04f3, 0x60, 0x55555587770e <KBOARD_OBJFWDP+24>, 0x0, 0xb, 0xb430, 0x555555b0d260 <lispsym>, 0x0, 0x0, 0x60, 0x7fffffffd340, 0x555555a7c260 <main_thread>, 0x7fffffffd380, 0x55555587c0d3 <push_handler_nosignal+242>, 0x100000030, 0x90, 0x30, 0x555555bdfdb0, 0x55555578fb19 <builtin_lisp_symbol+48>, 0x90, 0x7fffffffd3b0, 0x55555587bfc6 <push_handler+36>}
i = 1
last_pt = 27353857
prev_modiff = 25881
prev_buffer = 0x555557e87fb8
#12 0x000055555587bb35 in internal_condition_case (bfun=0x555555793b66 <command_loop_1>, handlers=0x90, hfun=0x555555792f64 <cmd_error>) at eval.c:1602
val = 0x55555578fb19 <builtin_lisp_symbol+48>
c = 0x555555bdfdb0
#13 0x0000555555793710 in command_loop_2 (handlers=0x90) at keyboard.c:1163
val = 0x90
#14 0x000055555587af29 in internal_catch (tag=0x12420, func=0x5555557936e2 <command_loop_2>, arg=0x90) at eval.c:1282
val = 0x61600000000
c = 0x555555bdfc70
#15 0x000055555579369e in command_loop () at keyboard.c:1141
#16 0x00005555557929e6 in recursive_edit_1 () at keyboard.c:749
count = {bytes = 32}
val = 0x5555558817e3 <record_unwind_protect+118>
#17 0x0000555555792c1a in Frecursive_edit () at keyboard.c:832
count = {bytes = 0}
buffer = 0x0
#18 0x000055555578e1d8 in main (argc=4, argv=0x7fffffffd778) at emacs.c:2562
stack_bottom_variable = 0x1
old_argc = 5
dump_file = 0x0
no_loadup = false
junk = 0x0
dname_arg = 0x0
ch_to_dir = 0x0
--Type <RET> for more, q to quit, c to continue without paging--
original_pwd = 0x0
dump_mode = 0x0
skip_args = 0
temacs = 0x0
attempt_load_pdump = true
only_version = false
rlim = {rlim_cur = 10022912, rlim_max = 18446744073709551615}
lc_all = 0x0
sockfd = -1
module_assertions = false
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#76538; Package
emacs.
(Mon, 03 Mar 2025 19:18:02 GMT)
Full text and
rfc822 format available.
Message #158 received at 76538 <at> debbugs.gnu.org (full text, mbox):
> Date: Mon, 03 Mar 2025 17:07:02 +0000
> From: Pip Cet <pipcet <at> protonmail.com>
> Cc: gerd.moellmann <at> gmail.com, eller.helmut <at> gmail.com, monnier <at> iro.umontreal.ca, yantar92 <at> posteo.net, joaomoreira <at> gmx.se, 76538 <at> debbugs.gnu.org
>
> "Eli Zaretskii" <eliz <at> gnu.org> writes:
>
> > As for the call with 'false' as 2nd arg: First, when it->bidi_p is
> > true, we don't use it->len for moving to the next display element.
>
> True. Moving the eassert inside the if might be the minimal change
> there.
No, I think setting it->len correctly would be a better, cleaner
change. Not that I understand how come you hit this assertion while
we never do on master...
This bug report was last modified 105 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.