GNU bug report logs - #30931
abort() due to CHECK_ALLOCATED_AND_LIVE failure during GC

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 30931 in the body.
You can then email your comments to 30931 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Michał Kondraciuk <k.michal <at> zoho.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 27.0.50; Crash in "Automatic GC"
Date: Sat, 24 Mar 2018 21:30:56 +0100

[Message part 1 (text/plain, inline)]

Newest Emacs (and 25.3) crashes for me. Unfortunately I can only 
reproduce this using newest versions of Ivy and yasnippet packages.

Backtrace (full backtrace in attachment):

#0  0x00007ffff1190269 in raise (sig=6) at 
../sysdeps/unix/sysv/linux/pt-raise.c:35
#1  0x0000000000568a3a in terminate_due_to_signal (sig=6, 
backtrace_limit=40) at ../../src/emacs.c:395
#2  0x000000000058fc5d in emacs_abort () at ../../src/sysdep.c:2426
#3  0x000000000061719a in signal_or_quit (error_symbol=XIL(0x7b60), 
data=XIL(0x232e613), keyboard_quit=false) at ../../src/eval.c:1595
#4  0x00000000006170f1 in Fsignal (error_symbol=XIL(0x7b60), 
data=XIL(0x232e613)) at ../../src/eval.c:1565
#5  0x00000000005663e3 in xsignal (error_symbol=XIL(0x7b60), 
data=XIL(0x232e613)) at ../../src/lisp.h:3944
#6  0x00000000006175d9 in xsignal1 (error_symbol=XIL(0x7b60), 
arg=XIL(0x2233484)) at ../../src/eval.c:1702
#7  0x00000000005cddda in compile_pattern_1 (cp=0xcc4440 
<searchbufs+6688>, pattern=XIL(0x1904bf4), translate=XIL(0), 
posix=false) at ../../src/search.c:165
#8  0x00000000005ce09a in compile_pattern (pattern=XIL(0x1904bf4), 
regp=0x0, translate=XIL(0), posix=false, multibyte=true) at 
../../src/search.c:248
#9  0x00000000005ceb94 in fast_string_match_internal 
(regexp=XIL(0x1904bf4), string=XIL(0x1c99f34), table=XIL(0)) at 
../../src/search.c:481
#10 0x0000000000566444 in fast_string_match (regexp=XIL(0x1904bf4), 
string=XIL(0x1c99f34)) at ../../src/lisp.h:4150
#11 0x00000000005b2ca9 in Ffind_file_name_handler 
(filename=XIL(0x1c99f34), operation=XIL(0x5580)) at ../../src/fileio.c:306
#12 0x00000000005b39f6 in Fexpand_file_name (name=XIL(0x1c99f34), 
default_directory=XIL(0)) at ../../src/fileio.c:783
#13 0x00000000005bf41f in Fdo_auto_save (no_message=XIL(0xc0f0), 
current_only=XIL(0)) at ../../src/fileio.c:5574
#14 0x000000000056b219 in shut_down_emacs (sig=6, stuff=XIL(0)) at 
../../src/emacs.c:2111
#15 0x00000000005689e1 in terminate_due_to_signal (sig=6, 
backtrace_limit=40) at ../../src/emacs.c:378
#16 0x000000000058fc5d in emacs_abort () at ../../src/sysdep.c:2426
#17 0x00000000005f4b7b in mark_object (arg=XIL(0x23b5e81)) at 
../../src/alloc.c:6707
#18 0x00000000005f4c4e in mark_object (arg=XIL(0x232e893)) at 
../../src/alloc.c:6725
#19 0x00000000005f4c4e in mark_object (arg=XIL(0x232e813)) at 
../../src/alloc.c:6725
#20 0x00000000005f19d5 in mark_maybe_object (obj=XIL(0x232e813)) at 
../../src/alloc.c:4844
#21 0x00000000005f1bb3 in mark_memory (start=0x7fffffff2d90, 
end=0x7fffffffdc58) at ../../src/alloc.c:4988
#22 0x00000000005f1be8 in mark_stack (bottom=0x7fffffffdc58 "", 
end=0x7fffffff2d90 "\240-\377\377\377\177") at ../../src/alloc.c:5195
#23 0x00000000006a0eff in mark_one_thread (thread=0xcfbac0 
<main_thread>) at ../../src/thread.c:616
#24 0x00000000006a1000 in mark_threads_callback (ignore=0x0) at 
../../src/thread.c:649
#25 0x00000000005f1c3e in flush_stack_call_func (func=0x6a0fba 
<mark_threads_callback>, arg=0x0) at ../../src/alloc.c:5222
#26 0x00000000006a102c in mark_threads () at ../../src/thread.c:656
#27 0x00000000005f3670 in garbage_collect_1 (end=0x7fffffff2f90) at 
../../src/alloc.c:6000
#28 0x00000000005f3d73 in Fgarbage_collect () at ../../src/alloc.c:6171
#29 0x000000000056655d in maybe_gc () at ../../src/lisp.h:4838
#30 0x000000000061a3ee in Ffuncall (nargs=3, args=0x7fffffff3048) at 
../../src/eval.c:2803
#31 0x0000000000666193 in exec_byte_code (bytestr=XIL(0x21552e4), 
vector=XIL(0x2157365), maxdepth=make_number(9), args_template=XIL(0), 
nargs=0, args=0x0) at ../../src/bytecode.c:632
#32 0x000000000061b2f1 in funcall_lambda (fun=XIL(0x21573d5), nargs=2, 
arg_vector=0x2157365) at ../../src/eval.c:3104
#33 0x000000000061a4f3 in Ffuncall (nargs=3, args=0x7fffffff3500) at 
../../src/eval.c:2823
#34 0x0000000000666193 in exec_byte_code (bytestr=XIL(0x2157a54), 
vector=XIL(0x2157775), maxdepth=make_number(7), args_template=XIL(0), 
nargs=0, args=0x0) at ../../src/bytecode.c:632
#35 0x000000000061b2f1 in funcall_lambda (fun=XIL(0x215ac45), nargs=2, 
arg_vector=0x2157775) at ../../src/eval.c:3104
#36 0x000000000061a4f3 in Ffuncall (nargs=3, args=0x7fffffff39c8) at 
../../src/eval.c:2823
#37 0x0000000000666193 in exec_byte_code (bytestr=XIL(0x2157ab4), 
vector=XIL(0x215ac75), maxdepth=make_number(7), args_template=XIL(0), 
nargs=0, args=0x0) at ../../src/bytecode.c:632
#38 0x000000000061b2f1 in funcall_lambda (fun=XIL(0x215ad55), nargs=2, 
arg_vector=0x215ac75) at ../../src/eval.c:3104
#39 0x000000000061a4f3 in Ffuncall (nargs=3, args=0x7fffffff3f38) at 
../../src/eval.c:2823
#40 0x0000000000666193 in exec_byte_code (bytestr=XIL(0x21651b4), 
vector=XIL(0x2165fb5), maxdepth=make_number(3), args_template=XIL(0), 
nargs=0, args=0x0) at ../../src/bytecode.c:632
#41 0x000000000061b2f1 in funcall_lambda (fun=XIL(0x2165fd5), nargs=0, 
arg_vector=0x2165fb5) at ../../src/eval.c:3104
#42 0x000000000061a4f3 in Ffuncall (nargs=1, args=0x7fffffff43b0) at 
../../src/eval.c:2823
#43 0x0000000000618c3f in eval_sub (form=XIL(0x232e6c3)) at 
../../src/eval.c:2257
#44 0x0000000000614e4f in Fprogn (body=XIL(0)) at ../../src/eval.c:455
#45 0x00000000006160c2 in Flet (args=XIL(0x232e6a3)) at ../../src/eval.c:969
#46 0x0000000000618a08 in eval_sub (form=XIL(0x232e693)) at 
../../src/eval.c:2238
#47 0x0000000000618443 in Feval (form=XIL(0x232e693), lexical=XIL(0)) at 
../../src/eval.c:2106
#48 0x000000000061a8f4 in funcall_subr (subr=0xc63a20 <Seval>, 
numargs=1, args=0x7fffffff4860) at ../../src/eval.c:2898
#49 0x000000000061a4af in Ffuncall (nargs=2, args=0x7fffffff4858) at 
../../src/eval.c:2821
#50 0x0000000000666193 in exec_byte_code (bytestr=XIL(0x2165114), 
vector=XIL(0x2166005), maxdepth=make_number(8), args_template=XIL(0), 
nargs=0, args=0x0) at ../../src/bytecode.c:632

Lisp backtrace:

"Automatic GC" (0x0)
"mapcar" (0xffff3050)
"yas-active-snippets" (0xffff3508)
"yas--place-overlays" (0xffff39d0)
"yas--move-to-field" (0xffff3f40)
0x2165fd0 PVEC_COMPILED
"funcall" (0xffff43b0)
"let" (0xffff4600)
"eval" (0xffff4860)
"yas-expand-snippet" (0xffff4f60)
"yas--expand-or-prompt-for-template" (0xffff5440)
"yas-expand" (0xffff58f0)
"save-restriction" (0xffff5b40)
"save-current-buffer" (0xffff5cc0)
"update-fn" (0xffff5f10)
"ivy--insert-minibuffer" (0xffff63d0)
"ivy--exhibit" (0xffff6980)
"ivy--queue-exhibit" (0xffff6e30)
"execute-kbd-macro" (0xffff7230)
"while" (0xffff7350)
"let" (0xffff7570)
0x1251040 Lisp type 3
"apply" (0xffff7958)
"timer-event-handler" (0xffff7ea8)
"input-pending-p" (0xffff81e8)
"sit-for" (0xffff86c0)
"yas-expand-snippet" (0xffff8dc0)
"yas--expand-or-prompt-for-template" (0xffff92a0)
"yas-expand" (0xffff9750)
"save-restriction" (0xffff99a0)
"save-current-buffer" (0xffff9b20)
"update-fn" (0xffff9d70)
"ivy--insert-minibuffer" (0xffffa230)
"ivy--exhibit" (0xffffa7e0)
"ivy--minibuffer-setup" (0xffffacd0)
0x21d5720 PVEC_COMPILED
"read-from-minibuffer" (0xffffb598)
"ivy-read" (0xffffbd10)
"catch" (0xffffbfc0)
"while" (0xffffc170)
"let" (0xffffc390)
"let" (0xffffc5b0)
"progn" (0xffffc700)
"eval" (0xffffc990)
"eval-expression" (0xffffcfb0)
"funcall-interactively" (0xffffcfa8)
"call-interactively" (0xffffd3e0)
"command-execute" (0xffffd928)

Emacs first crashed for me on 25.3, when I was calling 
`yas-expand-snippet` inside `save-restriction` form inside `ivy-read`. I 
wrote this form to try to reproduce this:

(progn
  (package-initialize)
  (require 'yasnippet)
  (require 'ivy)

  (yas-reload-all)

  (switch-to-buffer "*scratch*")
  (lisp-interaction-mode)
  (yas-minor-mode-on)

  (let ((templates (mapcar #'yas--template-key
                           (yas--all-templates 
(yas--get-snippet-tables)))))
    (defun update-fn ()
      (with-current-buffer "*scratch*"
        (save-restriction
          (narrow-to-region 3 3)
          (insert " " (ivy-state-current ivy-last))
          (yas-expand)
          (mapc #'yas-abort-snippet (yas-active-snippets)))
        (redisplay)))
    (let ((buffer-undo-list t))
      (while t
        (erase-buffer)
        (insert "aaaaaa")
        (redisplay)
        (sit-for 0.1)
        (run-with-timer
         0.01 nil (lambda ()
                    (dotimes (_ (% (random) 15))
                      (execute-kbd-macro "\C-n")
                      (sit-for 0.05)
                      (redisplay)
                      (when (zerop (% (random) 4))
                        (execute-kbd-macro "\C-p")
                        (sit-for 0.05)
                        (redisplay)))
                    (sit-for 0.1)
                    (redisplay)
                    (abort-recursive-edit)))
        (catch 'exit
          (condition-case err
              (ivy-read "foo: " templates
                        :update-fn #'update-fn)
            (quit)))))))


Steps to reproduce:

1. Install newest ivy, yasnippet and yasnippet-snippets

2. Run emacs -Q

3. Eval form above

4. Typically emacs crashes after about 1-2 minutes


In GNU Emacs 27.0.50 (build 13, x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
 of 2018-03-24 built on mkc
Repository revision: 4cfd2a2a92e2890af76bcd8375f966ce67ef2482
Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
System Description: Linux Mint 18.2

Configured using:
 'configure 'CFLAGS=-O0 -g2''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 THREADS LIBSYSTEMD LCMS2

Important settings:
  value of $LC_CTYPE: en_US.UTF-8
  value of $LC_MONETARY: pl_PL.UTF-8
  value of $LC_NUMERIC: pl_PL.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

[gdb.txt (text/plain, attachment)]

Message #8 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michał Kondraciuk <k.michal <at> zoho.com>
Cc: 30931 <at> debbugs.gnu.org
Subject: Re: bug#30931: 27.0.50; Crash in "Automatic GC"
Date: Sun, 25 Mar 2018 05:33:19 +0300

> From: Michał Kondraciuk <k.michal <at> zoho.com>
> Date: Sat, 24 Mar 2018 21:30:56 +0100
> 
> Newest Emacs (and 25.3) crashes for me. Unfortunately I can only 
> reproduce this using newest versions of Ivy and yasnippet packages.

Please post a reproduction recipe starting with "emacs -Q" and loading
the necessary packages.

Thanks.

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Michał Kondraciuk <k.michal <at> zoho.com>
To: bug-gnu-emacs <at> gnu.org
Cc: Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#30931: 27.0.50; Crash in "Automatic GC"
Date: Sun, 25 Mar 2018 11:46:16 +0200

[Message part 1 (text/plain, inline)]

On 03/25/2018 04:33 AM, Eli Zaretskii wrote:
>> From: Michał Kondraciuk<k.michal <at> zoho.com>
>> Date: Sat, 24 Mar 2018 21:30:56 +0100
>>
>> Newest Emacs (and 25.3) crashes for me. Unfortunately I can only
>> reproduce this using newest versions of Ivy and yasnippet packages.
> Please post a reproduction recipe starting with "emacs -Q" and loading
> the necessary packages.
>
> Thanks.

1. emacs -Q
2. M-x package-initialize
3. Evaluate: |(add-to-list 'package-archives '("melpa" . 
"https://melpa.org/packages/")) 4. M-x package-refresh-contents 5. M-x 
package-install RET ivy RET |6||. M-x package-install RET yasnippet RET 
|||7. M-x package-install RET yasnippet-snippets RET|
8. Evaluate form below.
9. Wait 1-10 minutes.

(progn
  (package-initialize)
  (require 'yasnippet)
  (require 'ivy)

  (yas-reload-all)

  (switch-to-buffer "*scratch*")
  (lisp-interaction-mode)
  (yas-minor-mode-on)

  (let ((templates (mapcar #'yas--template-key
                           (yas--all-templates 
(yas--get-snippet-tables)))))
    (defun update-fn ()
      (with-current-buffer "*scratch*"
        (save-restriction
          (narrow-to-region 3 3)
          (insert " " (ivy-state-current ivy-last))
          (yas-expand)
          (mapc #'yas-abort-snippet (yas-active-snippets)))
        (redisplay)))
    (let ((buffer-undo-list t))
      (while t
        (erase-buffer)
        (insert "aaaaaa")
        (redisplay)
        (sit-for 0.1)
        (run-with-timer
         0.01 nil (lambda ()
                    (dotimes (_ (% (random) 15))
                      (execute-kbd-macro "\C-n")
                      (sit-for 0.05)
                      (redisplay)
                      (when (zerop (% (random) 4))
                        (execute-kbd-macro "\C-p")
                        (sit-for 0.05)
                        (redisplay)))
                    (sit-for 0.1)
                    (redisplay)
                    (abort-recursive-edit)))
        (catch 'exit
          (condition-case err
              (ivy-read "foo: " templates
                        :update-fn #'update-fn)
            (quit)))))))
||||

[Message part 2 (text/html, inline)]

Message #14 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Michał Kondraciuk <k.michal <at> zoho.com>
Cc: 30931 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#30931: 27.0.50; Crash in "Automatic GC"
Date: Mon, 26 Mar 2018 08:45:55 -0400

[Message part 1 (text/plain, inline)]

found 30931 25.3
found 30931 26.0.91
tags 30931 + confirmed
retitle 30931 abort() due to CHECK_ALLOCATED_AND_LIVE failure during GC
quit

Michał Kondraciuk <k.michal <at> zoho.com> writes:
>
> 1. emacs -Q
> 2. M-x package-initialize
> 3. Evaluate: |(add-to-list 'package-archives '("melpa"
> . "https://melpa.org/packages/")) 4. M-x package-refresh-contents
> 5. M-x 
> package-install RET ivy RET |6||. M-x package-install RET yasnippet
> RET |||7. M-x package-install RET yasnippet-snippets RET|
> 8. Evaluate form below.
> 9. Wait 1-10 minutes.

I'm able to reproduce this in a fresh HOME (I've tried with versions
25.3 and 26.0.91), with the attached init.el.

[init.el (text/plain, attachment)]

[Message part 3 (text/plain, inline)]

It looks like there is a Lisp_Misc_Free object in the undo-list. At
least, I think it's the undo-list.  One time I actually got the
mark_object (BVAR (nextb, undo_list)) in the backtrace, although the
next time it was just mark_stack, but the object being marked still
looks like an undo-list.

Message #25 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Michał Kondraciuk <k.michal <at> zoho.com>
Cc: 30931 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#30931: 27.0.50; Crash in "Automatic GC"
Date: Mon, 26 Mar 2018 10:20:47 -0400

[Message part 1 (text/plain, inline)]

Noam Postavsky <npostavs <at> gmail.com> writes:

> next time it was just mark_stack, but the object being marked still
> looks like an undo-list.

Here's the gdb evidence of that.  I note that the reproduction recipe
disables undo via (let ((buffer-undo-list t))...), but looking at its
code, I see that yasnippet unconditionally pushes entries onto the undo
list anyway.  Michał, is the binding of buffer-undo-list significant for
your reproduction (i.e., did the original crash happen in an
undo-disabled buffer)?

[gdb.log (text/plain, attachment)]

[xxlist.gdb (text/plain, attachment)]

Message #28 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Michał Kondraciuk <k.michal <at> zoho.com>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 30931 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#30931: 27.0.50; Crash in "Automatic GC"
Date: Mon, 26 Mar 2018 16:57:53 +0200

[Message part 1 (text/plain, inline)]

On 03/26/2018 04:20 PM, Noam Postavsky wrote:
> Michał, is the binding of buffer-undo-list significant for
> your reproduction (i.e., did the original crash happen in an
> undo-disabled buffer)?
Yes, originally it crashed with buffer-undo-list bound to t.


Today I was able to reproduce this crash without using Ivy. It crashes 
much faster (typically about 15 seconds):

(progn
  (package-initialize)
  (require 'yasnippet)

  (defun crash (key)
    (save-restriction
      (narrow-to-region 1 1)
      (insert " " key)
      (yas-expand)))

  (with-current-buffer "*scratch*"
    (yas-minor-mode-on)
    (let ((buffer-undo-list t))
      (while t
        (erase-buffer)
        (dolist (key '("def" "ah"))
          (run-with-idle-timer 0.0 nil #'crash key)
          (crash key))))))


Also, If I remove calls to save-restriction and narrow-to-region, Emacs 
doesn't crash. Or at least, it was running fine for 30 minutes until I 
killed Emacs.

[Message part 2 (text/html, inline)]

Message #31 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Michał Kondraciuk <k.michal <at> zoho.com>
To: 30931 <30931 <at> debbugs.gnu.org>
Subject: Re: bug#30931: 27.0.50; Crash in "Automatic GC"
Date: Wed, 28 Mar 2018 23:02:40 +0200

I don't know if it helps, but I was able to reproduce a crash in the 
same place (mark_object()) with bare Emacs, however only in 27.0.50.

I have no idea what this code does at all. The way I wrote this is I 
kept rerunning the previous recipe but each time I commented out some
part of yasnippet.el library. If Emacs crashed, I kept the change, 
otherwise I uncommented those parts and focused on another region, and 
so on until I was left with very simple functions. I hope someone can 
confirm that it crashes.

(progn
  (require 'cl-lib)
  (defvar overlays nil)

  (defun on-overlay-modification (&rest args)
    (push nil buffer-undo-list))

  (defun crash ()
    (save-restriction
      (narrow-to-region 1 1)

      (let (start end field)
        (goto-char 2)
        (insert " 123aaaa")
        (narrow-to-region 2 5)

        (setq start (set-marker (make-marker) 2))
        (setq end (set-marker (make-marker) 3))
        (indent-according-to-mode)

        (sit-for 0)
        (cond (overlays
               (move-overlay (nth 1 overlays) end (1+ end)))
              (t
               (setq overlays
                     (list (make-overlay (1- start) start nil t nil)
                           (make-overlay end (1+ end) nil t nil)))
               (dolist (ov (list (make-overlay (1- start) start nil t nil)
                           (make-overlay end (1+ end) nil t nil)))
                 (overlay-put ov 'modification-hooks 
'(on-overlay-modification))))))))

  (with-current-buffer "*scratch*"
    (let ((buffer-undo-list t))
      (while t
        (erase-buffer)
        (crash)))))

Message #34 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Michał Kondraciuk <k.michal <at> zoho.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 30931 <30931 <at> debbugs.gnu.org>
Subject: Re: bug#30931: 27.0.50; Crash in "Automatic GC"
Date: Thu, 29 Mar 2018 17:52:49 +0200

I was able to reproduce crash in the same place in bare Emacs versions 
27.0.50, 25.3, 24.4 with the form below.

(progn
  (defun on-overlay-modification (&rest args)
    (push nil buffer-undo-list))

  (defun crash ()
    (save-restriction
      (narrow-to-region 1 1)

      (let (end)
        (insert " 123aaa")
        (narrow-to-region 2 3)

        (setq end (set-marker (make-marker) 3))
        (save-restriction
          (widen)
          (lisp-indent-line))

        (garbage-collect)

        (overlay-put (make-overlay end (1+ end))
                     'modification-hooks '(on-overlay-modification)))))

  (with-current-buffer "*scratch*"
    (let ((buffer-undo-list t))
      (while t
        (erase-buffer)
        (crash)))))

Message #37 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Michał Kondraciuk <k.michal <at> zoho.com>
Cc: 30931 <30931 <at> debbugs.gnu.org>
Subject: Re: bug#30931: 27.0.50; Crash in "Automatic GC"
Date: Thu, 29 Mar 2018 19:47:13 -0400

Michał Kondraciuk <k.michal <at> zoho.com> writes:

> I don't know if it helps, but I was able to reproduce a crash in the
> same place (mark_object()) with bare Emacs, however only in 27.0.50.

This is very helpful, thank you.

> I have no idea what this code does at all. The way I wrote this is I
> kept rerunning the previous recipe but each time I commented out some
> part of yasnippet.el library. If Emacs crashed, I kept the change,
> otherwise I uncommented those parts and focused on another region, and
> so on until I was left with very simple functions.

I was going to do the same thing, just no time to get to it this week.

> I hope someone can confirm that it crashes.

I can confirm it crashes, and much faster too, just two calls of the
crash function are enough:

(progn
  (defun on-overlay-modification (&rest args)
    (push nil buffer-undo-list))

  (defun crash ()
    (save-restriction
      (narrow-to-region 1 1)

      (let (end)
        (insert " 123aaa")
        (narrow-to-region 2 3)

        (setq end (set-marker (make-marker) 3))
        (save-restriction
          (widen)
          (lisp-indent-line))

        (garbage-collect)

        (overlay-put (make-overlay end (1+ end))
                     'modification-hooks '(on-overlay-modification)))))

  (with-current-buffer "*scratch*"
    (erase-buffer)
    (let ((buffer-undo-list t))
      (crash)
      (erase-buffer)
      (crash))))

Message #40 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Michał Kondraciuk <k.michal <at> zoho.com>
Cc: 30931 <30931 <at> debbugs.gnu.org>
Subject: Re: bug#30931: 27.0.50; Crash in "Automatic GC"
Date: Fri, 30 Mar 2018 01:39:30 -0400

Noam Postavsky <npostavs <at> gmail.com> writes:

> Michał Kondraciuk <k.michal <at> zoho.com> writes:
>
>> I don't know if it helps, but I was able to reproduce a crash in the
>> same place (mark_object()) with bare Emacs, however only in 27.0.50.
>
> This is very helpful, thank you.

Okay, with this recipe, I was able to put a breakpoint in free_marker
and find where the problem is: save_restriction_save creates markers to
represent the current restriction, record_marker_adjustments (called
from delete-region) puts all buffer markers onto the buffer undo list,
and save_restriction_restore frees the markers created in
save_restriction_save.

    save_restriction_save (void)
    {
    [...]
          beg = build_marker (current_buffer, BEGV, BEGV_BYTE);
          end = build_marker (current_buffer, ZV, ZV_BYTE);

    record_marker_adjustments (ptrdiff_t from, ptrdiff_t to)
    {
    [...]
      for (m = BUF_MARKERS (current_buffer); m; m = m->next)
        {
        [...]
                  XSETMISC (marker, m);
                  bset_undo_list
                    (current_buffer,
                     Fcons (Fcons (marker, make_number (adjustment)),
                            BVAR (current_buffer, undo_list)));

    save_restriction_restore (Lisp_Object data)
    {
    [...]
          /* These aren't needed anymore, so don't wait for GC.  */
          free_marker (XCAR (data));
          free_marker (XCDR (data));

So the following is enough to trigger the bug:

  (with-temp-buffer
    (insert "1234567890")
    (setq buffer-undo-list nil)
    (narrow-to-region 2 5)
    (save-restriction
      (widen)
      (delete-region 1 6))
    (princ (format "%S" buffer-undo-list) #'external-debugging-output)
    (type-of (car (nth 1 buffer-undo-list))))

Surprisingly, printing works on a Lisp_Misc_Free object, so this prints

    (("12345" . 1) (#<misc free cell> . -1) (#<misc free cell> . 1))

before aborting in Ftype_of.  Calling garbage-collect instead of type-of
also aborts.  Calling neither seems to be okay, since in that case the
Lisp_Misc_Free objects somehow turn into #<marker in no buffer>.

For solving this, I guess the easiest would be to have
save_restriction_restore do (set-marker m nil) instead of free_marker.
Might make more sense to avoid putting those markers onto the undo list
in the first place though.

Message #43 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michał Kondraciuk <k.michal <at> zoho.com>
Cc: 30931 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Noam Postavsky <npostavs <at> gmail.com>
Subject: Re: abort() due to CHECK_ALLOCATED_AND_LIVE failure during GC
Date: Thu, 29 Mar 2018 23:19:56 -0700

[Message part 1 (text/plain, inline)]

Thanks for reporting that. I installed into master the attached patch, which 
fixed the crash for me, for the test case in Bug#30931#37.

This patch puts more pressure on the GC. Although it might be nice to relieve 
the pressure, I'm not sure it's worth the trouble.

[0001-Fix-CHECK_ALLOCATED_AND_LIVE-abort-during-GC.txt (text/plain, attachment)]

Message #46 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 30931 <at> debbugs.gnu.org, k.michal <at> zoho.com
Subject: Re: bug#30931: 27.0.50; Crash in "Automatic GC"
Date: Fri, 30 Mar 2018 11:16:36 +0300

> From: Noam Postavsky <npostavs <at> gmail.com>
> Date: Fri, 30 Mar 2018 01:39:30 -0400
> Cc: 30931 <30931 <at> debbugs.gnu.org>
> 
> So the following is enough to trigger the bug:
> 
>   (with-temp-buffer
>     (insert "1234567890")
>     (setq buffer-undo-list nil)
>     (narrow-to-region 2 5)
>     (save-restriction
>       (widen)
>       (delete-region 1 6))
>     (princ (format "%S" buffer-undo-list) #'external-debugging-output)
>     (type-of (car (nth 1 buffer-undo-list))))

Since Paul fixed the bug now (thanks, Paul!), would you like to add
this test to the test suite?

Thanks.

Message #49 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 30931 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Michał Kondraciuk <k.michal <at> zoho.com>
Subject: Re: bug#30931: abort() due to CHECK_ALLOCATED_AND_LIVE failure during
 GC
Date: Fri, 30 Mar 2018 08:56:04 -0400

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> This patch puts more pressure on the GC. Although it might be nice to
> relieve the pressure, I'm not sure it's worth the trouble.

> -      /* These aren't needed anymore, so don't wait for GC.  */
> -      free_marker (XCAR (data));
> -      free_marker (XCDR (data));
> +      /* This isn’t needed anymore, so don’t wait for GC.
> +         Do not call free_marker on XCAR (data) or XCDR (data),
> +         though, since record_marker_adjustments may have put
> +         them on the buffer’s undo list (Bug#30931).  */
>        free_cons (XCONS (data));

Shouldn't we at least do set-marker?

    Fset_marker (XCAR (data), Qnil, Qnil)
    Fset_marker (XCDR (data), Qnil, Qnil)

Message #52 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 30931 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Michał Kondraciuk <k.michal <at> zoho.com>
Subject: Re: bug#30931: abort() due to CHECK_ALLOCATED_AND_LIVE failure during
 GC
Date: Fri, 30 Mar 2018 09:23:03 -0700

On 03/30/2018 05:56 AM, Noam Postavsky wrote:
> Shouldn't we at least do set-marker?
>
>      Fset_marker (XCAR (data), Qnil, Qnil)
>      Fset_marker (XCDR (data), Qnil, Qnil)

Possibly. And perhaps we should making similar changes to the other two 
calls to free_marker (in insdel.c's signal_before_change), and removing 
the definition of free_marker while we're at it. Unfortunately I don't 
understand that part of the code well enough to say for sure.

Message #55 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 30931 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
 Michał Kondraciuk <k.michal <at> zoho.com>
Subject: Re: bug#30931: abort() due to CHECK_ALLOCATED_AND_LIVE failure during
 GC
Date: Fri, 30 Mar 2018 16:50:53 -0400

[Message part 1 (text/plain, inline)]

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> On 03/30/2018 05:56 AM, Noam Postavsky wrote:
>> Shouldn't we at least do set-marker?
>>
>>      Fset_marker (XCAR (data), Qnil, Qnil)
>>      Fset_marker (XCDR (data), Qnil, Qnil)
>
> Possibly. And perhaps we should making similar changes to the other
> two calls to free_marker (in insdel.c's signal_before_change),

Oh, good catch.  Here's a test-case for that one:

  (with-temp-buffer
    (insert "1234567890")
    (setq buffer-undo-list nil)
    (let ((before-change-functions
           (list (lambda (beg end)
                   (delete-region (1- beg) (1+ end))))))
      (delete-region 2 5))
    (princ (format "%S" buffer-undo-list) #'external-debugging-output)
    (type-of (car (nth 3 buffer-undo-list))))

This prints

    (("678" . 1) ("12345" . 1) (#<marker in no buffer> . -1) (#<misc free cell> . -1) (#<misc free cell> . -4))

and then aborts.

> and removing the definition of free_marker while we're at it.

I think that is sensible, see following patch.

[0001-Fix-another-case-of-freed-markers-in-the-undo-list-B.patch (text/x-diff, inline)]

From cf327524c9d66969051f429da6f004003850ffef Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs <at> gmail.com>
Date: Fri, 30 Mar 2018 16:44:24 -0400
Subject: [PATCH] Fix another case of freed markers in the undo-list
 (Bug#30931)

* src/alloc.c (free_marker): Remove.
* src/editfns.c (save_restriction_restore):
* src/insdel.c (signal_before_change): Detach the markers from the
buffer when we're done with them instead of calling free_marker on
them.
* test/src/editfns-tests.el (delete-region-undo-markers-1)
(delete-region-undo-markers-2): New tests.
---
 src/alloc.c               |  9 ---------
 src/editfns.c             | 10 ++++++----
 src/insdel.c              |  7 +++++--
 src/lisp.h                |  1 -
 test/src/editfns-tests.el | 51 +++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 62 insertions(+), 16 deletions(-)

diff --git a/src/alloc.c b/src/alloc.c
index 369592d70e..9fdcc7306a 100644
--- a/src/alloc.c
+++ b/src/alloc.c
@@ -3878,15 +3878,6 @@ build_marker (struct buffer *buf, ptrdiff_t charpos, ptrdiff_t bytepos)
   return obj;
 }
 
-/* Put MARKER back on the free list after using it temporarily.  */
-
-void
-free_marker (Lisp_Object marker)
-{
-  unchain_marker (XMARKER (marker));
-  free_misc (marker);
-}
-
 
 /* Return a newly created vector or string with specified arguments as
    elements.  If all the arguments are characters that can fit
diff --git a/src/editfns.c b/src/editfns.c
index 727f2d0080..84de679273 100644
--- a/src/editfns.c
+++ b/src/editfns.c
@@ -3899,10 +3899,12 @@ save_restriction_restore (Lisp_Object data)
 
 	  buf->clip_changed = 1; /* Remember that the narrowing changed. */
 	}
-      /* This isn’t needed anymore, so don’t wait for GC.
-         Do not call free_marker on XCAR (data) or XCDR (data),
-         though, since record_marker_adjustments may have put
-         them on the buffer’s undo list (Bug#30931).  */
+      /* This isn’t needed anymore, so don’t wait for GC.  Do not call
+         free_marker on XCAR (data) or XCDR (data), though, since
+         record_marker_adjustments may have put them on the buffer’s
+         undo list (Bug#30931).  Just detach them instead.  */
+      Fset_marker (XCAR (data), Qnil, Qnil);
+      Fset_marker (XCDR (data), Qnil, Qnil);
       free_cons (XCONS (data));
     }
   else
diff --git a/src/insdel.c b/src/insdel.c
index 02e3f41bc9..697395c507 100644
--- a/src/insdel.c
+++ b/src/insdel.c
@@ -2148,10 +2148,13 @@ signal_before_change (ptrdiff_t start_int, ptrdiff_t end_int,
 				   FETCH_START, FETCH_END, Qnil);
     }
 
+  /* Detach the markers now that we're done with them.  Don't directly
+     free them, since the change functions could have caused them to
+     be inserted into the undo list (Bug#30931).  */
   if (! NILP (start_marker))
-    free_marker (start_marker);
+    Fset_marker (start_marker, Qnil, Qnil);
   if (! NILP (end_marker))
-    free_marker (end_marker);
+    Fset_marker (end_marker, Qnil, Qnil);
   RESTORE_VALUE;
 
   unbind_to (count, Qnil);
diff --git a/src/lisp.h b/src/lisp.h
index b931d23bf3..f471b21658 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3812,7 +3812,6 @@ make_uninit_sub_char_table (int depth, int min_char)
 extern Lisp_Object make_save_memory (Lisp_Object *, ptrdiff_t);
 extern void free_save_value (Lisp_Object);
 extern Lisp_Object build_overlay (Lisp_Object, Lisp_Object, Lisp_Object);
-extern void free_marker (Lisp_Object);
 extern void free_cons (struct Lisp_Cons *);
 extern void init_alloc_once (void);
 extern void init_alloc (void);
diff --git a/test/src/editfns-tests.el b/test/src/editfns-tests.el
index 442ad08937..2e20c9dd12 100644
--- a/test/src/editfns-tests.el
+++ b/test/src/editfns-tests.el
@@ -288,4 +288,55 @@ transpose-test-get-byte-positions
                  (buffer-string)
                  "foo bar baz qux"))))))
 
+(ert-deftest delete-region-undo-markers-1 ()
+  "Make sure we don't end up with freed markers reachable from Lisp."
+  ;; https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30931#40
+  (with-temp-buffer
+    (insert "1234567890")
+    (setq buffer-undo-list nil)
+    (narrow-to-region 2 5)
+    ;; `save-restriction' in a narrowed buffer creates two markers
+    ;; representing the current restriction.
+    (save-restriction
+      (widen)
+      ;; Any markers *within* the deleted region are put onto the undo
+      ;; list.
+      (delete-region 1 6))
+    ;; (princ (format "%S" buffer-undo-list) #'external-debugging-output)
+    ;; `buffer-undo-list' is now
+    ;; (("12345" . 1) (#<temp-marker1> . -1) (#<temp-marker2> . 1))
+    ;;
+    ;; If temp-marker1 or temp-marker2 are freed prematurely, calling
+    ;; `type-of' on them will cause Emacs to abort.  Calling
+    ;; `garbage-collect' will also abort if it finds any reachable
+    ;; freed objects.
+    (should (eq (type-of (car (nth 1 buffer-undo-list))) 'marker))
+    (should (eq (type-of (car (nth 2 buffer-undo-list))) 'marker))
+    (garbage-collect)))
+
+(ert-deftest delete-region-undo-markers-2 ()
+  "Make sure we don't end up with freed markers reachable from Lisp."
+  ;; https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30931#55
+  (with-temp-buffer
+    (insert "1234567890")
+    (setq buffer-undo-list nil)
+    ;; signal_before_change creates markers delimiting a change
+    ;; region.
+    (let ((before-change-functions
+           (list (lambda (beg end)
+                   (delete-region (1- beg) (1+ end))))))
+      (delete-region 2 5))
+    ;; (princ (format "%S" buffer-undo-list) #'external-debugging-output)
+    ;; `buffer-undo-list' is now
+    ;; (("678" . 1) ("12345" . 1) (#<marker in no buffer> . -1)
+    ;;  (#<temp-marker1> . -1) (#<temp-marker2> . -4))
+    ;;
+    ;; If temp-marker1 or temp-marker2 are freed prematurely, calling
+    ;; `type-of' on them will cause Emacs to abort.  Calling
+    ;; `garbage-collect' will also abort if it finds any reachable
+    ;; freed objects.
+    (should (eq (type-of (car (nth 3 buffer-undo-list))) 'marker))
+    (should (eq (type-of (car (nth 4 buffer-undo-list))) 'marker))
+    (garbage-collect)))
+
 ;;; editfns-tests.el ends here
-- 
2.11.0

Message #60 received at 30931-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 30931-done <at> debbugs.gnu.org,
 Michał Kondraciuk <k.michal <at> zoho.com>
Subject: Re: bug#30931: abort() due to CHECK_ALLOCATED_AND_LIVE failure during
 GC
Date: Fri, 30 Mar 2018 14:29:07 -0700

[Message part 1 (text/plain, inline)]

Thanks, I installed that along with the attached minor refactoring so 
that the bugfix is in just one place. Closing the bug.

I assume that this fix should not be backported into the emacs-26 
branch, as the bug is in Emacs 25.

[0001-Centralize-Bug-30931-fix.patch (text/x-patch, attachment)]

Message #63 received at 30931-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 30931-done <at> debbugs.gnu.org, npostavs <at> gmail.com, k.michal <at> zoho.com
Subject: Re: bug#30931: abort() due to CHECK_ALLOCATED_AND_LIVE failure during
 GC
Date: Sat, 31 Mar 2018 10:42:09 +0300

> Cc: 30931-done <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>,
>  Michał Kondraciuk <k.michal <at> zoho.com>
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Fri, 30 Mar 2018 14:29:07 -0700
> 
> I assume that this fix should not be backported into the emacs-26 
> branch, as the bug is in Emacs 25.

At least not now.  Maybe after Emacs 26.1 is released.

Thanks.

Message #66 received at 30931 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Michał Kondraciuk <k.michal <at> zoho.com>
Cc: 30931 <at> debbugs.gnu.org
Subject: Re: bug#30931: 27.0.50; Crash in "Automatic GC"
Date: Thu, 19 Apr 2018 19:42:41 -0400

Michał Kondraciuk <k.michal <at> zoho.com> writes:

> Newest Emacs (and 25.3) crashes for me. Unfortunately I can only
> reproduce this using newest versions of Ivy and yasnippet packages.

I have just merged a patch into yasnippet's github repo so that it will
no longer trigger this bug.  This will be in the next yasnippet version
from GNU ELPA as well (0.13), which I plan to release soonish.

https://patch-diff.githubusercontent.com/raw/joaotavora/yasnippet/pull/928.patch

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.