GNU bug report logs - #75357
[scratch/elisp-benchmarks] elb-scroll window size assumption breaks on repeated runs

Previous Next

To reply to this bug, email your comments to 75357 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> protonmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [scratch/elisp-benchmarks] elb-scroll window size assumption breaks
 on repeated runs
Date: Sat, 04 Jan 2025 16:36:20 +0000

On scratch/elisp-benchmarks, the following emacs -Q invocation:

./src/emacs --batch -Q -l ./elisp-benchmarks/elisp-benchmarks.el --eval '(progn (elisp-benchmarks-run "scroll" t 1) (elisp-benchmarks-run "scroll" t 1))'

produces a fatal Lisp error, attempting to invoke the debugger in a
non-interactive session:

Loading /home/pip/emacs-elisp-benchmarks/elisp-benchmarks/benchmarks/elb-smie...
Window size: 19 x 80
Debugger entered--Lisp error: (error "Window size not as stipulated by the benchmark")
  signal(error ("Window size not as stipulated by the benchmark"))
  error("Window size not as stipulated by the benchmark")
  elb-scroll-entry()

There are several bugs here:

1. The test itself incorrectly assumes that no secondary window is
displayed which would affect the ability of the test to resize its
window to the specified dimensions.

2. elisp-benchmarks does not handle Lisp errors thrown by a test
gracefully if debug-on-error is initially t, since it uses
with-demoted-errors without let-binding debug-on-error.

3. elisp-benchmarks destructively uses setq to modify the session's
debug-on-error value, leaving it at a different value on exit.

(1) can be fixed by a call to (delete-other-windows) in noninteractive
mode, but cannot be fixed that way in interactive mode.

(2) can be fixed by let-binding debug-on-error in non-interactive mode,
or using condition-case properly.

(3) can and should be fixed in the obvious fashion.

(2) and (3) can also be fixed by moving to ERT, which has an established
API for handling both the interactive and non-interactive scenario.

In all cases, we need a way to indicate for a benchmark whether it
modifies Emacs state destructively.  We need to handle debug-on-quit
differently, as ERT does.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <acorallo <at> gnu.org>
To: Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text
 editors" <bug-gnu-emacs <at> gnu.org>
Cc: Pip Cet <pipcet <at> protonmail.com>, 75357 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#75357: [scratch/elisp-benchmarks] elb-scroll window size
 assumption breaks on repeated runs
Date: Mon, 06 Jan 2025 04:35:59 -0500

Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text
editors" <bug-gnu-emacs <at> gnu.org> writes:

> On scratch/elisp-benchmarks, the following emacs -Q invocation:
>
> ./src/emacs --batch -Q -l ./elisp-benchmarks/elisp-benchmarks.el --eval '(progn (elisp-benchmarks-run "scroll" t 1) (elisp-benchmarks-run "scroll" t 1))'
>
> produces a fatal Lisp error, attempting to invoke the debugger in a
> non-interactive session:
>
> Loading /home/pip/emacs-elisp-benchmarks/elisp-benchmarks/benchmarks/elb-smie...
> Window size: 19 x 80
> Debugger entered--Lisp error: (error "Window size not as stipulated by the benchmark")
>   signal(error ("Window size not as stipulated by the benchmark"))
>   error("Window size not as stipulated by the benchmark")
>   elb-scroll-entry()

I'm not sure this is a bug as we want always to run the benchmark from a
known state, anyway adding Stefan as he's the author of the benchmark.

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Andrea Corallo <acorallo <at> gnu.org>
Cc: "Pip Cet via Bug reports for GNU Emacs, the Swiss army knife of text
 editors" <bug-gnu-emacs <at> gnu.org>, 75357 <at> debbugs.gnu.org,
 Pip Cet <pipcet <at> protonmail.com>
Subject: Re: bug#75357: [scratch/elisp-benchmarks] elb-scroll window size
 assumption breaks on repeated runs
Date: Mon, 06 Jan 2025 09:24:13 -0500

>> On scratch/elisp-benchmarks, the following emacs -Q invocation:
>>
>> ./src/emacs --batch -Q -l ./elisp-benchmarks/elisp-benchmarks.el --eval '(progn (elisp-benchmarks-run "scroll" t 1) (elisp-benchmarks-run "scroll" t 1))'
>>
>> produces a fatal Lisp error, attempting to invoke the debugger in a
>> non-interactive session:
>>
>> Loading /home/pip/emacs-elisp-benchmarks/elisp-benchmarks/benchmarks/elb-smie...
>> Window size: 19 x 80
>> Debugger entered--Lisp error: (error "Window size not as stipulated by the benchmark")
>>   signal(error ("Window size not as stipulated by the benchmark"))
>>   error("Window size not as stipulated by the benchmark")
>>   elb-scroll-entry()
>
> I'm not sure this is a bug as we want always to run the benchmark from a
> known state, anyway adding Stefan as he's the author of the benchmark.

Sounds like something has changed in Emacs's C code, or maybe it's
something in some system-depend code?
`elb-scroll` cares about the window size because it's a benchmark that
is influenced by (and thus partly measures) the redisplay:

    (unless (and noninteractive (not (boundp 'redisplay-skip-initial-frame)))
      (defun elb-scroll-entry ()
        ;; FIXME: This relies on `elb-smie.el' being compiled already which is
        ;; not necessarily the case if we're only running some of the benchmarks.
        (load (expand-file-name "elb-smie" elb-bench-directory) nil 'nomessage)
        (setq redisplay-skip-initial-frame nil)
        (with-temp-buffer
          (rename-buffer (generate-new-buffer-name "elb-scroll"))
          (switch-to-buffer (current-buffer))
          (insert-file-contents (expand-file-name
                                 "../resources/xmenu.c" elb-bench-directory))
          (redisplay 'force) ;; Refresh the window dimensions.
          (enlarge-window (- 23 (window-height)))
          (enlarge-window (- 80 (window-width)) 'horiz)
          (redisplay 'force) ;; Refresh the window dimensions.
          (unless (and (equal 23 (window-height))
                       (equal 80 (window-width)))
            (error "Window size %S x %S not as stipulated by the benchmark"
                    (window-height) (window-width)))
          (dotimes (_ 10)
            (elb-smie-mode)
            (goto-char (point-min))
            (condition-case nil
                (while t (scroll-up nil) (redisplay 'force))
              (end-of-buffer nil))))))

Try and figure out why window size is 19x80 after

          (enlarge-window (- 23 (window-height)))
          (enlarge-window (- 80 (window-width)) 'horiz)

- Stefan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.