GNU bug report logs - #75357
[scratch/elisp-benchmarks] elb-scroll window size assumption breaks on repeated runs

Previous Next

Full log

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> protonmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [scratch/elisp-benchmarks] elb-scroll window size assumption breaks
 on repeated runs
Date: Sat, 04 Jan 2025 16:36:20 +0000

On scratch/elisp-benchmarks, the following emacs -Q invocation:

./src/emacs --batch -Q -l ./elisp-benchmarks/elisp-benchmarks.el --eval '(progn (elisp-benchmarks-run "scroll" t 1) (elisp-benchmarks-run "scroll" t 1))'

produces a fatal Lisp error, attempting to invoke the debugger in a
non-interactive session:

Loading /home/pip/emacs-elisp-benchmarks/elisp-benchmarks/benchmarks/elb-smie...
Window size: 19 x 80
Debugger entered--Lisp error: (error "Window size not as stipulated by the benchmark")
  signal(error ("Window size not as stipulated by the benchmark"))
  error("Window size not as stipulated by the benchmark")
  elb-scroll-entry()

There are several bugs here:

1. The test itself incorrectly assumes that no secondary window is
displayed which would affect the ability of the test to resize its
window to the specified dimensions.

2. elisp-benchmarks does not handle Lisp errors thrown by a test
gracefully if debug-on-error is initially t, since it uses
with-demoted-errors without let-binding debug-on-error.

3. elisp-benchmarks destructively uses setq to modify the session's
debug-on-error value, leaving it at a different value on exit.

(1) can be fixed by a call to (delete-other-windows) in noninteractive
mode, but cannot be fixed that way in interactive mode.

(2) can be fixed by let-binding debug-on-error in non-interactive mode,
or using condition-case properly.

(3) can and should be fixed in the obvious fashion.

(2) and (3) can also be fixed by moving to ERT, which has an established
API for handling both the interactive and non-interactive scenario.

In all cases, we need a way to indicate for a benchmark whether it
modifies Emacs state destructively.  We need to handle debug-on-quit
differently, as ERT does.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.