GNU bug report logs - #75356
impossible to benchmark byte-compiled code in a native-comp build

Previous Next

Full log

View this message in rfc822 format

From: Pip Cet <pipcet <at> protonmail.com>
To: 75356 <at> debbugs.gnu.org
Subject: bug#75356: impossible to benchmark byte-compiled code in a native-comp build
Date: Sat, 04 Jan 2025 16:35:01 +0000

elisp-benchmarks.el unconditionally forces use of native compilation
if run on an Emacs which is compiled with native-comp enabled.  This
is a minor issue, but it's still important to benchmark this
configuration once in a while.

For example, running

./src/emacs -Q --batch --load elisp-benchmarks/elisp-benchmarks.el --eval '(progn (setq no-native-compile t) (elisp-benchmarks-run))'

produces error messages and an "empty" results table:

  | test  | non-gc (s) | gc (s) | gcs | total (s) | err (s) |
  |-------+------------+--------+-----+-----------+---------|
  |-------+------------+--------+-----+-----------+---------|
  | total |       0.00 |   0.00 |   0 |      0.00 |    0.00 |

In particular, I believe it is legitimate to use a native-comp build
on a system which no longer supports compiling additional code.

In general, the compilation logic of elisp-benchmarks.el is fragile
and will lead to unreliabe test results, which is the worst possible
outcome for a benchmark (in some cases, we simply fail to produce
data, which is better but still not good).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.