GNU bug report logs - #75356
impossible to benchmark byte-compiled code in a native-comp build

Previous Next

Full log

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <acorallo <at> gnu.org>
To: Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text
 editors" <bug-gnu-emacs <at> gnu.org>
Cc: Pip Cet <pipcet <at> protonmail.com>, 75356 <at> debbugs.gnu.org
Subject: Re: bug#75356: impossible to benchmark byte-compiled code in a
 native-comp build
Date: Mon, 06 Jan 2025 04:24:44 -0500

Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text
editors" <bug-gnu-emacs <at> gnu.org> writes:

> elisp-benchmarks.el unconditionally forces use of native compilation
> if run on an Emacs which is compiled with native-comp enabled.  This
> is a minor issue, but it's still important to benchmark this
> configuration once in a while.

Why is this important?  On a native compiled Emacs all code is supposed
to run native compiled, not only the test, but runtime libraries as
well, further more even Emacs primitives are different in order to
accomodate native code execution.  My opinion is that results of such a
mixed tests would make little no sense in general.

> For example, running
>
> ./src/emacs -Q --batch --load elisp-benchmarks/elisp-benchmarks.el --eval '(progn (setq no-native-compile t) (elisp-benchmarks-run))'

For the resons above I don't think was never supported.  Anyway if you
really want to run bytecode on a native compiled Emacs elisp-benchamerks
let you do this with:

emacs -batch -l ./elisp-benchmarks.el --eval '(progn (setq elb-speed -1) (elisp-benchmarks-run))'

Alternativelly if you are interested in working with a function
granularity you can set manually function speeds to -1.

> produces error messages and an "empty" results table:
>
>   | test  | non-gc (s) | gc (s) | gcs | total (s) | err (s) |
>   |-------+------------+--------+-----+-----------+---------|
>   |-------+------------+--------+-----+-----------+---------|
>   | total |       0.00 |   0.00 |   0 |      0.00 |    0.00 |
>
> In particular, I believe it is legitimate to use a native-comp build
> on a system which no longer supports compiling additional code.
>
> In general, the compilation logic of elisp-benchmarks.el is fragile
> and will lead to unreliabe test results,

Why do you think so?

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.